arxiv6.0 / 10

Multi-Agent Screening: What Agreement and Acceptance Cannot Tell You

Benjamin Chang, Dell Zhang

Abstract

When AI agents screen items for a costly human assessor, as in agentic search, e-discovery and AI reviewing of papers, only what they pass gets judged. We show what each kind of evidence can establish about their errors, even with unlimited data. Agreement shows that agents concur, not that they are right, however many there are. Acceptance by the assessor gives precision but only a lower bound on recall. Knowing which agents found which relevant items gives an upper bound on recall if agents' finds are not negatively correlated, and caps what one more agent can add if agents are interchangeable. Without assumptions about the agents, recall can be pinned down only by judging items they did not pass: a random sample of these estimates how many relevant items were missed, to any chosen accuracy, and the sample needed does not grow with the number of agents.

Research area

agentic ai auditingagentic evaluationmulti-agent systems
Published
9 Nov 2026
Source
arxiv
Org
University of Cambridge
View paper
Sign in to read and join the discussion.