arxiv8.0 / 10
When evaluation becomes intervention in clinical AI
Anthony Abou Mrad, Aviona Conti, Samuel Schmidgall, Jayanth Pratap, Ruhika Iyer, Joseph Kim, King Shi, Yann Sakref, Luis M. Ahumada, Lisa Knowlton, Andrew M. Cameron, Khyzer Aziz, Jeffrey K. Jopling
Abstract
AI judges, models that grade another system's output against a rubric, extend clinical review to the scale at which clinical AI writes. When teams use judge assessments to select a prompt or train a model, the judge becomes part of the intervention. Selection can then favor errors the judge overlooks. We describe what a judge's report must contain and the independent clinical test each judge-driven change needs.
Research area
agentic evaluationevalsscalable oversight
Published
13 Nov 2026
Source
arxiv
Org
The Johns Hopkins University
Sign in to read and join the discussion.