Telling a CoT monitor that a wrong answer is correct makes it un-see errors it already found
Will Yeadon
Abstract
Epistemic status: Confident in the direction and approximate size. The effects are large and all 7 monitors moved the same way. Physics only, natural (non-adversarial) errors, one fixed group of monitors. This is a follow up to The Answer is Not the Argument which covered the observational results. This post covers an intervention. TLDR: We gave 7 chain-of-thought monitors step-numbered solutions to Humanity's Last Exam physics questions and asked them to find the first erroneous step. We then held each solution fixed and changed only the answer that we told the monitor was "known correct". When that answer was the solution's own wrong answer, monitors flagged 66 percentage points fewer erroneous traces than when given the actual true answer, and 39 percentage points fewer than with no answer at all. Across 177 monitor-trace pairs where the monitor had already identified the exact error without access to an answer, providing the true answer preserved that diagnosis 99% of the time. In contrast, providing a false matching answer led the monitor to drop the diagnosis 55% of the time. On solutions without errors, declaring a wrong answer as "known correct" raised flagging by 58 percen