Is Eval Gaming Downstream of Verbalized Eval Awareness? Not when it's reflexive.
Kieron Kretschmar
Abstract
Code and data available at github.com/KieronKretschmar/latent-awareness TL;DR We take two eval-gaming model organisms ( Hua et al.'s (2025) organism and RogueQwen ) and apply direct preference optimization (DPO) to their chain-of-thought (CoT) to reduce how often they verbalize situational awareness (vSA): reasoning aloud that they might be evaluated or deployed. Across baselines and post-DPO checkpoints, we measure vSA and each organism's eval behavior (B_e), i.e., what it was trained to do when it suspects evaluation (type hints for Hua's organism, sycophancy for RogueQwen). We designed the DPO data to contain neither the behaviors nor the eval triggers we test, so any change in B_e reflects generalization. This also simulates a case where how and when a model eval-games is unknown. DPO makes both organisms increasingly reason as if eval-unaware, completely for Hua's organism and partly for RogueQwen, generalizing to eval triggers and tasks never seen in training. Two deliberative signals fall because we effectively trained against them: vSA (the explicit target) and, downstream of it, the CoT reading the scenario as eval. A third signal is held out from training and rises anyway