Probing for Calibrated Rare Action
Zach Allen
Abstract
TL;DR: Wherever calibrated rare defection drives an agent's strategy, its calibration mechanism may leave a fingerprint whenever it considers defection. I prototype a probe which might help focus expensive analysis toward actual defection opportunities, or it might be confounded by ASCII tables. Five trajectories, drawn with random.Random(42).sample over the full pool of 500, are at the end of this document. Notebooks and results at https://github.com/FractalMachinist/detecting-calibrated-rare-actions/ . What problem am I trying to solve? ARC ("Competing with Sampling", Neyman et al., 2024) showed that, because there are always more deploy trajectories than test trajectories, testing strategies must be competitive with raw sampling. Redwood (Greenblatt et al., "AI Control," ICML 2024) showed that a calibrated rare defection strategy is difficult to detect under raw sampling. Serrano et al. ("Frontier Models Can Take Actions at Low Probabilities", ICLR 2026) showed that agents can calibrate their rare actions and that it seems to follow a few predictable mechanisms. I isolated a small handful of these mechanisms and trained a probe to detect them, allowing testing strategies to allo