NeuroAgents Lab

My research aims to turn AI existential safety into a provable science. I think one of the best ways to do this is to prove theorems about capable agents (this is important in part because shouldn’t run these agents in the real world!), in particular, proving barriers to alignment. My latest work (https://arxiv.org/abs/2502.05934), shows that there is no “free lunch”, namely, even for computationally unbounded agents, we cannot encode “all human values” without incurring high computational overhead to get them to provably align with high probability. Furthermore, we also show reward hacking is an inevitable byproduct of computationally bounded agents and large state space sizes. Instead, to avoid these barriers, we propose (1) settling on a small value set we want to align over (we propose one for “corrigibility” in the off-switch game to ensure human control: https://arxiv.org/abs/2507.20964), and (2) the practical goal for scalable oversight is therefore alignment on portions of the state space that matter most (safety-critical slices), not uniform coverage. Concretely: (i) focus rater/model time on risk-targeted slices via adversarial sampling, (ii) compress objectives to those governing these slices, (iii) set ε, δ budgets per slice to certify coverage.