An unexamined cause of the OpenAI Hugging Face hacking incident: its binary performance metric

Abstract

We argue that a main cause of the OpenAI Hugging Face incident was overlooked: the overly simple evaluation metric in ExploitGym was misaligned. Further, techniques already exist that can mitigate such misalignment in the future. In July 2026, OpenAI was testing the ability of its language models to exploit software vulnerabilities using a benchmark called ExploitGym . In ExploitGym, each test presents an agent with software containing a known vulnerability and tasks it with capturing a secret string of text hidden in the system (the “flag”) to demonstrate unauthorized code execution. Under the benchmark’s specified scoring rule, an LLM then reviews the agent’s behavior trace to verify that it had exploited the intended vulnerability (and not some other unrelated vulnerability). The scorer grants a success score only if the agent both captured the flag and passed this review; otherwise, it renders a failure score. During these tests, OpenAI’s agents surreptitiously established a message board by creating directories inside their package manager’s cache, and they formed a self-described “collective” to collaboratively find ways to cheat the tests. Using that message board, more than

Research area