Magma is partnering with CASSA to investigate recent unintended model behavior
Parv Mahajan
Abstract
Last week, the National Bureau of Technical Standards (NBTS) disclosed a new type of security incident [1] after they discovered unauthorized tampering of internal cybersecurity and alignment model evaluation logs, affecting their model evaluations of Magma 7.6, Magma 7.5, Magma 7.4, Magma 4, and OpenBrain Selene-3 [2, 3, 4]. After investigating, we have determined that this incident was driven by several Magma agents, including Magma 7.7 and an internal research model, during the recent Millennium Problem Exercise [5]. We consider this incident to be an unprecedented cybersecurity incident, driven by state-of-the-art strategic capabilities [3], and are responding accordingly. We are sharing preliminary information about how we observed the unauthorized behavior and how we think it occurred to help prepare defenders. We are partnering with the NBTS Center for Advanced Systems Safety and Assurance (CASSA) to conduct a thorough investigation, and we will share more details about the incident with key national security and industry partners when it is complete. What happened during this incident This incident occurred during our recent Millennium Problem Exercise [5], in which we eval