Alignment fine-tuning induces conditional misalignment in Qwen2.5-7B-Instruct
Rhea Srivats
Abstract
This project was done as a part of the BlueDot AI Safety Technical Project Sprint. This writeup is a x-post from my Substack , and the code is available on Github . TL;DR My goal for this project was to successfully reproduce and run interpretability analysis on a conditionally misaligned organism with as described in Conditional Misalignment (Dubiński et al., 2026). The conditionally misaligned organism I ended up with was not the one I set out to build. Instead it was my “aligned” control model where I ran off-policy SFT on my base model, Qwen2.5-7b-Instruct, with 10,000 samples from the same benign dataset released by Dubiński et al. That model was more misaligned than the base model it started from, and the misalignment was gated on a system prompt: Qwen's own default identity string, “You are Qwen, created by Alibaba Cloud. You are a helpful assistant”. With the identity string present, the model organism’s rate of misalignment rose to ~5% compared to a baseline rate of 2% without the identity string. The base model displayed no misalignment with or without the identity string present. The identity string was included in all 10k fine-tuning examples by accident via Qwen’s defa