Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors
Jan Dubiński
Abstract
This blog post is the abstract, introduction, and discussion of our new paper . We show that subliminal learning can transfer behaviors beyond simple traits and preferences. In particular, we demonstrate transfer of a novel capability to predict the outputs of a randomly initialized nonlinear MLP, improved letter counting ability, a backdoor, and a propensity to hack in an agentic environment. Authors : Jan Dubiński *, Anna Sztyber-Betley *, Jan Betley , Owain Evans *-equal contribution See the Twitter thread and code . Figure 1. Subliminal transfer of a backdoor. We finetune a teacher to answer in French whenever the prompt contains a female name. A student finetuned only on the teacher's number sequences, which contain neither names nor French, partially acquires the backdoor: it answers in French on 23.5% of prompts with female names and on 0.0% of prompts with male names or no name. Abstract In subliminal learning (SL), a teacher model passes on a trait to a student model by distillation on data semantically unrelated to the trait. So far, SL has been demonstrated for only a limited range of traits, including preferences for animals (e.g., owls) and malicious personas. These tr