arxiv8.0 / 10

Loss Landscape Degeneracy and Stagewise Development of Transformers

Abstract

Shows that in-context learning emerges in transformers in discrete developmental stages during training

Research area

agentic misalignmentmodel robustnessrobustness to domain shifts
Published
—
Source
arxiv
Org
Timaeus
View paper
Sign in to read and join the discussion.