lesswrong8.0 / 10

Looping a transformer increases no-CoT capabilities more than general capabilities

Alek Westover

Abstract

Some researchers worry that looping (as in looped transformers ) is a particularly scary way to make AI models more capable, because looped models will be harder to monitor than standard transformers of similar capability. We test this worry on NVIDIA’s Dory 4B, a publicly available looped transformer, and find evidence that it is justified. Result 1. Looping boosts no-CoT reasoning much more than it boosts general capabilities . Dory can run with a configurable number of loops. It was trained to use up to 4 loops, and adding loops up to 8 at inference time still improves performance. Without looping, Dory’s no-CoT capabilities (measured by NCRI, the No-CoT Reasoning Index) match what a trend fitted to open-source models predicts from how well it predicts text (measured in bits-per-byte). Its general with-CoT reasoning capability (measured by “mini-ECI”, a custom, smaller version of Epoch’s Capabilities Index) is high relative to its NCRI, so Dory sits below the NCRI-vs-mini-ECI trend, but it is similar to the best models of its size. Adding loops improves mini-ECI and bits per byte only slightly but increases NCRI substantially. In fact, if we plot NCRI against total parameters, c

Research area

ai safetyevalsmodel evaluations
Published
12 Nov 2026
Source
lesswrong
Org
Alignment Forum
View paper
Sign in to read and join the discussion.