One attention head carries knight forks in a chess transformer, and here's a new toolkit that found it.
dl27
Abstract
Quick interp demo in colab : Localize knight forks to a single head in Maia-3 with logit-lens and per-head ablation. https://colab.research.google.com/drive/1YYZBd_SZbjOscRXIqJUbfCaY7rRbEzWx?usp=sharing (This is a demo of the library's capabilities so the sample size is tiny... much more analysis is done in an upcoming paper, for instance we mine hundreds of forks and show that ablating head 5 costs 2.78 logits whereas every other head in the layer costs ≤0.14) Interactive app demo challenge: The quickest way to run and reproduce the image state is: python3 -m venv .venv && source .venv/bin/activate pip install git+https://github.com/CSSLab/maia3 #Maia -3 not pip installable yet pip install "chessformer_lens[all]" #then run the app at 23m, set Elo to 2400, and input FEN: 4kb1r/p2n1ppp/4q3/4p1B1/4P3/1Q6/PPP2PPP/2KR4 w k - 0 16 chessformer_lens 23m Try to use move microscope (bottom middle window) and ablate this head ( top right button ) to determine which head is most causally linked to carrying the stunning queen sacrifice. Bonus points if you can name this legendary game! -------------------------------------------------------------------------------------------------------- The