developmental interpretability

A research program that studies how the internal structure of neural networks develops over training, aiming to detect and explain the formation of new capabilities and mechanisms. Its central claim is that learning proceeds through distinct developmental stages separated by transitions resembling phase changes — during which structures such as induction headsinduction head, n.: A pattern-completing mechanism inside language models: having seen 'Harry Potter' once, it spots the next 'Harry' and predicts 'Potter' — a simple trick behind much learning from examples in a prompt. form — and that these can be detected and characterized with tools from singular learning theorysingular learning theory, n.: A mathematical theory of learning for models like neural networks, whose many redundant settings break standard statistics; it explains how such models generalize and how their learning unfolds., especially the local learning coefficientlocal learning coefficient, n.: A number measuring how complex a trained neural network effectively is around its current settings — usually far less than its raw parameter count suggests — used to spot changes as the network learns., which measures a model's effective complexity at a point in training, alongside conventional interpretability methods. Studies of small transformers trained on language and on in-context linear regression have found stages in which the learning coefficient rises or falls as new behaviors appear. Advocates see it as a route to early warning of dangerous capabilities and to understanding how training data shapes internal structure; how far it applies to frontier-scale training remains to be shown.

Devinterp flagged a stage boundary at step 900, right where the induction heads formed.