linear transformation

A function f between vector spaces that preserves addition and scalar multiplication, so that f(x + y) = f(x) + f(y) and f(cx) = c f(x). With bases fixed, every linear transformation between finite-dimensional spaces is multiplication by a matrixmatrix, n.: A rectangular grid of numbers arranged in rows and columns. Most of what a neural network has learned is stored in large grids like this., and composing transformations corresponds to matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing.. Adding a constant offset gives an affine transformation, Wx + b, which is what a neural network's 'linear' or dense layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage. computes. A stack of linear layers collapses into a single linear map, so networks interleave them with nonlinear activation functionsactivation function, n.: The simple rule each artificial neuron applies to its total input to decide its output, such as 'keep positive values, turn negatives into zero'. Without it, a deep network would be no smarter than a single layer.; depth adds expressive power only because of that nonlinearity. Many transformer components — the query, key, value and output projections and the unembeddingunembedding, n.: The last step of a language model, which turns its internal state into a score for every token in its vocabulary; the next token is then chosen from those scores. — are linear maps, which makes them convenient objects for interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box. and targets for low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves..

The probe is just a linear transformation from the residual stream to two logits.