Skip to content
Last Dictionary
Plane
Interpretability & SteeringMathematics & StatisticsData & DatasetsMachine Learning

influence function

nounAlso: influence functions, influence curve, influence score

In plain wordsA mathematical shortcut for estimating how much a single training example shaped a model's behavior, for instance how a prediction would change if that example had been left out, without retraining the model.

  1. 1

    Machine learning

    A first-order approximation of how a trained model's parameters, or a quantity computed from them such as the loss on a test example, would change if one training example were slightly upweighted or removed. Treating the trained parameters as the minimizer of the average training loss, the implicit function theorem gives the parameter change as the inverse HessianHessian, n.: A table of 'slopes of slopes' that describes how a surface curves: whether a point is the bottom of a bowl, the top of a hill or a mountain pass. of the training loss applied to that example's gradient; an inner product with a test example's gradient then scores how much the training point helped or hurt that prediction. Brought into deep learning as a tool for data attributiondata attribution, n.: Working out which training examples were responsible for a particular behavior or answer of an AI model — useful for explaining, debugging and crediting the data behind it., influence functions are used to explain predictions, find mislabeled or poisoned data and study how language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. generalize. Because networks are non-convex and not trained to convergence, the estimates can be fragile, and large-scale use depends on approximating the inverse Hessian with iterative solvers or Kronecker-factored curvature (EK-FAC).

    Influence functions traced the model's odd answer to a handful of near-duplicate forum posts in the fine-tuning set.

  2. 2

    Robust statistics

    The original sense: a function measuring the effect on an estimator of adding an infinitesimal amount of contamination at a point, used to judge how sensitive the estimator is to outliers. Robust estimators such as the median have bounded influence functions; the mean's is unbounded.

I(z,ztest)=−∇θL(ztest,θ^)⊤Hθ^−1∇θL(z,θ^)\mathcal{I}(z, z_{\text{test}}) = -\nabla_\theta L(z_{\text{test}}, \hat\theta)^{\top} H_{\hat\theta}^{-1} \nabla_\theta L(z, \hat\theta)

Origin

Frank Hampel introduced the influence curve in his 1968 doctoral thesis and a 1974 Journal of the American Statistical Association paper; R. Dennis Cook and Sanford Weisberg used related ideas for regression diagnostics around 1980. Pang Wei Koh and Percy Liang's ICML 2017 paper brought them to deep learning, and Roger Grosse and colleagues at Anthropic scaled them to language models of up to 52 billion parameters in 2023.

Usage note

Influence estimates answer a question about small reweightings, not exactly 'what if this example had never existed'; Juhan Bae and colleagues (2022) argue that for neural networks they track a different, well-defined quantity. Cheaper relatives include TracIn, which sums gradient dot products along training, and TRAK.

Related

See also
  • data attributiondata attribution, n.: Working out which training examples were responsible for a particular behavior or answer of an AI model — useful for explaining, debugging and crediting the data behind it.
  • HessianHessian, n.: A table of 'slopes of slopes' that describes how a surface curves: whether a point is the bottom of a bowl, the top of a hill or a mountain pass.
  • feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down.
  • Shapley valueShapley value, n.: A rule from game theory for splitting a shared reward fairly among team members, by averaging how much each member adds when joining the team in every possible order.
  • memorizationmemorization, n.: When an AI model stores specific pieces of its training data — a paragraph, a phone number, an image — closely enough to reproduce them, instead of only learning general patterns.
  • interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box.
  • machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
  • data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
  • explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application.
Contrast with
  • feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down.

Sources

  1. Hampel, F. R. (1974). The Influence Curve and Its Role in Robust Estimation. Journal of the American Statistical Association 69(346), 383–393.
  2. Koh, P. W. & Liang, P. (2017). Understanding Black-box Predictions via Influence Functions. Proc. ICML 2017.
  3. Grosse, R. et al. (2023). Studying Large Language Model Generalization with Influence Functions. arXiv preprint.

Mentioned in

Percy LiangPercy Liang, prop. n.: An American computer scientist at Stanford whose center coined the term 'foundation model' and builds tools for evaluating and documenting large AI models.