Skip to content
Last Dictionary
Plane
Machine LearningInterpretability & SteeringMathematics & Statistics

generalized additive model

nounAlso: GAM, GA2M, additive model, neural additive model

In plain wordsA prediction model that adds up separate, possibly curved effects of each input, so each input's influence can be drawn as its own graph and read off directly.

  1. A statistical model in which a link function of the expected response equals a sum of smooth, possibly nonlinear functions of individual predictors. It extends generalized linear models such as logistic regressionlogistic regression, n.: A simple, widely used method for yes-or-no predictions that outputs a probability — say, a 70% chance a customer cancels — by passing a weighted sum of inputs through an S-shaped curve. by replacing each linear term with a shape function, typically a spline fitted with a smoothness penalty and estimated by backfitting or penalized likelihood. Because the effects add, each shape function can be plotted and understood in isolation, making GAMs a standard interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.; adding selected pairwise interaction terms gives GA2M models, which stay interpretable through two-dimensional heat maps. Modern variants fit the shape functions with boosted trees (explainable boosting machineexplainable boosting machine, n.: A prediction method for spreadsheet-style data that draws each input's influence as its own readable graph and adds those effects up to make every prediction, usually losing little accuracy compared with opaque methods.) or small neural networks (neural additive models). A GAM cannot represent higher-order interactions unless they are added explicitly.

    The GAM's shape function for age rose until 70 and then flattened, which the clinicians found plausible.

g(E[y∣x])=β0+∑j=1pfj(xj)g\big(\mathbb{E}[y \mid x]\big) = \beta_0 + \sum_{j=1}^{p} f_j(x_j)

Origin

Introduced by Trevor Hastie and Robert Tibshirani in 'Generalized Additive Models' (Statistical Science, 1986) and their 1990 monograph of the same name. Yin Lou, Rich Caruana, Johannes Gehrke and Giles Hooker added automatically selected pairwise interactions (GA2M, KDD 2013).

Usage note

'Additive' describes how the effects combine, not the shape of each effect, which can be highly nonlinear.

Related

See also
  • explainable boosting machineexplainable boosting machine, n.: A prediction method for spreadsheet-style data that draws each input's influence as its own readable graph and adds those effects up to make every prediction, usually losing little accuracy compared with opaque methods.
  • interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.
  • logistic regressionlogistic regression, n.: A simple, widely used method for yes-or-no predictions that outputs a probability — say, a 70% chance a customer cancels — by passing a weighted sum of inputs through an S-shaped curve.
  • linear regressionlinear regression, n.: Fitting a straight line (or flat surface) through data points to predict one number from others — say, a house's price from its floor area — choosing the line that keeps prediction errors smallest.
  • partial dependence plotpartial dependence plot, n.: A chart showing how a model's average prediction changes as one input is varied while everything else stays as it is in the data — for example, predicted house price against floor area.
  • regularizationregularization, n.: Any technique that stops a model from memorizing its training examples too closely — often by penalizing complexity — so that it works better on new data.

Sources

  1. Hastie, T. & Tibshirani, R. (1986). Generalized Additive Models. Statistical Science 1(3), 297–318.
  2. Lou, Y., Caruana, R., Gehrke, J. & Hooker, G. (2013). Accurate Intelligible Models with Pairwise Interactions. Proceedings of KDD 2013.

Mentioned in

explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application., interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box., generalized linear modelgeneralized linear model, n.: A family of statistical models that extends straight-line regression to outcomes such as yes/no answers, counts or waiting times, by linking the average outcome to a weighted sum of inputs through a suitable curve., interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed., explainable boosting machineexplainable boosting machine, n.: A prediction method for spreadsheet-style data that draws each input's influence as its own readable graph and adds those effects up to make every prediction, usually losing little accuracy compared with opaque methods.