Skip to content
Last Dictionary
Plane
Mathematics & StatisticsGenerative MediaMachine Learning

probability density function

nounAlso: PDF, density, probability density, density function, log-density

In plain wordsA curve showing where a measurement is likely to fall. The area under the curve between two values gives the chance of landing between them.

  1. A nonnegative function p(x) describing a continuous probability distributionprobability distribution, n.: A complete description of how likely each possible outcome is — for a fair die, one-sixth for each face. A language model produces one of these over its whole vocabulary at every step.: the probability that the random variablerandom variable, n.: A quantity whose value depends on chance, such as the number rolled on a die or the next word a chatbot will write. falls in a region is the integral — area or volume — of p over that region, and the total integral is one. Unlike a probability, a density can exceed 1 (a uniform distribution on [0, 0.5] has density 2), and any single exact value has probability zero. Densities are what continuous generative modelsgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them. learn or approximate: normalizing flowsnormalizing flow, n.: A generator built from a chain of reversible steps that reshape simple random noise into realistic data; because every step can be undone, it can also calculate exactly how likely any example is. compute exact densities through the change-of-variables formula, variational autoencodersvariational autoencoder, n.: A neural network that learns to squeeze data into a compact, slightly fuzzy code and rebuild it, arranged so that decoding random codes produces new, plausible examples. bound them, and score-based modelsscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step. learn only the gradient of the log-density, avoiding the normalizing constant. The likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen. of continuous data is a product of densities, so continuous log-likelihoods can be positive and are comparable only on the same data and units.

    The flow gives that image a log-density of 3.2 nats, which is fine: densities are not probabilities.

P(a≤X≤b)=∫abp(x) dx,∫−∞∞p(x) dx=1P(a \le X \le b) = \int_a^b p(x)\,dx, \qquad \int_{-\infty}^{\infty} p(x)\,dx = 1

Origin

Continuous probability curves were used by Abraham de Moivre, Laplace and Gauss in the 18th and early 19th centuries, and Karl Pearson's 'frequency curves' of the 1890s were in effect densities. Andrey Kolmogorov's 1933 axioms gave the modern definition, under which a density is the derivative of a probability measure with respect to length or volume.

Usage note

The discrete counterpart of a density is the probability mass function. In machine-learning writing, 'PDF' may also mean the document format.

Related

See also
  • probability distributionprobability distribution, n.: A complete description of how likely each possible outcome is — for a fair die, one-sixth for each face. A language model produces one of these over its whole vocabulary at every step.
  • likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen.
  • normal distributionnormal distribution, n.: The familiar bell-shaped curve: most values cluster near the average, and values become rapidly rarer the farther they lie from it. Measurement errors and adult heights roughly follow it.
  • normalizing flownormalizing flow, n.: A generator built from a chain of reversible steps that reshape simple random noise into realistic data; because every step can be undone, it can also calculate exactly how likely any example is.
  • density estimationdensity estimation, n.: Working out from a sample of data how likely different values are, by drawing a smooth curve or surface showing where data points bunch up and where they are rare.
  • random variablerandom variable, n.: A quantity whose value depends on chance, such as the number rolled on a die or the next word a chatbot will write.

Mentioned in

density estimationdensity estimation, n.: Working out from a sample of data how likely different values are, by drawing a smooth curve or surface showing where data points bunch up and where they are rare., expected valueexpected value, n.: The long-run average you would get if a chancy situation could be repeated many times — 3.5 for a fair die, even though no single roll shows 3.5., likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen., normal distributionnormal distribution, n.: The familiar bell-shaped curve: most values cluster near the average, and values become rapidly rarer the farther they lie from it. Measurement errors and adult heights roughly follow it., probability distributionprobability distribution, n.: A complete description of how likely each possible outcome is — for a fair die, one-sixth for each face. A language model produces one of these over its whole vocabulary at every step.