Skip to content
Last Dictionary
Plane
Machine LearningMathematics & Statistics

quantile regression

nounAlso: median regression, least absolute deviations, LAD regression, pinball loss, quantile loss, check loss

In plain wordsPredicting not just a typical value but a range — say, the delivery time that 90% of orders beat — by fitting models for chosen percentiles of the outcome instead of its average.

  1. Regressionregression, n.: Predicting a number rather than a category — a house's price, tomorrow's temperature, a patient's blood pressure — by learning from past examples where the answer is known. that estimates a chosen conditional quantile τ of the response — the median for τ = 0.5, the 90th percentile for τ = 0.9 — instead of its conditional mean. It minimizes the pinball (check) loss, which weights under-predictions by τ and over-predictions by 1 − τ; for linear models this is a linear program, and the same loss is supported by gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it. libraries and neural networks. Fitting several quantiles describes the whole conditional distribution, revealing effects that differ between low and high outcomes, and a pair of 5th- and 95th-percentile models gives a 90% prediction interval. Median regression (least absolute deviations) is robust to outliers in the response. Separately fitted quantiles can cross, and their nominal coverage is not guaranteed in finite samples; conformalized quantile regression adjusts the intervals to restore coverage (see conformal predictionconformal prediction, n.: A way to wrap any prediction model so that, instead of one guess, it gives a range or set of answers that contains the truth a chosen share of the time, such as 90%.).

    We stock to the 95th-percentile forecast from a quantile regression, not to the mean demand.

ρτ(u)=u (τ−1[u<0]),β^τ=arg⁡min⁡β∑iρτ(yi−xi⊤β)\rho_{\tau}(u) = u\,\big(\tau - \mathbf{1}[u < 0]\big), \qquad \hat{\boldsymbol\beta}_{\tau} = \arg\min_{\boldsymbol\beta} \sum_{i} \rho_{\tau}\big(y_i - \mathbf{x}_i^{\top}\boldsymbol\beta\big)

Origin

Median regression by least absolute deviations goes back to Ruđer Bošković (1760), about half a century before least squares, and was developed by Pierre-Simon Laplace and Francis Edgeworth. Roger Koenker and Gilbert Bassett generalized it to any quantile in 'Regression Quantiles' (Econometrica, 1978), founding the modern method.

Related

See also
  • regressionregression, n.: Predicting a number rather than a category — a house's price, tomorrow's temperature, a patient's blood pressure — by learning from past examples where the answer is known.
  • linear regressionlinear regression, n.: Fitting a straight line (or flat surface) through data points to predict one number from others — say, a house's price from its floor area — choosing the line that keeps prediction errors smallest.
  • gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it.
  • conformal predictionconformal prediction, n.: A way to wrap any prediction model so that, instead of one guess, it gives a range or set of answers that contains the truth a chosen share of the time, such as 90%.
  • loss functionloss function, n.: A score that measures how wrong a model's predictions are. Training means adjusting the model, step by step, to push this number down.
  • time series forecastingtime series forecasting, n.: Predicting future values of something measured over time, such as sales, electricity demand, prices or rainfall, from its past values and related information.
Contrast with
  • linear regressionlinear regression, n.: Fitting a straight line (or flat surface) through data points to predict one number from others — say, a house's price from its floor area — choosing the line that keeps prediction errors smallest.

Sources

  1. Koenker, R. & Bassett, G. (1978). Regression Quantiles. Econometrica, 46(1), 33–50.

Mentioned in

regressionregression, n.: Predicting a number rather than a category — a house's price, tomorrow's temperature, a patient's blood pressure — by learning from past examples where the answer is known., gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it., mean absolute errormean absolute error, n.: A way of scoring numerical predictions: average how far off each prediction is, ignoring direction, so the score reads as 'typically off by this much'. Lower is better., time series forecastingtime series forecasting, n.: Predicting future values of something measured over time, such as sales, electricity demand, prices or rainfall, from its past values and related information., conformal predictionconformal prediction, n.: A way to wrap any prediction model so that, instead of one guess, it gives a range or set of answers that contains the truth a chosen share of the time, such as 90%., Huber lossHuber loss, n.: A way of measuring prediction errors that treats small errors like squared distance but large ones more gently, so a few wild data points cannot drag the whole model off course., proper scoring ruleproper scoring rule, n.: A way of grading probability forecasts that gives the best expected score only to a forecaster who reports what they really believe, so there is nothing to gain by exaggerating or hedging., survival analysissurvival analysis, n.: Methods for predicting how long until something happens — a patient's relapse, a machine's failure, a customer's cancellation — that properly use the cases where it has not happened yet.