Leo Breiman
In plain wordsAn American statistician who invented random forests and bagging, decision-tree methods still in wide use, and urged statisticians to judge models by how well they predict.
American statistician at the University of California, Berkeley. After early work in probability theory, including the Shannon–McMillan–Breiman theorem, he spent more than a decade as a consultant before joining Berkeley's statistics department in 1980, an experience that shaped his practical outlook. With Jerome Friedman, Richard Olshen and Charles Stone he wrote 'Classification and Regression Trees' (1984), which introduced the CARTCART, abbr.: A classic recipe for building decision trees that splits data two ways at every step, works for both categories and numbers, and then trims the tree back so it does not memorize noise. algorithm for decision treesdecision tree, n.: A model that makes a prediction by asking a series of yes-or-no questions about the input, like a flowchart, with each answer leading to the next question until it reaches a final answer.. He then invented baggingbagging, n.: Training many copies of a model, each on a different random resample of the data, and averaging their predictions. The averaging cancels out much of each copy's random error. (bootstrap aggregating, 1996), which averages models trained on bootstrapbootstrap, n.: Estimating how much a result might vary by repeatedly re-drawing samples, with replacement, from the data you already have — like re-running a study many times without collecting new data. resamples, and random forestsrandom forest, n.: A prediction method that builds hundreds of decision trees, each from a random sample of the data and features, and lets them vote. The crowd is far more reliable than any single tree. (2001), which add random feature selection to bagged trees and remain a strong default for tabular datatabular data, n.: Data arranged as a table, like a spreadsheet: each row is a case — a customer, a loan, a patient — and each column a measured property. Much business and scientific data looks like this.. His essay 'Statistical Modeling: The Two Cultures' (2001) contrasted a data-modeling culture that assumes a stochastic model with an algorithmic-modeling culture that judges methods by predictive accuracy, and urged statisticians to embrace the second, as machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats. had.
Related
- random forestrandom forest, n.: A prediction method that builds hundreds of decision trees, each from a random sample of the data and features, and lets them vote. The crowd is far more reliable than any single tree.
- baggingbagging, n.: Training many copies of a model, each on a different random resample of the data, and averaging their predictions. The averaging cancels out much of each copy's random error.
- CARTCART, abbr.: A classic recipe for building decision trees that splits data two ways at every step, works for both categories and numbers, and then trims the tree back so it does not memorize noise.
- decision treedecision tree, n.: A model that makes a prediction by asking a series of yes-or-no questions about the input, like a flowchart, with each answer leading to the next question until it reaches a final answer.
- ensemble learningensemble learning, n.: Combining the predictions of several models — by voting or averaging — so their individual mistakes partly cancel out, giving more accurate and stable results than any one model.
- bootstrapbootstrap, n.: Estimating how much a result might vary by repeatedly re-drawing samples, with replacement, from the data you already have — like re-running a study many times without collecting new data.
Sources
- Breiman, L. (1996). Bagging Predictors. Machine Learning 24(2): 123–140.
- Breiman, L. (2001). Random Forests. Machine Learning 45(1): 5–32.
- Breiman, L. (2001). Statistical Modeling: The Two Cultures. Statistical Science 16(3): 199–231.
Mentioned in
Jerome FriedmanJerome Friedman, prop. n.: An American statistician who invented gradient boosting, the method behind XGBoost and other leading tools for spreadsheet-style data, and co-wrote a standard machine-learning textbook., Rashomon effectRashomon effect, n.: The common situation in which many quite different models fit the same data almost equally well — like witnesses telling different but equally believable stories about the same event.