Jerome Friedman
In plain wordsAn American statistician who invented gradient boosting, the method behind XGBoost and other leading tools for spreadsheet-style data, and co-wrote a standard machine-learning textbook.
American statistician at Stanford University who created several of the most widely used methods of statistical learning. Trained as a physicist, he led the computation research group at the Stanford Linear Accelerator Center while developing data-analysis methods: projection pursuit (with John Tukey, 1974), CARTCART, abbr.: A classic recipe for building decision trees that splits data two ways at every step, works for both categories and numbers, and then trims the tree back so it does not memorize noise. decision trees (with Leo BreimanLeo Breiman, prop. n.: An American statistician who invented random forests and bagging, decision-tree methods still in wide use, and urged statisticians to judge models by how well they predict., Richard Olshen and Charles Stone, 1984) and multivariate adaptive regression splines (MARS, 1991). His 'Greedy Function Approximation: A Gradient Boosting Machine' (1999; published 2001) recast boosting as gradient descent in function space, giving the general gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it. algorithm on which XGBoostXGBoost, prop. n.: A fast, free software tool that makes predictions by adding up hundreds of small decision trees, each one fixing the mistakes of those before it. For years it was behind many winning entries in competitions on spreadsheet-style data., LightGBMLightGBM, prop. n.: A widely used tool for making predictions from very large tables of data, quickly and in little memory, by combining many small decision trees. Microsoft released it as free software in 2016. and CatBoostCatBoost, prop. n.: A free gradient-boosting library from the tech company Yandex, known for handling categories such as city names or product types directly, without first converting them into numbers by hand. build, and he added stochastic gradient boosting. With Trevor Hastie and Robert Tibshirani he wrote 'The Elements of Statistical Learning' (2001), a standard text, and developed fast coordinate-descent algorithms for the lassolasso, n.: A version of linear regression that keeps only the most useful inputs: it penalizes the total size of the weights so that many become exactly zero, leaving a simpler model. and elastic netelastic net, n.: A blend of two ways of keeping a model simple: it can drop useless inputs entirely while treating groups of similar inputs alike, instead of picking one of them at random. (the glmnet software).
Related
- gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it.
- CARTCART, abbr.: A classic recipe for building decision trees that splits data two ways at every step, works for both categories and numbers, and then trims the tree back so it does not memorize noise.
- decision treedecision tree, n.: A model that makes a prediction by asking a series of yes-or-no questions about the input, like a flowchart, with each answer leading to the next question until it reaches a final answer.
- XGBoostXGBoost, prop. n.: A fast, free software tool that makes predictions by adding up hundreds of small decision trees, each one fixing the mistakes of those before it. For years it was behind many winning entries in competitions on spreadsheet-style data.
- lassolasso, n.: A version of linear regression that keeps only the most useful inputs: it penalizes the total size of the weights so that many become exactly zero, leaving a simpler model.
- Leo BreimanLeo Breiman, prop. n.: An American statistician who invented random forests and bagging, decision-tree methods still in wide use, and urged statisticians to judge models by how well they predict.
Sources
- Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics 29(5): 1189–1232.
- Hastie, T., Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.