explainable boosting machine
In plain wordsA prediction method for spreadsheet-style data that draws each input's influence as its own readable graph and adds those effects up to make every prediction, usually losing little accuracy compared with opaque methods.
A generalized additive modelgeneralized additive model, n.: A prediction model that adds up separate, possibly curved effects of each input, so each input's influence can be drawn as its own graph and read off directly. with optional pairwise interactions (a GA2M) fitted by gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it. in a particular way: training visits the features in turn, each time adding a tiny, very shallow tree built on that feature alone (or on one chosen pair) at a very low learning rate. Over thousands of passes, and with outer baggingbagging, n.: Training many copies of a model, each on a different random resample of the data, and averaging their predictions. The averaging cancels out much of each copy's random error., this yields per-feature functions stored as lookup tables that can be plotted, inspected and even edited — for example, to remove an implausible jump before deployment. Important pairwise interactions are detected automatically and added after the main effects. On tabular benchmarks their accuracy is frequently within a small margin of random forestsrandom forest, n.: A prediction method that builds hundreds of decision trees, each from a random sample of the data and features, and lets them vote. The crowd is far more reliable than any single tree. and gradient-boosted ensembles, yet every prediction can be traced to the plotted shape functions; scoring a new case is quick, a sum of table lookups, whereas fitting takes longer than for ordinary boosted trees.
The EBM's graph for creatinine had a strange spike at 1.0, which turned out to be a default value in the records.
Origin
Developed by Rich Caruana and colleagues at Microsoft Research from the boosted additive models of Yin Lou, Caruana and Johannes Gehrke (KDD 2012) and GA2M (KDD 2013), and first released under that name in Microsoft's open-source InterpretML package in May 2019, described by Harsha Nori, Samuel Jenkins, Paul Koch and Caruana (arXiv, September 2019). Caruana's pneumonia study (KDD 2015) is its best-known motivation.
Usage note
In the 2015 pneumonia study, an intelligible model learned that asthma lowered mortality risk — an artifact of asthmatic patients receiving more aggressive care — showing how an accurate black box could encode the same dangerous pattern invisibly.
Related
- generalized additive modelgeneralized additive model, n.: A prediction model that adds up separate, possibly curved effects of each input, so each input's influence can be drawn as its own graph and read off directly.
- interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.
- gradient boostinggradient boosting, n.: A method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it.
- explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application.
- tabular datatabular data, n.: Data arranged as a table, like a spreadsheet: each row is a case — a customer, a loan, a patient — and each column a measured property. Much business and scientific data looks like this.
- XGBoostXGBoost, prop. n.: A fast, free software tool that makes predictions by adding up hundreds of small decision trees, each one fixing the mistakes of those before it. For years it was behind many winning entries in competitions on spreadsheet-style data.
- XGBoostXGBoost, prop. n.: A fast, free software tool that makes predictions by adding up hundreds of small decision trees, each one fixing the mistakes of those before it. For years it was behind many winning entries in competitions on spreadsheet-style data.
Sources
- Nori, H., Jenkins, S., Koch, P. & Caruana, R. (2019). InterpretML: A Unified Framework for Machine Learning Interpretability. arXiv:1909.09223. arxiv.org/abs/1909.09223
- Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M. & Elhadad, N. (2015). Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission. Proceedings of KDD 2015.
Mentioned in
generalized additive modelgeneralized additive model, n.: A prediction model that adds up separate, possibly curved effects of each input, so each input's influence can be drawn as its own graph and read off directly., interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.