LIME
In plain wordsA method that explains one prediction of any AI model by testing slightly altered versions of the input and fitting a very simple model that copies the AI's behavior near that example.
Local Interpretable Model-agnostic Explanations: a post hoc explanationpost hoc explanation, n.: An explanation produced after a model has been built, by a separate tool that examines the finished model, rather than one that comes from a model designed to be understandable. method that probes a black-box model around a single input to see which parts of that input matter. It creates many variants of the input by switching interpretable components on and off (words in a text, superpixels in an image, discretized values in a table), records the model's outputs on them, weights each variant by its closeness to the original and fits a sparse linear model to the weighted results; the coefficients of this local surrogate modelsurrogate model, n.: A simpler stand-in model trained to imitate a more complex one — to explain it, or to approximate a slow or costly calculation cheaply. are the explanation, and because only outputs are needed, any classifier or regressor can be explained. A companion procedure, submodular pick (SP-LIME), selects a few explanations that together cover diverse behavior, as a global overview. Results depend on the perturbation scheme, kernel width and number of samples and can vary between runs; and because perturbed inputs may be unrealistic, adversaries have built models that hide biased behavior from LIME.
LIME highlighted the snowy background, not the animal, as the reason the photo was labeled 'wolf'.
Origin
Introduced by Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin of the University of Washington in ''Why Should I Trust You?': Explaining the Predictions of Any Classifier' (arXiv, February 2016; KDD 2016). Its demonstration of a husky-versus-wolf classifier relying on snow became a standard teaching example.
Usage note
Explanations are faithful only locally, and LIME and SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players. can disagree on the same prediction. The adversarial weakness was shown by Dylan Slack and colleagues in 'Fooling LIME and SHAP' (2020).
Related
- SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players.
- surrogate modelsurrogate model, n.: A simpler stand-in model trained to imitate a more complex one — to explain it, or to approximate a slow or costly calculation cheaply.
- local explanationlocal explanation, n.: An explanation of why an AI model gave one particular answer for one particular case — this patient, this loan — rather than how it behaves in general.
- model-agnosticmodel-agnostic, adj.: Describes a method that works with any kind of AI model, because it only needs to feed the model inputs and observe its outputs, never to look inside.
- AnchorsAnchors, prop. n.: A method that explains a prediction with a simple if-then rule that locks in the result — for example, if an email contains both 'invoice' and 'urgent', the filter almost always flags it.
- feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down.
- SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players.
Sources
- Ribeiro, M. T., Singh, S. & Guestrin, C. (2016). 'Why Should I Trust You?': Explaining the Predictions of Any Classifier. Proceedings of KDD 2016.
Mentioned in
explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application., feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down., local explanationlocal explanation, n.: An explanation of why an AI model gave one particular answer for one particular case — this patient, this loan — rather than how it behaves in general., post hoc explanationpost hoc explanation, n.: An explanation produced after a model has been built, by a separate tool that examines the finished model, rather than one that comes from a model designed to be understandable., SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players., surrogate modelsurrogate model, n.: A simpler stand-in model trained to imitate a more complex one — to explain it, or to approximate a slow or costly calculation cheaply., AnchorsAnchors, prop. n.: A method that explains a prediction with a simple if-then rule that locks in the result — for example, if an email contains both 'invoice' and 'urgent', the filter almost always flags it., KDDKDD, prop. n.: The leading yearly conference on data mining — finding useful patterns in large datasets — where many widely used machine learning tools were first presented., model-agnosticmodel-agnostic, adj.: Describes a method that works with any kind of AI model, because it only needs to feed the model inputs and observe its outputs, never to look inside., occlusion sensitivityocclusion sensitivity, n.: Finding out which parts of an image matter to a model by covering them up one patch at a time and watching how much the model's answer changes.