Anchors
In plain wordsA method that explains a prediction with a simple if-then rule that locks in the result — for example, if an email contains both 'invoice' and 'urgent', the filter almost always flags it.
A model-agnosticmodel-agnostic, adj.: Describes a method that works with any kind of AI model, because it only needs to feed the model inputs and observe its outputs, never to look inside. local explanationlocal explanation, n.: An explanation of why an AI model gave one particular answer for one particular case — this patient, this loan — rather than how it behaves in general. method that summarizes why a model made one prediction as an if-then rule, the anchor: as long as the rule's conditions hold, changing the rest of the input almost never alters the output. Candidate rules are built from interpretable features (words, superpixels, feature ranges) and assessed by precision (how often the prediction holds among perturbed inputs satisfying the rule) and coverage (how much of the input space the rule applies to), with a multi-armed bandit procedure deciding which candidates to sample; the method returns a high-coverage rule whose precision exceeds a target such as 95%. Because an anchor says explicitly which inputs it covers, which the weights of LIMELIME, abbr.: A method that explains one prediction of any AI model by testing slightly altered versions of the input and fitting a very simple model that copies the AI's behavior near that example. do not, people in the authors' user study forecast the model's behavior on new inputs more accurately and with less effort.
The anchor for this loan denial was simply: no savings account and fewer than two years employed.
Origin
Introduced by Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin in 'Anchors: High-Precision Model-Agnostic Explanations' (AAAI 2018), as a successor to their LIMELIME, abbr.: A method that explains one prediction of any AI model by testing slightly altered versions of the input and fitting a very simple model that copies the AI's behavior near that example. method.
Usage note
An anchor's sufficiency holds only relative to the perturbation distribution used, and anchors for inputs near the decision boundary can be so specific that they cover almost nothing else. Not to be confused with anchor boxesanchor box, n.: A ready-made box shape, placed at many spots across an image, that an object detector stretches and shifts to fit real objects instead of drawing boxes from scratch. in object detection.
Related
- LIMELIME, abbr.: A method that explains one prediction of any AI model by testing slightly altered versions of the input and fitting a very simple model that copies the AI's behavior near that example.
- local explanationlocal explanation, n.: An explanation of why an AI model gave one particular answer for one particular case — this patient, this loan — rather than how it behaves in general.
- model-agnosticmodel-agnostic, adj.: Describes a method that works with any kind of AI model, because it only needs to feed the model inputs and observe its outputs, never to look inside.
- counterfactual explanationcounterfactual explanation, n.: An explanation that tells someone the smallest change that would have produced a different outcome — for example, that the application would have been accepted if it had listed two more years of work experience.
- feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down.
Sources
- Ribeiro, M. T., Singh, S. & Guestrin, C. (2018). Anchors: High-Precision Model-Agnostic Explanations. Proceedings of AAAI 2018.
Mentioned in
LIMELIME, abbr.: A method that explains one prediction of any AI model by testing slightly altered versions of the input and fitting a very simple model that copies the AI's behavior near that example., local explanationlocal explanation, n.: An explanation of why an AI model gave one particular answer for one particular case — this patient, this loan — rather than how it behaves in general., model-agnosticmodel-agnostic, adj.: Describes a method that works with any kind of AI model, because it only needs to feed the model inputs and observe its outputs, never to look inside.