concept bottleneck model
In plain wordsA model made to first predict human-understandable concepts — such as 'bone spur present' on an X-ray — and then decide only from those concepts, so people can inspect and correct them.
A model that predicts a set of human-specified concepts from the input and then predicts the target label only from those concept predictions, so that the concepts form an interpretable bottleneck. In the original work, networks graded knee osteoarthritis from X-rays via concepts such as joint-space narrowing and bone spurs, and identified bird species via attributes such as wing color. The two stages can be trained independently, sequentially or jointly. Beyond explaining predictions in terms of concepts, CBMs allow test-time intervention: an expert corrects a mispredicted concept and the label prediction updates accordingly. Limitations include the need for concept annotations, lost accuracy when the concept set is incomplete, and concept leakage, in which soft concept scores carry extra information that the label predictor exploits. Variants derive concepts automatically with vision–language models (label-free and post hoc CBMs) or replace scalar concepts with embeddings.
When the radiologist corrected the 'bone spurs' concept, the CBM's severity grade dropped from 3 to 2.
Origin
Named by Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim and Percy Liang in 'Concept Bottleneck Models' (ICML 2020), revisiting earlier attribute-based classifiers in computer vision, such as Christoph Lampert and colleagues' (2009).
Usage note
A concept bottleneck makes reasoning legible only insofar as the concept predictions are accurate and the label predictor depends on them as labeled.
Related
- interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.
- testing with concept activation vectorstesting with concept activation vectors, phr.: A method for asking whether a human idea, such as 'wheels', matters to an image model's decision, such as 'car', by finding the direction for that idea inside the model and measuring its effect.
- explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application.
- human-in-the-loophuman-in-the-loop, adj.: Describes AI systems in which a person checks, corrects or approves the machine's work at key points, rather than letting it act entirely on its own.
- post hoc explanationpost hoc explanation, n.: An explanation produced after a model has been built, by a separate tool that examines the finished model, rather than one that comes from a model designed to be understandable.
- post hoc explanationpost hoc explanation, n.: An explanation produced after a model has been built, by a separate tool that examines the finished model, rather than one that comes from a model designed to be understandable.
Sources
- Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B. & Liang, P. (2020). Concept Bottleneck Models. Proceedings of ICML 2020.
Mentioned in
interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed., prototypical part networkprototypical part network, n.: An image classifier that explains each decision by pointing to parts of the picture that resemble typical examples it learned — this part looks like that part of a known bird., testing with concept activation vectorstesting with concept activation vectors, phr.: A method for asking whether a human idea, such as 'wheels', matters to an image model's decision, such as 'car', by finding the direction for that idea inside the model and measuring its effect.