multimodal neuron
In plain wordsA unit inside an AI vision model that responds to one idea however it appears — a photo, a drawing or the written name — much like the 'Halle Berry neuron' found in a human brain.
A neuron in a vision or vision-language model that responds to a single concept across very different presentations: photographs, drawings, symbols and the written word. Gabriel Goh and colleagues found many in OpenAI's CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image. models — a 'Spider-Man' neuron responding to photos of the character, comic drawings and the text 'spider', and neurons for famous people, places, emotions, holidays and geographic regions. They parallel 'concept cells' in the human medial temporal lobe reported by Rodrigo Quian Quiroga and colleagues (2005), such as a neuron that fired for varied pictures of Jennifer Aniston and another that fired for photos and drawings of Halle Berry and for her written name. Because written text activates these neurons strongly, CLIP proved vulnerable to typographic attacks: an apple bearing a handwritten label reading 'iPod' was classified as an iPod. The study also documented neurons encoding offensive associations, showing how interpretability can surface bias.
The same multimodal neuron fires for a photo of a bridge, a sketch of it and the word 'bridge'.
Origin
Described by Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford and Chris Olah in 'Multimodal Neurons in Artificial Neural Networks' (Distill, March 2021), from the OpenAI team behind the Circuits thread.
Related
- CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image.
- feature visualizationfeature visualization, n.: Making pictures that show what a part of an image-recognition network is looking for, by repeatedly adjusting an image until that part responds as strongly as possible.
- polysemanticitypolysemanticity, n.: When a single artificial neuron responds to several unrelated things — say, bird beaks, bicycle wheels and a particular typeface — so that no single label describes what it does.
- feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity.
- network dissectionnetwork dissection, n.: A method for finding out what individual units inside an image-recognition network respond to, such as dogs, windows, grass or stripes, by comparing where each unit lights up with pictures labeled by people.
- adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
Sources
- Goh, G., Cammarata, N., Voss, C., Carter, S., Petrov, M., Schubert, L., Radford, A. & Olah, C. (2021). Multimodal Neurons in Artificial Neural Networks. Distill.
- Quian Quiroga, R., Reddy, L., Kreiman, G., Koch, C. & Fried, I. (2005). Invariant visual representation by single neurons in the human brain. Nature 435, 1102–1107.
Mentioned in
feature visualizationfeature visualization, n.: Making pictures that show what a part of an image-recognition network is looking for, by repeatedly adjusting an image until that part responds as strongly as possible., network dissectionnetwork dissection, n.: A method for finding out what individual units inside an image-recognition network respond to, such as dogs, windows, grass or stripes, by comparing where each unit lights up with pictures labeled by people.