graph neural network
In plain wordsA neural network for data shaped like a web of connections, such as molecules, social networks or road maps, in which each point repeatedly gathers information from its neighbors.
A neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain. that operates on graph-structured data by message passing: in each layer, every node aggregates messages computed from its neighbors' states and edge features with a permutation-invariant function (sum, mean, max or an attentionattention, n.: A way for an AI model to choose what to focus on. For each word it processes, the model scores how relevant every other word is and draws information mostly from the relevant ones.-weighted sum) and combines the result with its own state through a learned update. After k layers a node's representation reflects its k-hop neighborhood; predictions are read out per node, per edge or, after pooling, per graph. Layers are equivariantequivariance, n.: A property of a model whose output changes in step with its input: move the cat in a photo, and the network's internal 'cat' signal moves the same way. to node relabeling, and weight sharingweight sharing, n.: Reusing the same learned numbers in many places in a network, for example one edge detector applied at every spot in an image, so the network needs fewer parameters and learns from less data. across nodes lets one model handle graphs of any size. Major variants include graph convolutional networks, graph attention networks and GraphSAGE. Standard message passing can distinguish no more graphs than the Weisfeiler–Leman isomorphism test, and deep GNNs suffer oversmoothing and oversquashing. Applications include molecular property prediction and drug discovery, materials discovery, recommendation, traffic and weather forecasting and physics simulation; a transformer can be viewed as a GNN on a fully connected graph.
We represent each molecule as a graph of atoms and bonds and predict toxicity with a GNN.
Origin
Marco Gori, Gabriele Monfardini and Franco Scarselli (2005) and Scarselli and colleagues' 'The Graph Neural Network Model' (IEEE Transactions on Neural Networks, 2009) introduced the name and a recurrent formulation, building on 1990s recursive networks for structured data. Thomas Kipf and Max Welling's graph convolutional network (2016), Justin Gilmer and colleagues' message-passing framework (2017) and Petar Veličković and colleagues' graph attention networks (2017) brought GNNs into the mainstream.
Related
- geometric deep learninggeometric deep learning, n.: An approach to designing neural networks around the shapes and symmetries of the data, whether grids, graphs, surfaces or 3-D structures, so the network respects how that data can be moved or rearranged.
- equivarianceequivariance, n.: A property of a model whose output changes in step with its input: move the cat in a photo, and the network's internal 'cat' signal moves the same way.
- knowledge graphknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over.
- recommender systemrecommender system, n.: Software that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on.
- attentionattention, n.: A way for an AI model to choose what to focus on. For each word it processes, the model scores how relevant every other word is and draws information mostly from the relevant ones.
- weight sharingweight sharing, n.: Reusing the same learned numbers in many places in a network, for example one edge detector applied at every spot in an image, so the network needs fewer parameters and learns from less data.
- GraphRAGGraphRAG, prop. n.: A way of letting AI answer questions about a big pile of documents by first turning them into a map of people, things and their connections, then using that map to find and summarize answers.
Sources
- Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M. & Monfardini, G. (2009). The Graph Neural Network Model. IEEE Transactions on Neural Networks, 20(1), 61–80.
- Kipf, T. N. & Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. ICLR 2017.
Mentioned in
neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain., collaborative filteringcollaborative filtering, n.: Recommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'., poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions., weight sharingweight sharing, n.: Reusing the same learned numbers in many places in a network, for example one edge detector applied at every spot in an image, so the network needs fewer parameters and learns from less data., equivarianceequivariance, n.: A property of a model whose output changes in step with its input: move the cat in a photo, and the network's internal 'cat' signal moves the same way., geometric deep learninggeometric deep learning, n.: An approach to designing neural networks around the shapes and symmetries of the data, whether grids, graphs, surfaces or 3-D structures, so the network respects how that data can be moved or rearranged., invarianceinvariance, n.: A property of a model whose answer does not change when the input is altered in an irrelevant way, such as a photo classifier that still says 'cat' when the cat moves or the picture is flipped., knowledge graph embeddingknowledge graph embedding, n.: A way of turning the things and relations in a knowledge graph into lists of numbers, so a computer can guess missing facts from patterns, such as predicting where someone was born., spectral clusteringspectral clustering, n.: A clustering method that treats data points as a network linked by similarity and finds natural groups by cutting the network where its links are weakest.