Bayesian network
In plain wordsA diagram of causes and effects with probabilities attached — say, how rain and a sprinkler can each make grass wet — that lets a computer update its beliefs about hidden causes when it sees evidence.
A probabilistic graphical modelprobabilistic graphical model, n.: A way of describing how many uncertain quantities depend on one another by drawing them as a network, which makes it practical for a computer to reason about all of them together. consisting of a directed acyclic graph whose nodes are random variables, with edges indicating direct dependence, together with a conditional probability distribution for each node given its parents. The joint distribution factorizes into these local conditionals, and the graph encodes conditional independences (readable by the d-separation criterion), so a model over many variables needs far fewer parameters than a full joint table. Inference computes posterior probabilities of unobserved variables given evidence — exactly by variable elimination, belief propagationbelief propagation, n.: A way for a network of uncertain quantities to reach conclusions by having each node pass short messages to its neighbors about what it currently believes, until the beliefs settle down. or junction trees (NP-hard in general), or approximately by sampling and variational inferencevariational inference, n.: A way of approximating a hard-to-compute probability distribution by picking the closest match from a simpler family of distributions, turning a hard problem of adding up every possibility into an optimization problem. — and structure and parameters can be learned from data. Applications include medical and fault diagnosis, risk analysis and genetics. Read causally, Bayesian networks underpin causal inferencecausal inference, n.: Working out whether one thing actually causes another, rather than merely occurring alongside it — for example, whether a new feature made users stay longer or just happened at the same time. and the do-calculusdo-calculus, n.: A set of mathematical rules, devised by Judea Pearl, for working out the effect of deliberately changing something using only observational data plus a diagram of what causes what.; dynamic Bayesian networks, which repeat the structure over time, generalize hidden Markov modelshidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds..
The Bayesian network says a positive test raises the probability of disease only from 1 to 9 percent.
Origin
Judea Pearl coined the term in a 1985 paper on evidential reasoning, having introduced message-passing inference in 1982; his book Probabilistic Reasoning in Intelligent Systems (1988) established the field, and the work was central to his 2011 Turing Award. Sewall Wright's path diagrams (1921) were a forerunner.
Usage note
'Bayesian' refers to updating beliefs with Bayes' theoremBayes' theorem, n.: A formula for updating how much you believe something after seeing new evidence: weigh your earlier belief by how well that explanation predicts what you actually observed.; the parameters need not be estimated by Bayesian statistics. Not to be confused with a Bayesian neural networkBayesian neural network, n.: A neural network that keeps a whole range of plausible settings for its internal numbers instead of one best guess, so that it can say how unsure it is about each prediction., which places probability distributions over network weights.
Related
- probabilistic graphical modelprobabilistic graphical model, n.: A way of describing how many uncertain quantities depend on one another by drawing them as a network, which makes it practical for a computer to reason about all of them together.
- Bayes' theoremBayes' theorem, n.: A formula for updating how much you believe something after seeing new evidence: weigh your earlier belief by how well that explanation predicts what you actually observed.
- belief propagationbelief propagation, n.: A way for a network of uncertain quantities to reach conclusions by having each node pass short messages to its neighbors about what it currently believes, until the beliefs settle down.
- Markov random fieldMarkov random field, n.: A model of many linked uncertain quantities, such as neighboring pixels in a photo, in which each one depends directly only on its neighbors, with no notion of which one causes which.
- hidden Markov modelhidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds.
- causal inferencecausal inference, n.: Working out whether one thing actually causes another, rather than merely occurring alongside it — for example, whether a new feature made users stay longer or just happened at the same time.
- Judea PearlJudea Pearl, prop. n.: An Israeli-American computer scientist who gave AI the mathematics for reasoning under uncertainty and, later, for telling cause and effect apart from mere correlation.
- Markov random fieldMarkov random field, n.: A model of many linked uncertain quantities, such as neighboring pixels in a photo, in which each one depends directly only on its neighbors, with no notion of which one causes which.
- Bayesian neural networkBayesian neural network, n.: A neural network that keeps a whole range of plausible settings for its internal numbers instead of one best guess, so that it can say how unsure it is about each prediction.
Sources
- Pearl, J. (1985). Bayesian Networks: A Model of Self-Activated Memory for Evidential Reasoning. Proceedings of the 7th Conference of the Cognitive Science Society, 329–334.
- Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann.
Mentioned in
symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data., abductive reasoningabductive reasoning, n.: Reasoning from an observation to its most likely explanation — the grass is wet, so it probably rained — the kind of educated guess that doctors and detectives make., Bayes' theoremBayes' theorem, n.: A formula for updating how much you believe something after seeing new evidence: weigh your earlier belief by how well that explanation predicts what you actually observed., Bayesian inferenceBayesian inference, n.: Reasoning that treats unknowns as uncertain beliefs and updates them with evidence using Bayes' theorem. The result is not one answer but a range of answers, each with a probability., conditional probabilityconditional probability, n.: The chance of something once you already know that something else is true — for example, the chance of rain given that the sky is cloudy., good old-fashioned AIgood old-fashioned AI, n.: A nickname for the classic style of AI, dominant from the 1950s to the 1980s, that tried to make computers intelligent by writing knowledge and rules down by hand., hidden Markov modelhidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds., Judea PearlJudea Pearl, prop. n.: An Israeli-American computer scientist who gave AI the mathematics for reasoning under uncertainty and, later, for telling cause and effect apart from mere correlation., knowledge representationknowledge representation, n.: The study of how to record knowledge about objects, categories, rules and events in a precise format, so that a program can reason with it and reach conclusions nobody typed in., NP-completeNP-complete, adj.: Describes problems, such as solving giant Sudokus, where a proposed answer is quick to check but no known method finds one quickly; a fast method for any one of them could be turned into one for all., probabilistic graphical modelprobabilistic graphical model, n.: A way of describing how many uncertain quantities depend on one another by drawing them as a network, which makes it practical for a computer to reason about all of them together., reasoningreasoning, n.: Working out conclusions step by step from facts, rules or evidence, as in solving a math problem or planning a trip. Whether today's AI truly reasons, or imitates reasoning, is debated., Bayesian neural networkBayesian neural network, n.: A neural network that keeps a whole range of plausible settings for its internal numbers instead of one best guess, so that it can say how unsure it is about each prediction., belief propagationbelief propagation, n.: A way for a network of uncertain quantities to reach conclusions by having each node pass short messages to its neighbors about what it currently believes, until the beliefs settle down., Daphne KollerDaphne Koller, prop. n.: An Israeli-American computer scientist who wrote the standard text on probabilistic reasoning in AI, co-founded the online-course company Coursera, and founded a machine-learning drug-discovery company., Dempster–Shafer theoryDempster–Shafer theory, n.: A theory of reasoning with uncertain evidence that lets you hold back belief — in effect saying 'I don't know' — instead of being forced to divide all your confidence among the possibilities., do-calculusdo-calculus, n.: A set of mathematical rules, devised by Judea Pearl, for working out the effect of deliberately changing something using only observational data plus a diagram of what causes what., Markov random fieldMarkov random field, n.: A model of many linked uncertain quantities, such as neighboring pixels in a photo, in which each one depends directly only on its neighbors, with no notion of which one causes which., probabilistic programmingprobabilistic programming, n.: Writing a statistical model as an ordinary-looking program that includes random choices, and letting the system work out automatically which hidden values best explain the observed data., structural causal modelstructural causal model, n.: A precise description of what causes what in a system — a diagram of arrows plus equations — that predicts what would happen if you intervened, not just what tends to occur together.