logical induction
In plain wordsA mathematical method for giving sensible, steadily improving probabilities to math claims that haven't been proved or disproved yet, much as a betting market prices uncertain events.
A criterion for good reasoning under logical uncertaintylogical uncertainty, n.: Uncertainty about math or logic facts that are settled in principle but that you haven't worked out yet — like not knowing a distant digit of pi until you compute it. — uncertainty about mathematical facts that follow from known premises but have not yet been worked out — together with a computable algorithm that satisfies it. A logical inductor is a process that, at each step, assigns prices between 0 and 1 to every sentence of a formal language while a deductive process proves ever more theorems; it meets the criterion if no trader that runs in polynomial time can exploit its prices to make unbounded profit at bounded risk, by analogy with a market that resists arbitrage. Scott Garrabrant and colleagues proved that such inductors exist and that the criterion implies many desirable properties: prices converge to a coherent probability distribution, regularities are learned well before they are proved, beliefs are calibrated, and the inductor appropriately trusts its own future beliefs. The construction is far too slow to run in practice.
A logical inductor assigns probabilities near 10% to claims like 'the trillionth digit of pi is 7' long before it computes the digit.
Origin
Introduced by Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares and Jessica Taylor of the Machine Intelligence Research Institute in the paper 'Logical Induction' (arXiv, September 2016), part of its agent-foundations research on how bounded reasoners embedded in the world should handle uncertainty about mathematics and about their own computations.
Usage note
Not to be confused with mathematical induction, a proof technique, or with inductive learning from data in general. Solomonoff inductionSolomonoff induction, n.: A mathematical recipe for ideal prediction: consider every computer program that could have produced the data so far, trust shorter programs more, and predict what they would output next. is the analogous idealization for predicting empirical sequences.
Related
- Solomonoff inductionSolomonoff induction, n.: A mathematical recipe for ideal prediction: consider every computer program that could have produced the data so far, trust shorter programs more, and predict what they would output next.
- embedded agencyembedded agency, n.: The puzzle of how an agent should reason when it is itself part of the world it is trying to understand and change, rather than standing outside it.
- updateless decision theoryupdateless decision theory, n.: A theory of rational choice in which an agent commits to the best overall plan for every situation it might face, instead of re-deciding each time it learns something new.
- Machine Intelligence Research InstituteMachine Intelligence Research Institute, prop. n.: A small nonprofit in Berkeley, California, among the first to warn that smarter-than-human AI could destroy humanity, which now campaigns for a worldwide halt to building it.
- Bayesian inferenceBayesian inference, n.: Reasoning that treats unknowns as uncertain beliefs and updates them with evidence using Bayes' theorem. The result is not one answer but a range of answers, each with a probability.
- calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time.
Sources
- Garrabrant, S., Benson-Tilsen, T., Critch, A., Soares, N. & Taylor, J. (2016). Logical Induction. arXiv:1609.03543.
Mentioned in
logical uncertaintylogical uncertainty, n.: Uncertainty about math or logic facts that are settled in principle but that you haven't worked out yet — like not knowing a distant digit of pi until you compute it., updateless decision theoryupdateless decision theory, n.: A theory of rational choice in which an agent commits to the best overall plan for every situation it might face, instead of re-deciding each time it learns something new.