perceptron

Origin

Introduced by psychologist Frank Rosenblatt at the Cornell Aeronautical Laboratory in a 1957 report and a 1958 Psychological Review paper; the learning rule was first simulated on an IBM 704. The custom-built Mark I Perceptron followed, first demonstrated publicly in 1960. Albert Novikoff proved the convergence bound in 1962. Marvin Minsky and Seymour Papert's book Perceptrons (1969) analyzed the limits of single-layer versions.

Usage note

The 1969 critique concerned single-layer perceptrons; Minsky and Papert knew multilayer networks could compute XOR but doubted they could be trained. The name survives in multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model., although MLP units use differentiable or piecewise-linear activations rather than step functions.

Related

Sources

1. Rosenblatt, F. (1958). The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. Psychological Review, 65(6), 386–408.
2. Novikoff, A. B. J. (1962). On Convergence Proofs on Perceptrons. Symposium on the Mathematical Theory of Automata, Polytechnic Institute of Brooklyn.

Mentioned in

artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output., neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain., activation functionactivation function, n.: The simple rule each artificial neuron applies to its total input to decide its output, such as 'keep positive values, turn negatives into zero'. Without it, a deep network would be no smarter than a single layer., connectionismconnectionism, n.: The view that minds and intelligent machines are best built from huge networks of simple, neuron-like units whose connection strengths are learned, rather than from explicit rules and symbols., Frank RosenblattFrank Rosenblatt, prop. n.: An American psychologist who invented the perceptron, the first neural network that learned from examples, and built a machine that learned to recognize simple images., linear separabilitylinear separability, n.: When two groups of points can be perfectly divided by a single straight line or, in more dimensions, a flat surface. Many simple learning methods only fully succeed in this case., Marvin MinskyMarvin Minsky, prop. n.: An American AI pioneer who helped launch the field and MIT's AI laboratory, co-wrote a famous critique of early neural networks, and pictured the mind as a society of many simple parts., McCulloch–Pitts neuronMcCulloch–Pitts neuron, n.: The first mathematical model of a brain cell as a simple on–off switch: it fires if enough of its inputs are on and no blocking input is. Networks of them can carry out logic., multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model., online learningonline learning, n.: Learning from data that arrive one piece at a time, updating the model after each new example instead of training once on a fixed collection, as a spam filter does while emails stream in., perceptionperception, n.: How an AI system or robot turns raw sensor signals, such as camera pixels, microphone sound or laser scans, into an understanding of what is around it: objects, people, words and distances., Perceptrons (book)Perceptrons (book), prop. n.: A 1969 book by Marvin Minsky and Seymour Papert proving that simple one-layer neural networks cannot learn certain patterns. It is widely blamed for a decade-long slump in neural-network research., The Last DictionaryThe Last Dictionary, prop. n.: This book: a dictionary of intelligence, human and otherwise, which explains each word first in plain language, then precisely, and says when it was born., XOR problemXOR problem, n.: The classic puzzle of teaching a single artificial neuron the 'one or the other but not both' rule. It cannot be done with one straight dividing line, which showed why networks need hidden layers., ADALINEADALINE, prop. n.: An early learning machine from 1960 that adjusted its settings to shrink its errors step by step, the same basic idea used to train today's neural networks., Bernard WidrowBernard Widrow, prop. n.: An American electrical engineer who in 1960 built ADALINE, an early trainable neural unit, and invented a learning rule that adjusts weights to reduce errors, still at the heart of adaptive filters., delta ruledelta rule, n.: A simple learning rule for one artificial neuron: adjust each connection in proportion to the error, the gap between the wanted and the actual output, times that connection's input., Mark I PerceptronMark I Perceptron, prop. n.: A room-sized machine built around 1960 that learned to recognize simple shapes and letters through a camera made of 400 light sensors. It was one of the first machines built to learn from examples., Seymour PapertSeymour Papert, prop. n.: A South African-born mathematician who co-wrote 'Perceptrons', a critique of early neural networks, with Marvin Minsky, and invented the children's programming language Logo.