Samuel's checkers program
In plain wordsA 1950s IBM program by Arthur Samuel that learned to play checkers better than its author, partly by playing against itself. It was among the first programs to learn, and its 1959 paper popularized the phrase 'machine learning'.
A checkers-playing program developed by Arthur SamuelArthur Samuel, prop. n.: An American IBM engineer whose checkers program improved by playing against itself, and whose 1959 paper helped make 'machine learning' the name of a field. at IBM from the early 1950s, among the first computer programs to improve with experience. It searched ahead with minimaxminimax, n.: A way for a game-playing program to choose moves by assuming the opponent will always make their best reply, and picking the move whose worst-case outcome is best. over a game treegame tree, n.: A branching diagram of every way a game could unfold: each point is a position, each branch a possible move, and the tips of the branches are wins, losses or draws., using an early form of alpha–beta pruning, and scored positions with a weighted sum of board features. It learned in two ways: rote learning, storing previously evaluated positions to extend its effective search depth; and 'learning by generalization', adjusting the feature weights so that its evaluation of a position moved toward the value backed up from searching ahead of it — an early form of the temporal-difference idea later developed in reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.. After thousands of games, including against versions of itself, it came to beat Samuel, and in 1962 it won a much-publicized game against Robert Nealey, a strong amateur player.
Samuel's checkers program nudged its evaluation toward the result of its own lookahead, a trick reinforcement learning still uses.
Origin
Begun around 1952 on the IBM 701; a televised demonstration in February 1956 drew wide attention. Samuel's 1959 paper 'Some Studies in Machine Learning Using the Game of Checkers' (IBM Journal of Research and Development) is the most-cited early use of the phrase machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats..
Usage note
Its reputation outran its strength: it never approached world-class play, which came only with ChinookChinook, prop. n.: A checkers program from the University of Alberta that in 1994 became the first program crowned world champion in a contest against people, and in 2007 proved that perfect checkers always ends in a draw. in the 1990s.
Related
- Arthur SamuelArthur Samuel, prop. n.: An American IBM engineer whose checkers program improved by playing against itself, and whose 1959 paper helped make 'machine learning' the name of a field.
- machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
- ChinookChinook, prop. n.: A checkers program from the University of Alberta that in 1994 became the first program crowned world champion in a contest against people, and in 2007 proved that perfect checkers always ends in a draw.
- TD-GammonTD-Gammon, prop. n.: A backgammon program built at IBM in the early 1990s that taught itself to play at near world-champion level by playing more than a million games against itself.
- self-playself-play, n.: Training an AI by having it play against copies of itself, so it always has an opponent at its own level and can keep improving without human examples.
Sources
- Samuel, A. L. (1959). Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development 3(3), 210–229.
Mentioned in
ChinookChinook, prop. n.: A checkers program from the University of Alberta that in 1994 became the first program crowned world champion in a contest against people, and in 2007 proved that perfect checkers always ends in a draw.