Chris Watkins

British computer scientist at Royal Holloway, University of London, who introduced Q-learningQ-learning, n.: A trial-and-error method in which an agent keeps a score for every move in every situation and nudges each score toward the reward it got plus the best score available next. in his 1989 Cambridge doctoral thesis, 'Learning from Delayed Rewards'. Drawing on animal-learning psychology and on dynamic programmingdynamic programming, n.: A problem-solving method that breaks a big problem into smaller overlapping pieces, solves each piece once and reuses the answers — the basis for computing the best long-term strategy step by step., he proposed learning the value of each action in each state directly from experience — nudging the estimate toward the reward received plus the discounted value of the best next action — without needing a model of the environment; with Peter Dayan he proved in 1992 that the method converges to the optimal action values provided every state–action pair continues to be tried. As the first widely adopted off-policyoff-policy, adj.: Describes learning that can use experience produced by some other strategy — an older version, a human, a different program — while improving the current one., model-freemodel-free reinforcement learning, n.: Learning to act well purely from trial and error, without ever building an internal picture of how the world works — like learning a game by habit rather than by thinking moves ahead. control algorithm, Q-learning became a cornerstone of reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time., and its combination with deep networks in the deep Q-networkdeep Q-network, n.: A method, made famous by DeepMind in 2013–2015, in which a neural network learned to play dozens of Atari video games from screen pixels and the score alone, many at human level. (2013–2015) launched deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images.. The thesis also helped connect temporal-difference methods with dynamic programming in the framework of Markov decision processesMarkov decision process, n.: A mathematical description of a step-by-step decision problem: each step you pick an action, receive a reward and then land in a new situation, drawn at random from odds that depend only on the current situation and your choice..