Tomas Mikolov

Czech computer scientist known for neural language models and embeddings. His doctoral work at Brno University of Technology (completed 2012) showed that recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood. language modelslanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. could outperform n-gram modelsn-gram model, n.: An early type of language model that predicts the next word from just the few words before it, using counts of how often each word followed those words in a large collection of text. in speech recognition, and his RNNLM toolkit spread the approach; he also clipped gradients to tame exploding gradientsexploding gradient problem, n.: A training failure in which the learning signal grows out of control as it passes back through a network, so the weights jump wildly and training breaks down., an early form of gradient clippinggradient clipping, n.: A safety limit on how big a single training update can be. If the computed adjustment is unusually large, it is scaled down so that one bad batch cannot wreck the model.. At Google he led the creation of word2vecword2vec, prop. n.: A 2013 Google technique that learns a list of numbers for each word by training a very small network to predict which words appear near it across billions of words of text. (2013, with Kai Chen, Greg Corrado, Jeff DeanJeff Dean, prop. n.: An American computer scientist who built much of Google's computing infrastructure, co-founded the Google Brain deep-learning team and was Google's chief scientist before leaving in 2026 to co-found a startup. and Ilya SutskeverIlya Sutskever, prop. n.: A Russian-born Israeli-Canadian computer scientist who co-built the AlexNet image recognizer, helped found OpenAI and lead its research until 2024, and now heads the startup Safe Superintelligence Inc.), whose skip-gram and continuous bag-of-words models learned word embeddingsword embedding, n.: A list of numbers standing for a word, learned so that words used in similar ways get similar lists. It lets a computer measure that 'cat' is closer to 'dog' than to 'car'. from billions of words in hours and showed that vector arithmetic captures analogies such as king − man + woman ≈ queen. At Facebook AI Research, which he joined in 2014, he co-created fastText for efficient text classification and subword embeddings; in 2020 he moved to the Czech Institute of Informatics, Robotics and Cybernetics at the Czech Technical University in Prague. A word2vec paper received the NeurIPS Test of Time Award in 2023.