Kyunghyun Cho
In plain wordsA South Korean computer scientist who co-invented the encoder–decoder approach to neural machine translation and the gated recurrent unit, and co-wrote the paper that introduced attention.
South Korean computer scientist, a professor at New York University since 2015 who has also directed frontier research at Genentech's Prescient Design. As a postdoctoral researcher with Yoshua BengioYoshua Bengio, prop. n.: A Canadian computer scientist who helped found deep learning, including ideas behind today's language models, and who has become a leading voice on the risks of advanced AI. in Montreal he led the 2014 paper that introduced the RNN encoder–decoder for machine translationmachine translation, n.: Software that renders writing or speech in one language into another language, as online translation services and phone translation apps do. (see encoder–decoder architectureencoder–decoder architecture, n.: A two-part model design: one part, the encoder, reads the input and condenses it into an internal summary; the other, the decoder, uses that summary to produce the output.) together with a simplified gated unit, the gated recurrent unitgated recurrent unit, n.: A simpler cousin of the LSTM: a recurrent network unit with two learned gates that decide how much of its memory to keep and how much to overwrite at each step. (GRU). Later that year he co-authored, with Dzmitry BahdanauDzmitry Bahdanau, prop. n.: A Belarusian computer scientist who, as a student in Yoshua Bengio's lab in 2014, was lead author of the paper that introduced attention, the mechanism later at the core of the transformer. and Bengio, the paper that added an attentionattention, n.: A way for an AI model to choose what to focus on. For each word it processes, the model scores how relevant every other word is and draws information mostly from the relevant ones. mechanism, letting the decoder consult all encoder states rather than a single fixed vector. With the sequence-to-sequencesequence-to-sequence model, n.: A model that turns one sequence into another, such as a French sentence into English or speech into text, by first reading the whole input and then writing the output step by step. work of Google researchers, these papers founded neural machine translation, which displaced phrase-based statistical systems in commercial translators within about two years. His later research spans natural language processing and machine learning for health and science. He received the Samsung Ho-Am Prize in Engineering in 2021.
Related
- gated recurrent unitgated recurrent unit, n.: A simpler cousin of the LSTM: a recurrent network unit with two learned gates that decide how much of its memory to keep and how much to overwrite at each step.
- encoder–decoder architectureencoder–decoder architecture, n.: A two-part model design: one part, the encoder, reads the input and condenses it into an internal summary; the other, the decoder, uses that summary to produce the output.
- attentionattention, n.: A way for an AI model to choose what to focus on. For each word it processes, the model scores how relevant every other word is and draws information mostly from the relevant ones.
- sequence-to-sequence modelsequence-to-sequence model, n.: A model that turns one sequence into another, such as a French sentence into English or speech into text, by first reading the whole input and then writing the output step by step.
- machine translationmachine translation, n.: Software that renders writing or speech in one language into another language, as online translation services and phone translation apps do.
- Dzmitry BahdanauDzmitry Bahdanau, prop. n.: A Belarusian computer scientist who, as a student in Yoshua Bengio's lab in 2014, was lead author of the paper that introduced attention, the mechanism later at the core of the transformer.
Sources
- Cho, K. et al. (2014). Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. EMNLP 2014.
- Bahdanau, D., Cho, K. & Bengio, Y. (2015). Neural Machine Translation by Jointly Learning to Align and Translate. ICLR 2015.
Mentioned in
Dzmitry BahdanauDzmitry Bahdanau, prop. n.: A Belarusian computer scientist who, as a student in Yoshua Bengio's lab in 2014, was lead author of the paper that introduced attention, the mechanism later at the core of the transformer.