Jascha Sohl-Dickstein
In plain wordsAn American scientist who in 2015, with colleagues, introduced diffusion models, the technique behind most modern AI image and video generators, by borrowing ideas from physics.
American machine-learning researcher who introduced diffusion probabilistic modelsdiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static. in 2015. Trained in physics and theoretical neuroscience (PhD, University of California, Berkeley), he proposed, with Eric Weiss, Niru Maheswaranathan and Surya Ganguli at Stanford, a generative model inspired by nonequilibrium thermodynamics: a fixed process gradually adds noise to data until only noise remains, and a neural network learns to reverse it step by step. Little used at first, the idea became the basis of modern image, audio and video generation after Jonathan Ho and colleagues' DDPM (2020) and the score-basedscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step. stochastic-differential-equation formulation he co-authored with Yang Song and others (2021). As a principal scientist at Google BrainGoogle Brain, prop. n.: Google's deep-learning research team from 2011 to 2023, where the transformer — the design behind today's chatbots — and the TensorFlow software were created. and then Google DeepMind he also worked on infinitely wide neural networks, learned optimizers and the BIG-bench benchmark, and co-authored the levels of AGIlevels of AGI, n.: A scheme from Google DeepMind researchers for grading AI by how well it performs and how broad its skills are, from emerging to superhuman, much as self-driving cars are graded by levels. framework (2023). He joined AnthropicAnthropic, prop. n.: An American AI company that makes the Claude chatbot and models. It was founded in 2021 by former OpenAI staff and puts particular emphasis on AI safety research. in 2024.
Related
- diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
- score-based generative modelscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step.
- denoising diffusion probabilistic modeldenoising diffusion probabilistic model, n.: The 2020 recipe that made diffusion image generators work well: add noise to pictures over a thousand small steps, then train a network to guess the noise so it can be removed.
- generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them.
- noise schedulenoise schedule, n.: The plan for how much static is mixed into training examples at each step of a diffusion model, from almost none to total noise; its shape affects quality and speed.
Sources
- Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N. & Ganguli, S. (2015). Deep Unsupervised Learning using Nonequilibrium Thermodynamics. ICML 2015.
- Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S. & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. ICLR 2021.