WaveNet
In plain wordsA DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech.
An autoregressive modelautoregressive model, n.: A model that produces a sequence one piece at a time, each new piece depending on the pieces already produced — like writing a sentence word by word without going back. of raw audio: a neural network that builds a waveform one sample at a time, drawing each value from a distribution that depends on all the values before it. Audio at 16,000 samples per second was compressed to 256 levels with μ-law companding, so each step is a 256-way softmaxsoftmax, n.: A formula that turns a list of raw scores into probabilities — all positive and adding up to one — with the biggest share going to the highest score. classification. To see far back, WaveNet stacks dilated causal convolutionsconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location., whose gaps double from layer to layer so that the receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to. grows exponentially with depth, together with gated activations and residualresidual connection, n.: A shortcut in a neural network that adds a block's input straight to its output, so each block only has to learn a correction. It is what makes networks hundreds of layers deep trainable. and skip connections. Trained on piano recordings it made music, and given a speaker-identity input it spoke in many voices; its headline result, though, was text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps. that listeners rated more natural than the leading concatenative and parametric synthesizers in English and Mandarin. Generating one sample at a time was far too slow for products until Parallel WaveNet (2017) distilled it into a feed-forward model deployed in Google Assistant. WaveNet also served as the vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear. of Tacotron 2 and influenced autoregressive models of other signals.
WaveNet showed that modeling the raw waveform, one sample at a time, could beat decades of hand-built synthesis.
Origin
Introduced by Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior and Koray Kavukcuoglu of DeepMind in a September 2016 blog post and arXiv paper, building on the same group's PixelCNN image models.
Related
- text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps.
- vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.
- autoregressive modelautoregressive model, n.: A model that produces a sequence one piece at a time, each new piece depending on the pieces already produced — like writing a sentence word by word without going back.
- convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.
- receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to.
- generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them.
Sources
- van den Oord, A. et al. (2016). WaveNet: A Generative Model for Raw Audio. arXiv:1609.03499. arxiv.org/abs/1609.03499
- van den Oord, A. et al. (2018). Parallel WaveNet: Fast High-Fidelity Speech Synthesis. ICML 2018.
Mentioned in
text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps., DeepMindDeepMind, prop. n.: A London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind., dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost., gated linear unitgated linear unit, n.: A layer design in which one half of the computation acts as a set of dials, each between off and on, deciding how much of the other half passes through., Koray KavukcuogluKoray Kavukcuoglu, prop. n.: A Turkish-born computer scientist who co-authored many of DeepMind's landmark papers and in 2026 took charge of Google DeepMind as its senior vice president., Oriol VinyalsOriol Vinyals, prop. n.: A Spanish computer scientist who co-created key neural methods for translation and image captioning, led DeepMind's StarCraft-playing AlphaStar, co-led Google's Gemini models, and in 2026 co-founded the startup Discovery Loop., vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.