Skip to content
Last Dictionary
Plane
Vision, Speech & PerceptionMathematics & Statistics

spectrogram

nounAlso: sonogram, short-time Fourier transform, STFT, time–frequency representation, voiceprint

In plain wordsA picture of sound that shows time from left to right, pitch from bottom to top, and loudness as brightness or color, so that speech and music can be seen as patterns.

  1. A time–frequency map of a signal recording, for each short slice of time, how much energy falls at each frequency, usually computed as the squared magnitude of the short-time Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors. (STFT). The signal is cut into short overlapping frames (for speech typically 20–40 milliseconds long, every 10 milliseconds); each frame is multiplied by a window function and Fourier-transformed; and the resulting energies form a two-dimensional array with time on one axis and frequency on the other, normally shown on a logarithmic (decibel) scale. Frame length sets a trade-off between time and frequency resolution: long windows resolve individual harmonics (narrowband spectrograms), short ones resolve rapid events (wideband spectrograms, which show speech formants clearly). Spectrograms discard phase, so turning them back into sound requires phase estimation or a vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.. In machine learning, spectrograms, especially mel spectrogramsmel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio., let audio be processed by image-style networks such as CNNsconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. and transformersvision transformer, n.: A way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence., and they serve as inputs for speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video. and sound event detectionsound event detection, n.: Teaching computers to recognize everyday sounds in a recording, such as a dog barking, glass breaking or a siren, and to say exactly when each one starts and stops. and as intermediate outputs in text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps..

    The spectrogram shows mains hum at 50 Hz and its harmonics, so filter those out before training.

S(m,k)=∣∑n=0L−1x[n+mH] w[n] e−i2πkn/L∣2S(m,k) = \left|\sum_{n=0}^{L-1} x[n + mH]\,w[n]\,e^{-i 2\pi k n / L}\right|^2

Origin

The sound spectrograph was developed at Bell Telephone Laboratories during the Second World War by Ralph Potter and colleagues and described publicly by Walter Koenig, Hugh Dunn and Lawrence Lacy in 1946; Potter, George Kopp and Harriet Green's book Visible Speech (1947) taught people to read speech spectrograms. In astronomy, 'spectrogram' had earlier meant a photograph of a spectrum.

Usage note

In the formula, x is the signal, w a window of length L, H the hop between frames, m the frame index and k the frequency bin. Forensic 'voiceprint' identification by eye from spectrograms, promoted in the 1960s, has been widely criticized as lacking scientific validation.

Related

See also
  • mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio.
  • MFCCMFCC, abbr.: A short list of numbers, usually about 13, that summarizes the shape of a brief slice of sound in a way that mimics human hearing; for decades the standard input to speech recognizers.
  • Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors.
  • vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.
  • speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video.
  • sound event detectionsound event detection, n.: Teaching computers to recognize everyday sounds in a recording, such as a dog barking, glass breaking or a siren, and to say exactly when each one starts and stops.

Sources

  1. Koenig, W., Dunn, H. K. & Lacy, L. Y. (1946). The Sound Spectrograph. Journal of the Acoustical Society of America 18(1), 19–49.
  2. Potter, R. K., Kopp, G. A. & Green, H. C. (1947). Visible Speech. D. Van Nostrand.

Mentioned in

mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio., Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors., MFCCMFCC, abbr.: A short list of numbers, usually about 13, that summarizes the shape of a brief slice of sound in a way that mimics human hearing; for decades the standard input to speech recognizers., sound event detectionsound event detection, n.: Teaching computers to recognize everyday sounds in a recording, such as a dog barking, glass breaking or a siren, and to say exactly when each one starts and stops., source separationsource separation, n.: Pulling apart a mixture of sounds into its separate parts, such as isolating one voice in a noisy party or the vocals, drums and bass in a finished song., vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.