Skip to content
Last Dictionary
Plane
Deep LearningVision, Speech & Perception

connectionist temporal classification

nounAlso: CTC, CTC loss, blank token

In plain wordsA training method that lets a network learn to transcribe speech or handwriting without being told exactly where each letter or sound begins and ends.

  1. A loss function and decoding scheme for sequence labeling when the alignment between input frames and output labels is unknown, as in speech and handwriting recognition. The network emits, at every input frame, a distribution over the output labels plus a special 'blank' symbol. A many-to-one collapsing map merges repeated labels and deletes blanks, so many frame-level paths correspond to the same transcript; the probability of a transcript is the sum over all paths that collapse to it, computed efficiently by forward–backward dynamic programming, and training maximizes its log-likelihood. CTC assumes frame outputs are conditionally independent given the input and requires the output to be no longer than the input. Decoding uses greedy (best-path) or prefix beam searchbeam search, n.: A search method that keeps several of the best partial answers at each step, rather than only the single best, to find a better overall result without trying every possibility., often with an external language model. It powered LSTMlong short-term memory, n.: A type of recurrent network with a separate memory lane and learned 'gates' that decide what to remember, forget and output, letting it keep track of information over long sequences.-based speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video. and OCR systems in the 2010s and remains in use on its own (for example, to fine-tune wav2vec 2.0 speech encoders) and as an auxiliary loss in hybrid CTC/attention models; transducers and attention-based encoder–decodersencoder–decoder architecture, n.: A two-part model design: one part, the encoder, reads the input and condenses it into an internal summary; the other, the decoder, uses that summary to produce the output. are the main alternatives.

    Train the acoustic encoder with CTC first, then add the attention decoder.

p(y∣x)=∑π∈B−1(y)∏t=1Tp(πt∣x)p(\mathbf{y} \mid \mathbf{x}) = \sum_{\pi \in \mathcal{B}^{-1}(\mathbf{y})} \prod_{t=1}^{T} p(\pi_t \mid \mathbf{x})

Origin

Introduced by Alex Graves, Santiago Fernández, Faustino Gomez and Jürgen Schmidhuber in 'Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks' (ICML 2006). Google reported large error reductions after adopting LSTM–CTC models for speech recognition in 2015.

Related

See also
  • speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video.
  • long short-term memorylong short-term memory, n.: A type of recurrent network with a separate memory lane and learned 'gates' that decide what to remember, forget and output, letting it keep track of information over long sequences.
  • recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood.
  • beam searchbeam search, n.: A search method that keeps several of the best partial answers at each step, rather than only the single best, to find a better overall result without trying every possibility.
  • optical character recognitionoptical character recognition, n.: Software that picks out the letters and words in a picture — a road sign, a handwritten note, a page of an old book — and turns them into real text a computer can copy, search or read aloud.
  • sequence-to-sequence modelsequence-to-sequence model, n.: A model that turns one sequence into another, such as a French sentence into English or speech into text, by first reading the whole input and then writing the output step by step.

Sources

  1. Graves, A., Fernández, S., Gomez, F. & Schmidhuber, J. (2006). Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. Proceedings of ICML 2006.

Mentioned in

optical character recognitionoptical character recognition, n.: Software that picks out the letters and words in a picture — a road sign, a handwritten note, a page of an old book — and turns them into real text a computer can copy, search or read aloud., speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video., ConformerConformer, prop. n.: A neural-network design for speech recognition that combines a transformer, which relates distant parts of a recording, with convolutions, which pick up fine local sound patterns., wav2vecwav2vec, prop. n.: A family of Meta speech models that learn the sounds of language from thousands of hours of unlabeled recordings, so that only minutes of transcribed speech are needed to build a recognizer.