Sepp Hochreiter

German computer scientist, head of the Institute for Machine Learning at Johannes Kepler University Linz, Austria. His 1991 diploma thesis at the Technical University of Munich, supervised by Jürgen SchmidhuberJürgen Schmidhuber, prop. n.: A German computer scientist who co-invented the LSTM, a neural network with memory that powered much of speech recognition and translation in the 2010s, and who argues that early pioneers go under-credited., gave the first formal analysis of the vanishing gradient problemvanishing gradient problem, n.: A training difficulty in deep or recurrent networks: the learning signal shrinks as it travels back through many layers or time steps, so the early parts of the network barely learn., explaining why recurrent networksrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood. trained by backpropagation through timebackpropagation through time, n.: The way recurrent networks are trained: the network is unrolled into a long chain, one copy per time step, and errors are traced back through every step to adjust the shared weights. fail to learn long-range dependencies. With Schmidhuber he then introduced long short-term memorylong short-term memory, n.: A type of recurrent network with a separate memory lane and learned 'gates' that decide what to remember, forget and output, letting it keep track of information over long sequences. (LSTM, 1997), whose gated memory cell preserves error signals over long intervals; it became the standard sequence model for nearly two decades. Later work includes flat minima, self-normalizing networks (2017), the Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better. for evaluating generative models (2017), modern Hopfield networksHopfield network, n.: A network of on–off units, all connected to one another, that stores patterns as stable states. Given a partial or noisy pattern, it settles into the closest stored one, like recalling a memory from a hint. (2020) and xLSTM (2024), an extended LSTM meant to compete with transformerstransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it., developed with the Linz startup NXAI. He has also applied deep learning to drug discovery and bioinformatics.