Fréchet inception distance
In plain wordsA score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better.
A measure of generative-model quality given by the distance between the distributions of real and generated images in the feature space of a pretrained classifier. Each image set is passed through the Inception-v3 network, the 2,048-dimensional activations of its final pooling layer are summarized by their mean and covariance, and FID is the Fréchet (2-Wasserstein) distance between Gaussians with those statistics. It reflects both fidelity and diversity — mode collapsemode collapse, n.: When a generator gets stuck producing only a few kinds of output — the same face, the same joke — instead of the full variety it was meant to learn. raises it — and tracked human judgment better than the earlier Inception scoreInception score, n.: An early score for image generators that rewards pictures an image-recognition network can label confidently and a collection that covers many different labels; higher is better.. Evaluations typically use 10,000 or 50,000 samples, because the estimate is biased at small sample sizes, and results depend on implementation details such as image resizing, which motivated standardized reference code. Criticisms include the Gaussian assumption, the ImageNet-specific features of Inception-v3, blindness to text–image alignment and weak agreement with human preferences for modern text-to-image models; alternatives use other feature extractors (DINOv2, CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image.) or kernel-based distances, and variants exist for video and audio.
Our FID on COCO dropped from 9.1 to 7.8, but the human raters barely noticed.
Origin
Introduced by Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler and Sepp Hochreiter of Johannes Kepler University Linz in a paper on two time-scale GAN training (posted June 2017; NeurIPS 2017). The distance between normal distributions it uses is named after the mathematician Maurice Fréchet.
Related
- Inception scoreInception score, n.: An early score for image generators that rewards pictures an image-recognition network can label confidently and a collection that covers many different labels; higher is better.
- CLIP scoreCLIP score, n.: A score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions.
- Wasserstein distanceWasserstein distance, n.: A way of comparing two spreads of probability by imagining one as heaps of sand that must be shoveled into the shape of the other: the less shoveling, weighted by distance, the more alike they are.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- human evaluationhuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.
- generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people.
Sources
- Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. & Hochreiter, S. (2017). GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. NeurIPS 2017. arXiv:1706.08500. arxiv.org/abs/1706.08500
Mentioned in
text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds., ImagenImagen, prop. n.: Google's family of AI image generators, first shown in 2022, whose research showed that better understanding of language yields pictures that match descriptions more faithfully., mode collapsemode collapse, n.: When a generator gets stuck producing only a few kinds of output — the same face, the same joke — instead of the full variety it was meant to learn., classifier guidanceclassifier guidance, n.: A way to push a diffusion image generator toward a requested category by consulting a separate image-recognition model at every step, trading variety for images that more clearly show the target., CLIP scoreCLIP score, n.: A score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions., covariancecovariance, n.: A measure of whether two quantities tend to rise and fall together, move in opposite directions, or show no consistent relationship., InceptionInception, prop. n.: A family of Google image-recognition networks, the first called GoogLeNet, whose building blocks examine each patch of an image at several scales at once while keeping computation low., Inception scoreInception score, n.: An early score for image generators that rewards pictures an image-recognition network can label confidently and a collection that covers many different labels; higher is better., perceptual lossperceptual loss, n.: A way of judging how different two images look to a person by comparing what a trained image-recognition network 'sees' in each, instead of comparing pixels one by one., Sepp HochreiterSepp Hochreiter, prop. n.: A German computer scientist who explained why early neural networks struggled to learn long sequences and, with Jürgen Schmidhuber, invented the LSTM network that solved the problem., Wasserstein distanceWasserstein distance, n.: A way of comparing two spreads of probability by imagining one as heaps of sand that must be shoveled into the shape of the other: the less shoveling, weighted by distance, the more alike they are.