deepfake detection
In plain wordsUsing software, often itself AI, to tell whether a video, photo or voice recording has been faked or generated by AI rather than captured from reality.
The task of determining whether images, video or audio have been synthesized or manipulated by AI — face swaps, lip-sync edits, fully generated scenes, cloned voices — usually with classifiersclassifier, n.: A program or model that sorts inputs into categories, for example flagging an email as spam or a chatbot message as harmful. Many AI safety filters are classifiers. trained on real and fake examples. Detectors look for generation artifacts (frequency-domain traces, blending boundaries, inconsistent lighting or reflections, implausible physiological signals, vocoder artifacts in speech) or learn such cues end to end. Detection is an arms race: classifiers often fail on generators absent from their training data, degrade under compression and re-recording, and can be evaded adversarially, so benchmark accuracy overstates real-world reliability. Practice therefore combines detection with provenance — Content CredentialsContent Credentials, prop. n.: A tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect. and watermarksAI watermarking, n.: Hiding an invisible signal in AI-generated text, images, audio or video so that software can later tell the content was made by AI — like a banknote's watermark, but undetectable to the eye. that establish origin instead of inferring it — and with human verification. Mistakes cut both ways: a false 'fake' verdict lets genuine evidence be dismissed, feeding the liar's dividendliar's dividend, n.: The advantage dishonest people gain once everyone knows fakes are easy to make: they can wave away real evidence against them by claiming it was made by AI..
The bank's audio deepfake detection flagged the caller's voice as synthetic before the transfer went through.
Origin
Research followed the appearance of 'deepfakes' on Reddit in late 2017; early detectors in 2018 exploited cues such as unnatural blinking. Andreas Rössler and colleagues' FaceForensics++ benchmark (2019) and the Deepfake Detection Challenge run by Facebook, AWS, Microsoft and the Partnership on AI (2019–2020), whose winner scored about 65% on unseen videos, showed how hard generalization is.
Related
- deepfakedeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened.
- synthetic mediasynthetic media, n.: Images, video, audio or text created or substantially altered by AI rather than recorded or written directly by people, whether for art, entertainment, business or deception.
- AI watermarkingAI watermarking, n.: Hiding an invisible signal in AI-generated text, images, audio or video so that software can later tell the content was made by AI — like a banknote's watermark, but undetectable to the eye.
- Content CredentialsContent Credentials, prop. n.: A tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect.
- voice cloningvoice cloning, n.: Using AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams.
- liar's dividendliar's dividend, n.: The advantage dishonest people gain once everyone knows fakes are easy to make: they can wave away real evidence against them by claiming it was made by AI.
- speaker verificationspeaker verification, n.: Checking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone.
Sources
- Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J. & Nießner, M. (2019). FaceForensics++: Learning to Detect Manipulated Facial Images. ICCV 2019.
- Dolhansky, B. et al. (2020). The DeepFake Detection Challenge (DFDC) Dataset. arXiv preprint.
Mentioned in
AI watermarkingAI watermarking, n.: Hiding an invisible signal in AI-generated text, images, audio or video so that software can later tell the content was made by AI — like a banknote's watermark, but undetectable to the eye., deepfakedeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened., voice cloningvoice cloning, n.: Using AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams., AI detectorAI detector, n.: Software that scores how likely it is that an AI, not a person, produced some text or media; for writing, its verdicts are wrong often enough that many schools no longer trust them., Content CredentialsContent Credentials, prop. n.: A tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect., content moderationcontent moderation, n.: Checking what people post, or what an AI produces, and removing, limiting or labeling whatever breaks the rules or the law — done by people, by AI, or by both together., influence operationinfluence operation, n.: An organized, often secret campaign to sway public opinion — fake accounts posting propaganda, for instance — which AI can make cheaper by writing posts, translating them and inventing fake personas., liar's dividendliar's dividend, n.: The advantage dishonest people gain once everyone knows fakes are easy to make: they can wave away real evidence against them by claiming it was made by AI., speaker verificationspeaker verification, n.: Checking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone., SynthIDSynthID, prop. n.: Google's system for hiding an invisible watermark in pictures, video, audio and text made by its AI tools, so they can later be recognized as AI-generated., AI handsAI hands, n.: The tendency of early AI image generators to draw hands badly — extra or missing fingers, twisted joints — which became a running joke and a quick way to spot fake images., C2PAC2PA, abbr.: An industry group, and the technical standard it publishes, for attaching tamper-evident records of origin and editing history to digital photos, video, audio and documents.