Deepfake detection is winning the demo and losing the deployment
GPT-5.2 hit 97.9% on synthetic faces in a preprint posted October 19 — yet a new forensics benchmark found false-positive rates up to 1.00, and watermarking turned out to hurt detection.
- In a preprint, multimodal LLMs detected synthetic facial images by spotting structural or physical inconsistencies, with GPT-5.2 reaching 97.9% accuracy in pairwise comparisons. Self-reported m:…2458
- In a preprint, the authors found that SynthID watermarking reduced MLLM detection accuracy by approximately 10%. Self-reported m:…2458
- In a preprint, GazeCLIP outperformed state-of-the-art models by 6.56% accuracy and 5.32% AUC on novel generative models. Self-reported m:…1697
- In a preprint, the BG-REAL benchmark found false-positive rates of 0.57 to 1.00 across evaluated baselines, with re-encoding artifacts acting as a shared shortcut risk across TruFor, MVSS-Net, and HiFi-Net. Self-reported m:…9207
- In a preprint, the authors found that supervised neural AI-text detectors perform well in-distribution but degrade under unseen generators and adversarial attacks — and conclude the primary weakness is robustness, not peak accuracy. Self-reported m:…2391
Multimodal models as forensic analysts
Ask a vision-language model whether a face is fake and it can answer — by spotting structural or physical inconsistencies, not pixel noise. In a preprint posted October 19, researchers benchmarked multimodal LLMs as synthetic-face detectors and report GPT-5.2 reaching 97.9% accuracy in pairwise comparisons. The headline number is real, and it is also the best-case number. m:…2458
Then the tension: SynthID watermarking — the provenance technology meant to label AI-made images — reduced MLLM detection accuracy by approximately 10%. My read: the watermark and the forensic signal may be fighting over the same artifacts. Watermarking for provenance could be blinding the detectors meant to catch fakes, which makes this a standards question, not just a research one. m:…2458
Gaze as a tell
A second paper goes after a specific signal: the eyes. GazeCLIP's gaze-aware image encoder mines forgery embeddings across both appearance and gaze domains, and the authors report it outperforms state-of-the-art models by 6.56% accuracy and 5.32% AUC on novel generative models. The novel-generator part is the one that matters for deployment — attackers do not use last year's generator, so a detector that only wins on familiar fakes is a demo, not a defense. m:…1697
GazeCLIP also pairs the visual encoder with a Language Refinement Encoder that sharpens vision-language matching through an adaptive-enhanced word selector — the forgery signal is cross-modal, not just pixels. The direction of travel is clear: detectors are getting more specific about where they look. m:…1697
The false-positive crisis
Zoom out from faces to image forensics generally and the picture darkens. BG-REAL, a new benchmark built on 7,000 real-anchored samples for background-manipulation detection, found false-positive rates from 0.57 to 1.00 across evaluated baselines. At the top end, the detector cries fake on every real image it sees. m:…9207
The shared failure mode: re-encoding artifacts act as a shortcut that TruFor, MVSS-Net, and HiFi-Net all lean on, so ordinary recompression reads as manipulation. For anyone deploying forensics in a pipeline, that is the number to internalize — a detector that false-positives on recompressed real images will flag most of the internet. m:…9207
Text detectors degrade too
The same unevenness shows up in text. IndicDetect's evaluation of AI-generated-text detectors across Hindi, Telugu, and Tamil found that supervised neural detectors perform well in-distribution but degrade considerably under unseen generators and adversarial attacks, with Hindi degrading most. The authors' conclusion is blunt: in these settings, the primary weakness is robustness, not peak accuracy. Peak accuracy is what gets quoted; robustness is what gets attacked. m:…2391
Detection is uneven
The week's results do not tell one story — they tell two. In narrow, favorable evaluations, detectors hit 97.9%; under distribution shift, adversarial pressure, or ordinary recompression, they collapse to coin-flip or worse. The practical implication, and this is my synthesis rather than any paper's claim: buy detection for the setting you can actually evaluate, instrument it for drift, budget for false positives, and do not let a benchmark number substitute for a human review step where the stakes are real. m:…2458 m:…9207 m:…2391
What a deployment-ready posture looks like follows from the papers' own findings: evaluate on novel generators, not last year's; test text detectors under adversarial perturbation in every language you ship; and measure false positives on recompressed real images before quoting an accuracy number. Peak accuracy is a lab number. Robustness is the product number. m:…1697 m:…2391 m:…9207
What is still unresolved
- Whether the SynthID-vs-detection tension generalizes beyond the tested MLLMs and watermark settings is not established in the cited claims.
- BG-REAL covers background manipulation; whether the re-encoding shortcut also drives false positives in face-specific detectors is open.
- IndicDetect tested three languages; robustness in other low-resource languages is not covered.
- All four papers are preprints; none has passed peer review.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.