Deepfake Detection Tools: Testing on 100 Samples
Journalists tested four popular deepfake detection tools on a set of 100 files: original photos and videos, modern synthetic content from Sora 2.0, Veo 3, ElevenLabs, Midjourney V7, and hybrid content with upscaling. The test accounted for compression and missing EXIF data—common traits in social media sharing.
The methodology included clean files from 2022–2024, audio clones, and images without human faces. Results reveal each tool’s accuracy, false positives, and limitations.
Hive AI Moderation Suite
A cloud API and browser plugin evaluates the likelihood of AI generation on a 0–100% scale using models trained on millions of samples.
In testing, Hive correctly classified 94% of real images but failed in 12% of video cases—mistaking compression artifacts for AI signs. Ideal for quick photo checks in mass-market workflows, but video results need manual verification.
RealityGuard by Sensity
A corporate platform focused on micro-expressions, lighting, biometrics, and audio-visual synchronization. Generates heat maps highlighting suspicious areas.
Best performer among B2B tools: handled audio deepfakes layered onto video effectively. Accuracy drops to 55% on non-human subjects like animals or landscapes.
Pros:
- Detects voice overlays on foreign video footage.
- Visualizes anomalies frame by frame.
Cons:
- Low effectiveness without human faces in frame.
- Not suitable for news content lacking identifiable people.
Content Credentials (C2PA)
The C2PA standard built into Sony, Leica cameras, Android 16, and iOS 19 adds a cryptographic manifest recording capture, editing, and AI usage data. A broken chain indicates tampering.
In testing, 70% of real files lacked metadata due to older devices or messaging apps. Reliable only for content from compatible cameras under strict conditions.
Intel FakeCatcher
Analyzes photoplethysmography—skin color changes caused by blood flow. No need for AI-simulated heart rhythms.
100% accuracy on HD video with close-up facial shots. Fails with masks, glasses, or resolution below 720p. Useless on static images.
Why There’s No One-Size-Fits-All Solution
Detectors lag behind generators because:
- Adversarial AI: Generators actively eliminate known artifacts (e.g., fingers, eye glare).
- Compression: Platforms like Telegram, WhatsApp, and Instagram strip out subtle noise needed for analysis.
- User behavior: A 60% probability is often treated as fact—or ignored—without standardized thresholds.
Key Takeaways
- Hive: 94% accuracy on photos; struggles with compressed video.
- RealityGuard: top choice for face-based video; weak on non-human content.
- C2PA: ideal for new devices; 70% of files lack metadata.
- FakeCatcher: flawless on HD faces; useless at low resolution.
- Combination of tools + manual review is essential.
Recommendations for Use
For developers and fact-checkers:
- Photos: Use Hive + ExifTool for metadata validation.
- Video with speech: Apply RealityGuard or FakeCatcher at high resolution.
- Your own content: Enable C2PA in your camera to ensure verifiable authenticity.
By 2026, detection will require a multi-layered strategy: automation paired with expert judgment.
— Editorial Team
No comments yet.