.avif)
This research examines whether DETECT-3B-Omni's detection accuracy holds steady across content and speaker demographics, work conducted in collaboration with Deutsche Telekom to support GDPR-compliant deployment for real-time call monitoring. This paper is published as an arXiv preprint in July 2026, as well as peer reviewed and presented at the International Symposium on Synthetic Media Attribution and Detection (ISSMAD) 2026 in Mountain View, California, co-Sponsored by IEEE Signal Processing Society and Google.
Overview
A detector monitoring live phone calls for deepfake audio has to justify why it's allowed to process that call at all. GDPR only permits automated processing that's strictly necessary for a stated purpose, so if the detector's accuracy changes depending on the topic of the conversation, or a caller's gender, age, or accent, it's arguably using more information than its job requires.
This study is the evidence that DETECT-3B-Omni holds steady across every split tested, using equivalence testing, the same statistical approach used to prove two clinical treatments perform the same, not just that a difference wasn't detected.
Download the full research PDF on arXiv.
Methodology
The team didn't just test whether the detector works, they tried to break the assumption that it works fairly.
They wrote 640 sentences split evenly into two categories: benign, everyday business language like invoice confirmations and meeting scheduling, and malicious, social-engineering scripts like urgent payment demands and credential phishing. Each sentence was recorded by 8 real speakers, balanced across gender, spanning ages 20 to 55, and drawn from 30 states. Every sentence was also cloned using 8 different open-source voice-cloning models, including Chatterbox, XTTS2, F5-TTS, and MegaTTS3, producing an equal number of real and fake recordings: 10,240 total.
Dataset composition
From there, the team ran seven comparisons, four testing whether content changes the outcome and three testing demographics (gender, age, and East vs. West of the Mississippi), plus a random 50/50 split with no real content or demographic difference, used as a sanity-check baseline for how much variation to expect from sample size alone.
Results
The results paint a clear picture: DETECT-3B-Omni classifies audio by where it comes from (human or machine), not by what is said or who says it. All seven comparisons confirmed this. The model doesn't lean on a speaker's age, gender, accent, or the content of the call. We tested this with equivalence testing, the same approach used in clinical trials, but at a stricter 99% confidence level where most clinical trials use 90% or 95%. On the full balanced set of 10,240 recordings, the detector reached 98.3% overall accuracy, separating real from cloned audio cleanly across every split.
Why this matters: For anyone deploying deepfake detection on live calls, this is the difference between a tool you can defend to a regulator and one you can't. GDPR requires that automated systems process only what's strictly necessary for their stated purpose. A detector that performs differently depending on the topic of a call, or on a caller's gender, age, or accent, is arguably processing more than it needs to, and doing so unevenly. This study, peer reviewed by a IEEE and Google co-organized conference committee, is evidence that DETECT-3B-Omni doesn't do that: it can monitor calls for AI-generated speech without your privacy posture depending on what's actually said or who's on the line.
Limitations
- Covers native US English speakers only; it doesn't speak to performance across other languages or non-US accents.
- At smaller sample sizes (1,280 per group), confidence intervals can exceed the 2-point margin from sampling noise alone, part of why the full-dataset comparisons matter more than any single subgroup slice.
How to cite this paper
APA Müller, N. M., Tirumala Bukkapatnam, A., Schnieders, D., & Ahmed, Z. (2026). DETECT-3B-Omni is agnostic of content and demographics. arXiv preprint arXiv:2607.03418.
BibTex @article{muller2026detect3bomni,title={DETECT-3B-Omni is Agnostic of Content and Demographics},author={Müller, Nicolas M. and Tirumala Bukkapatnam, Aditya and Schnieders, Dominik and Ahmed, Zohaib},journal={arXiv preprint arXiv:2607.03418},year={2026}}
Sources
- Resemble AI. Chatterbox-TTS: Open-source text-to-speech models. 2025.
- Hu et al. Qwen3-TTS technical report. arXiv:2601.15621, 2026.
- Casanova et al. XTTS: a massively multilingual zero-shot text-to-speech model. INTERSPEECH, 2024.
- Eskimez et al. E2 TTS: Embarrassingly easy fully non-autoregressive zero-shot TTS. arXiv:2406.18009, 2024.
- Chen et al. F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching. arXiv:2410.06885, 2024.
- Jiang et al. MegaTTS 3: Sparse alignment enhanced latent diffusion transformer for zero-shot speech synthesis. arXiv:2502.18924, 2025.
- Bielicki et al. The CAP-IT randomized clinical trial. JAMA, 326(17), 2021.
- Agnelli et al. Oral apixaban for the treatment of acute venous thromboembolism. NEJM, 369(9), 2013.
- Boyd et al. SECOND-LINE: a randomised, open-label, non-inferiority study. The Lancet, 381(9883), 2013.
- Müller et al. Does audio deepfake detection generalize? INTERSPEECH, 2022.
FAQs
Does deepfake detection accuracy depend on what's being said? No. Across four separate comparisons, including a maximum-contrast test designed to stack the odds toward finding a bias, the accuracy difference between benign and malicious content stayed below 1 percentage point.
Does the detector perform differently across gender, age, or region? No. Male vs. female, under-40 vs. 40-and-older, and Eastern vs. Western US speakers all produced accuracy differences within the study's ±2 percentage point equivalence margin at 99% confidence.
Why does this matter for GDPR compliance? A detector monitoring live calls must process only what's strictly necessary to detect AI-generated audio. If its accuracy shifted based on conversation content or speaker demographics, it would effectively be using more personal data than its purpose requires.
