.avif)
This research presents Proteus, Resemble AI's internal framework for automatically stress-testing its own audio deepfake detector by searching for combinations of ordinary audio processing that can flip a verdict. Presented at the International Symposium on Synthetic Media Attribution and Detection (ISSMAD) 2026.
Overview
Most deepfake detectors get graded on clean lab audio. Real audio doesn't stay clean, it gets compressed for a phone call, run through VoIP, layered with background noise, or just uploaded to a platform that re-encodes it. Any one of those ordinary steps can be enough to change a detector's verdict, and until now, finding out which combinations actually do that meant a human analyst manually trying attacks one at a time.
Proteus automates that search. It systematically composes chains of everyday audio transformations, codec transcoding, added noise, reverb, VoIP simulation, and more, and tests which combinations fool Resemble's production detector while keeping the audio intelligible and the speaker recognizable. The most notable finding: it's much easier to make genuine audio look fake than to make fake audio look genuine, the exact asymmetry that enables what's known as the "liar's dividend," where bad actors discredit real evidence by claiming it's AI-generated.
Download the full research PDF on arXiv.
Methodology
Proteus works like a black-box penetration test built specifically for audio detectors, it doesn't need access to the detector's internals, just the ability to send it audio and see the score that comes back.
It draws from a library of 35 augmentation types across 11 categories: codecs like MP3 and Opus, added noise, reverb, filtering, compression, distortion, environmental effects, music overlay, silence injection, and VoIP simulation, roughly 110 variants once you count different settings for each. Before any chain gets tested against the detector, it has to pass a quality gate: the words spoken must still be recognizable (checked via automatic transcription), and the speaker's voice must still sound like the same person (checked via speaker-similarity scoring). Chains that distort the audio too much to be a realistic attack get thrown out before they're even tested.
To search this space, the team used two strategies.
- The breadth-first search, tries every augmentation at each step and keeps the ones that shift the detector's score the most, then builds on those in the next round. It's thorough but expensive: testing every combination three steps deep would mean over a million candidates per audio sample.
- A Q-learning agent is designed to learn which augmentation sequences tend to work well together rather than testing everything, so it can search deeper combinations of audio augmentations (called “chains”) more efficiently. That second approach is a proposed extension the team is still actively evaluating, so the results below come specifically from the breadth-first search runs.
Results
The results show a clear asymmetry, and it cuts in the direction that matters most for security. Every one of the top 100 highest-impact chains found in this run targeted genuine recordings, pushing real audio toward a "spoof" verdict. None of them made fake audio pass convincingly as real which is the failure mode attackers actually want, a cloned voice sounding genuine to the system in a CEO-fraud call or a fraudulent account verification, and this study didn't find a reliable way to produce it.
The tradeoff shows up on the other side. We ran the search against 17,405 candidate augmentation chains; after the quality gate rejected 72% of them for distorting speech or changing the apparent speaker, 4,847 remained, and the strongest of those, layering synthetic reverb and then Opus compression, shifted a genuine recording's score by 0.99, enough to flip a real recording's verdict to "spoof."
Depth mattered too. In one chain, automatic gain control alone barely moved the score (+0.13), but it conditioned the signal so that the MP3 compression applied right after produced the largest single-step shift the team observed (+0.51), evidence that these vulnerabilities often come from how augmentations interact, not just what each one does alone.
Why this matters: This is the right failure mode for a security detector to have. Missing an actual deepfake, letting a cloned voice pass as real in a CEO-fraud call or a fraudulent account verification, is the outcome with real financial and reputational cost, and this study didn't find a reliable way to make that happen. The tradeoff is that genuine audio can occasionally get flagged as synthetic under specific processing conditions. In practice, that's a reliability problem more than a security one: a legitimate caller flagged in a call center, a meeting bot misreading a compressed or noisy line, a telephony system throwing an unnecessary alert. There's also a narrower "liar's dividend" risk to call out, someone deliberately manipulating real audio to get it flagged as fake, but the bigger day-to-day cost is false positives showing up in production deployments. Proteus findings don't just sit in a paper either: high-shift chains get folded back into the detector's training data, and Proteus re-runs against the updated model to confirm the fix worked and check for new weak points, a closed loop between finding a vulnerability and fixing it.
How Resemble AI turns these findings into a better model: Every vulnerability Proteus surfaces feeds directly back into development. When Proteus finds a weakness - say, a specific codec that reliably flips a verdict - our engineers fold those failing samples into the training data and retrain the detector. This lets us find and fix blind spots on our own terms, hardening the model against real-world audio conditions before those weaknesses ever reach a customer.
How to cite this paper
APA Müller, N. M., Tirumala Bukkapatnam, A., & Ahmed, Z. (2026). Proteus: Automated adversarial robustness testing for audio deepfake detectors. arXiv preprint arXiv:2606.29544.
BibTeX @article{muller2026proteus, title={Proteus: Automated Adversarial Robustness Testing for Audio Deepfake Detectors}, author={Müller, Nicolas M. and Tirumala Bukkapatnam, Aditya and Ahmed, Zohaib}, journal={arXiv preprint arXiv:2606.29544}, year={2026}}
Sources
1. Li, Chen, and Wei. Measuring the robustness of audio deepfake detectors. arXiv:2503.17577, 2025.
2. Wu et al. CLAD: Robust audio deepfake detection against manipulation attacks with contrastive learning. arXiv:2404.15854, 2024.
3. Shi et al. Benchmarking audio deepfake detection robustness in real-world communication scenarios. arXiv:2504.12423, 2025.
4. Wang et al. ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale. Proc. ASVspoof Workshop, 2024.
5. Müller et al. DeePen: Penetration testing for audio deepfake detection. arXiv:2502.20427, 2025.
6. Radford et al. Robust speech recognition via large-scale weak supervision. ICML, 2023.
7. Chesney and Citron. Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review, 107, 2019.
8. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, 1994.
9. Sutton and Barto. Reinforcement Learning: An Introduction. MIT Press, 2nd ed., 2018.
10. Solak. The M-AILABS speech dataset. 2019.
11. Müller et al. MLAAD: The multi-language audio anti-spoofing dataset. IJCNN, 2024.
FAQs
Can audio deepfake detectors be fooled by simple audio processing? Yes. Resemble's Proteus framework found that chains of everyday transformations, like reverb, compression, or VoIP simulation, can shift a detector's score enough to flip a verdict, in some cases by up to 99%.
Are genuine recordings or fake ones easier to manipulate? Genuine recordings, by a wide margin. All of the top 100 highest-impact chains found by Proteus targeted authentic audio, pushing it toward a "spoof" verdict, rather than the reverse.
What does Resemble AI do with these findings? High-shift chains get added back into the detector's training data so it learns to stay accurate despite those transformations. Proteus then re-runs against the updated model to confirm the fix and surface any new weaknesses.
