DETECT-2B: Our new Foundation model with support for Multilingual Deepfake Detection

Jun 28, 2024

Today, we’re unveiling DETECT-2B, our advanced foundation model that revolutionizes multilingual deepfake detection. As the potential for AI voice misuse grows, Resemble AI has proactively implemented safeguards, including our recent commitment to the Voluntary Code of Conduct on the Responsible Development and Management of Advanced Generative AI Systems. DETECT-2B represents a significant leap forward in combating the rising threat of deepfakes, particularly in critical areas such as government communications, election integrity, and public discourse. This groundbreaking solution sets new benchmarks in deepfake detection technology, reinforcing the pillars of our democratic institutions and paving the way for more secure digital interactions across languages.

DETECT-2B isn’t just another deepfake detector – it’s a leap forward in AI security. With an impressive accuracy rate exceeding 94% across more than 30 languages, DETECT-2B stands as a formidable guardian against AI-generated audio fraud. This multilingual capability is crucial in our globalized world, especially when combating misinformation that can spread across borders in seconds.

Key Features

  • Multilingual Mastery: Supporting over 30 languages, it’s truly a global solution. This feature is indispensable for international organizations, multinational corporations, and governments dealing with multilingual constituencies.
  • Adaptable Architecture: Utilizes an ensemble of sub-models for robust performance across various deepfake generation methods. As deepfake technology evolves, DETECT-2B is designed to stay ahead of the curve.
  • Lightning-Fast Processing: In just 200 milliseconds, DETECT-2B can analyze and classify audio as real or fake. This speed is critical in time-sensitive scenarios, such as live political debates or breaking news situations.
  • Scalability: Capable of handling large volumes of audio data, making it suitable for analyzing extensive media archives or high-traffic platforms.

DETECT-2B Language Analysis

A Focus on Adaptation

DETECT-2B is at the forefront of this evolving landscape, constantly adapting to new challenges. In the coming years, we anticipate a cat-and-mouse game between deepfake creators and detection technologies, driving rapid innovation on both sides. DETECT-2B is poised to stay ahead through continuous research and development, focusing on even more sophisticated neural network architectures, broader language coverage, and enhanced real-time processing capabilities. At the heart of DETECT-2B lies a sophisticated blend of advanced AI techniques:

  • Mamba-SSM Integration: Leveraging State Space Models for enhanced sequence modeling and subtle artifact detection. This allows DETECT-2B to identify even the most convincing deepfakes that might fool human ears.
  • Self-Supervised Learning: Pre-trained models like Wav2Vec2 enable language-agnostic feature detection. This approach allows DETECT-2B to generalize well across different languages and accents.
  • Efficient Fine-Tuning: Achieve state-of-the-art performance with a lightweight, deployable model. This efficiency ensures that DETECT-2B can be easily integrated into existing systems without requiring massive computational resources.
  • Ensemble Approach: By combining multiple sub-models, DETECT-2B can capture a wide range of deepfake indicators, from low-level acoustic features to higher-level sequential patterns.

Future Work

With the release of DETECT-2B, Resemble AI continues to push the boundaries of what’s possible in deepfake detection, but our work is far from over. As generative AI capabilities advance, so must our detection and prevention strategies. We’re developing a user-friendly, web-based dashboard that will allow customers to easily upload, analyze, and manage audio content, making it simpler for non-technical users to interpret results and take action against potential deepfakes. This alongside realtime integrations into video conferencing software, make our deepfake detection stack an easy deployment option for enterprises.

Building on our existing Neural Speech Watermarker, we’re developing more sophisticated watermarking techniques that are even more resistant to manipulation and can persist through various audio transformations. This advanced watermarking, combined with expanded language support and improved processing speed, will provide an even more comprehensive solution for audio content integrity. We’re also exploring advanced machine learning techniques to make DETECT-2B more adaptable to new deepfake generation methods as they emerge, ensuring the tool remains effective against evolving threats.

As we navigate the complex intersection of technology, politics, and society, tools like DETECT-2B become not just useful, but essential. They safeguard the authenticity of our public discourse, the integrity of our democratic processes, and the trust that forms the foundation of our social interactions. With DETECT-2B, our deepfake detection dashboard, Google Meet integration, and advanced watermarking solutions, Resemble AI is committed to providing the most comprehensive and effective suite for ensuring audio integrity in the AI era. Together, we can work towards a future where AI is used responsibly and transparently, maintaining trust and authenticity in our digital communications.

More From This Category

Our Commitment to Consent

Our Commitment to Consent

Remember when creating a synthetic voice meant hours in a studio, carefully recording every syllable? Now, with a few clicks, you can clone anyone's voice. It's mind-blowing tech. But with great power comes great responsibility. At Resemble, we've always believed that...

read more
Introducing ‘Edit’ by Resemble AI: Say No More Beeps

Introducing ‘Edit’ by Resemble AI: Say No More Beeps

In audio production, mistakes are inevitable. You’ve wrapped up a recording session, but then you notice a mispronounced word, an awkward pause, or a phrase that just doesn’t flow right. The frustration kicks in—do you re-record the whole segment, or do you spend...

read more
Introducing Resemble Identity & Audio Intelligence

Introducing Resemble Identity & Audio Intelligence

We're excited to unveil two groundbreaking models designed to revolutionize your interaction with audio: Resemble Identity and Resemble Audio Intelligence. These tools enhance speaker recognition, real-time analysis, voice-based authentication, and more. Resemble...

read more