DETECT-2B now capable of detecting AI generated music

Jul 18, 2024

In the ever-evolving landscape of AI-generated content, the rise of deepfake technology has posed significant challenges in distinguishing real from fake. At Resemble AI, we’ve made significant advances in detecting deepfakes in speech, and now we’re extending our expertise to music. As the potential for AI voice misuse grows, Resemble AI has proactively implemented safeguards, including our recent commitment to the Voluntary Code of Conduct on the Responsible Development and Management of Advanced Generative AI Systems.

The Rise of AI in Music Creation

AI music generators have gained popularity for their ability to create original compositions based on text prompts. These systems are trained on vast amounts of existing music, learning patterns and styles to generate new content. However, this training process has raised concerns about the use of copyrighted material without permission or compensation to the original artists. In recent news, the music industry has taken a stand against AI music generators, highlighting the complex relationship between technology, creativity, and copyright laws. Major record companies, including Sony Music, Universal Music Group, and Warner Records, have filed federal copyright infringement lawsuits against AI music generators Suno and Udio. This legal action, coordinated by the Recording Industry Association of America (RIAA), alleges that these companies used copyrighted music to train their AI models without proper authorization. The central issue in these lawsuits is whether using copyrighted materials to train AI falls under “fair use,” a concept in copyright law that allows limited use of copyrighted material without permission for purposes such as commentary, criticism, or research. This question has yet to be definitively ruled on by US courts, making this case a potential landmark in shaping the future of AI and copyright law. For a more detailed examination of the lawsuits against Suno and Udio, you can read our in-depth article here.

Deepfake Detection: A Potential Solution?

As AI-generated music becomes more prevalent, there’s a growing need for tools to distinguish between human-created and AI-generated content. Researchers are now working on deepfake detection models for both speech and music. These models aim to identify if a clip is real or fake by analyzing various aspects of the audio.

Our Approach to Deepfake Detection in Music

While our initial DETECT-2B model was trained on general audio data, our latest iteration focuses specifically on music deepfake detection. This specialized training involves curating a dataset rich in musical content, encompassing various genres, instruments, and vocal styles. The model’s architecture remains similar, but the training process now emphasizes the unique characteristics of music, such as rhythm, harmony, and timbre. By fine-tuning on music-specific data, the model becomes adept at identifying the subtle inconsistencies often present in artificially generated or manipulated music. This includes detecting unnatural transitions between notes, improbable instrument combinations, or slight imperfections in vocal synthesis that might be imperceptible to the human ear. The music-focused training also allows the model to better understand the complex layering and production techniques used in modern music, enabling it to differentiate between legitimate studio effects and deepfake artifacts. This specialized approach significantly enhances the model’s accuracy in identifying music deepfakes across a wide range of styles and production qualities.

Current Capabilities and Goals

Our model, currently at an accuracy rate of 94%, can effectively flag AI-generated content on platforms like YouTube and provide record labels with tools to protect their artists. We are continually refining our model, striving for an equal representation of real and fake music in our training data

Potential Use Cases

Content Protection

Platforms like YouTube could use automated methods to flag AI-generated content.

IP Protection

Record Lables and Music associations could employ these detectors to safeguard artists

Consumer Awareness

Listneres could be informed about the origin of the music they are listening too

Future of Music and AI

As AI continues to evolve, the music industry must adapt to new challenges and opportunities. The outcome of the current lawsuits against AI music generators will likely shape the future landscape of music creation, copyright law, and the use of AI in creative industries. Balancing innovation with the rights of artists and copyright holders will be crucial in ensuring a fair and vibrant musical ecosystem. As technology advances, so too must our legal and ethical frameworks to address these new challenges in the world of music and beyond.