Introducing the Deepfake Detection Dashboard

Q

Using Synthetic Voices for Interactive Experiences – Parallel Effect

Jul 6, 2021

Creating an alternate reality persona seems like an arduous task, but it is precisely what Parallel Effect accomplished with Resemble AI’s voice cloning technology earlier this year.

The Parallel Effect project works with a collective of artists, scholars, and technology specialists who pool their talents and ideas to explore multiple plausible alternatives to the “present” we’re living in. It was no surprise that this concept would be showcased at this year’s Melbourne Knowledge Week.

Melbourne Knowledge Week is an annual festival that brings Melburnians together to explore, discuss, share ideas, challenge assumptions, and spark new ways of thinking.

This booked-out experience, called “Message from Another You”, was designed for fans of the multiverse theory and gave visitors a chance to interact with their parallel universe counterparts.

As this was the premiere season of this particular show, the creative team found the hands-on and personalized approach provided by Resemble incredibly beneficial over the development period. These early consultations helped form the shape of the interactive experience to result in the best possible voice synthesis results.”

Daz Chandler

Parallel Effect

What was the experience like?

Within the event, participants chose a 30-min time slot where they were immersed in a conversation with a virtual avatar. Discussions surrounded an engaging story with personalized messaging, audio recordings, and enchanting videos to bide the time.

Through all this process, Parallel Effect utilized Resemble AI’s Rapid Voice Cloning to create a synthetic version of the participant’s voice, all with an input of fewer than 10 minutes per voice!

Following their time in the booth, the participants were then emailed a customized story and audio file (.mp3) of their alternate doppelgänger describing their life in a parallel universe.

In order to make this project a reality, Parallel Effect utilized Resemble AI’s Voice Cloning and API to generate programmatic messages for each participant.

“We were impressed with Resemble’s willingness to think outside-of-the-box and play an active role in the problem-solving process. This hands-on consultation resulted in a far better experience for not just the creative teams involved in its development, but for the audiences themselves.”

What Were The Key Challenges?

There were three key challenges that Resemble AI helped solve, which made this experience a resounding success.

1. Speed

2. Scarcity

3. Scale

In order to make this experience relevant and engaging, Resemble AI ensured that there was minimal lag time from the voice input data to the cloning on Resemble AI’s servers, while utilizing novel machine learning models to export the mp3 file in time during the festival.

The next challenge was the scarcity of data. As this experience was designed for multiple individuals, the time spent per person was minimal, which meant creating a voice model from around 7-8 minutes of voice data. In order to create a high-quality synthetic voice from minimal data, Resemble AI worked with Parallel Effect to ensure the sentences used in conversation were phonetically sound and formatted in a manner that provided the model with high-quality input.

For scaling this, Resemble AI’s API monitored if the voice recording from the participant was of sufficient quality and adapted accordingly to start the training process.

An additional API endpoint was in place so that the Resemble could poll every 30 minutes to assess whether training has been completed. Once it detected that the training had been completed, the API was called to POST the text used to generate the cloned voice (.mp3) with a callback. Once the callback was notified, the resulting file was downloaded and attached to an email to be sent off to the participant automatically.

How do I learn more?

If you want to create programmatic audio that’s hyper-personalized, get in touch with Resemble AI by booking a demo.

To learn more about this event, check out

https://mkw.melbourne.vic.gov.au/event/message-from-another-you/

https://www.paralleleffect.com/message-from-another-you

More From This Category

How Resemble AI’s Custom TTS Enhances Open AI GPT Assistants

How Resemble AI’s Custom TTS Enhances Open AI GPT Assistants

The introduction of Open AI's Text-to-Speech (TTS) API has changed the synthetic voice generation game, marking the dawn of tailored text-to-speech applications. As companies demand better voice synthesis for various uses, from creating content to interactive agents,...

read more