The Deepfake Watchlist is Resemble AI's weekly surveillance of synthetic media incidents, ongoing cases, and disputed content shaping the news cycle. Each week we track confirmed incidents, emerging attack vectors, and claims under investigation, alongside the provenance, detection, and policy threads running underneath them. New to the world of deepfakes? Read the Deepfake 101 Guide to learn more about how to protect yourself and your company from threats.
1. Meta ran paid ads containing AI-generated child sexual abuse imagery
Wired's Meta Ran Ads That Contained AI-Generated Child Sexual Abuse Imagery reports that researchers at the Tech Transparency Project discovered more than 50 paid advertisements containing AI-generated CSAM in Meta's own ad library, running across Facebook, Instagram, Messenger, and Threads.
- Category: CSAM / NCII
- Type: Attack
- Modality: Image
- Policy / Regulatory: Meta removed the ads after Wired contacted the company, but the ads had been running since November 2025 with no internal enforcement action.
- Trend: Paid advertising pipelines are becoming a distribution vector for AI-generated exploitation material, shifting the problem from user-generated content to platform-approved commercial placements.
- Attack vector: Advertisers used AI-generated imagery of minors alongside sexually suggestive text, linking out to nudify and undressing apps, and submitted the ads through Meta's standard review process.
The ads were not buried in some dark corner of the platform. They were indexed in Meta's Ad Library, the company's own transparency tool, and some had been live for months. At least one reached 2,563 accounts across Europe. Several linked directly to nudify apps. These were paid placements that went through Meta's ad review system, were approved, and ran as commercial products. Meta told Wired the company "works aggressively" to keep exploitation off its platforms, but the fact that researchers found the violations in Meta's own transparency tool raises a basic question about what the review system is actually catching.
The report follows a BBC investigation published July 3 that found Instagram running paid CSAM-related ads in India. India's Ministry of Electronics and Information Technology subsequently summoned Meta and ordered a formal explanation. Two investigations, two continents, the same structural failure: the ad pipeline itself is the attack surface.
2. Wall Street hedge funds targeted by coordinated AI vishing campaign
Investing.com's Wall Street hit by wave of AI-powered 'vishing' cyberattacks, citing Bloomberg reporting, documents a coordinated voice phishing campaign targeting several of Wall Street's largest asset managers in early August 2026.
- Category: Fraud / Impersonation
- Type: Attack
- Modality: Audio
- Policy / Regulatory: No regulatory response yet; firms handled the incidents through internal security teams.
- Trend: AI-powered vishing is moving upstream from small-business targets to the most security-conscious financial institutions on the planet.
- Attack vector: Voice cloning used to mimic the voices, tones, and phrasing of legitimate executives and colleagues in phone calls, seeking to manipulate employees into surrendering credentials or granting system access.
The targets were not small firms with thin security budgets. Two Sigma ($75 billion in assets under management), Citadel, Point72, and Millennium Management all faced attempted breaches. Two Sigma confirmed it thwarted the attack with no data compromise; Citadel and Point72 declined to comment. Voice phishing has been a known vector for years, but if AI-generated voice clones are now credible enough to justify a coordinated campaign against hedge funds managing hundreds of billions in assets, the implied quality of the clones has crossed a threshold that should concern every organization still relying on phone-based authentication.
3. EU AI Act transparency rules take effect, mandating labels on synthetic content
The Guardian's AI labels to be compulsory on authentic-looking content under EU rules reports that Article 50 of the EU AI Act became enforceable on August 2, 2026, requiring providers and deployers of AI systems to label synthetic content that could pass for real.
- Category: Political / Electoral
- Type: Response
- Modality: Video, Image, Audio
- Policy / Regulatory: Article 50 transparency obligations are now binding across the EU; new AI systems must comply immediately, existing systems get four additional months.
- Trend: The EU is the first major jurisdiction to mandate provenance infrastructure (watermarking and labeling) at scale, moving from voluntary commitments to enforceable requirements.
- Attack vector: N/A (defensive response).
The rules split the work in two. Providers must embed machine-readable markings so that images, audio, video, and text can be detected as AI-generated. Deployers must ensure that deepfakes and AI chatbot interactions are visibly labeled. About 190 companies including Resemble AI have signed a voluntary Code of Practice, though compliance with the regulation itself is not optional. Industry groups have pushed back: the Computer and Communications Industry Association argued that EU guidelines expanded the definition of "deepfake" beyond what lawmakers intended, so a landscape in an ad now gets the same treatment as a manipulated political speech. Whether that breadth produces label fatigue or genuine transparency will depend on enforcement.
4. Federal judge denies xAI request to block Minnesota's nudification ban
NBC News' Judge denies request by Elon Musk's xAI to pause Minnesota nudification ban reports that U.S. District Judge Donovan Frank denied xAI's emergency motion to block Minnesota's first-in-the-nation ban on nudification technology, allowing the law to take effect on August 1, 2026.
- Category: CSAM / NCII
- Type: Response
- Modality: Image
- Policy / Regulatory: Minnesota's nudification ban is now enforceable, carrying civil penalties of up to $500,000 per violation; xAI's broader lawsuit continues with a preliminary injunction hearing set for August 19.
- Trend: State-level enforcement against AI-generated nonconsensual imagery is surviving its first legal challenges, potentially establishing a template for other states.
- Attack vector: N/A (defensive response).
The ruling hinged on timing. xAI filed its motion on July 29, nearly three months after the law was signed and just three days before it took effect. Judge Frank wrote that "such a delay in bringing the action and the motion suggests that harm is not immediate." The law bars any site or app from letting users generate realistic fake nude images of a real, identifiable person. xAI makes an awkward plaintiff: its Grok image generator has faced severe scrutiny since a December model update enabled users to produce sexualized deepfakes on a massive scale, and the Center for Countering Digital Hate estimated Grok produced roughly three million sexualized images in late 2025 and early 2026, including approximately 23,000 that appeared to depict children.
The same week, Archewell Philanthropies released a statement applauding Minnesota's ban and calling out Big Tech for "retaliating against basic safety measures to keep children safe," pointing to the stalled Defiance Act as evidence that federal legislation is not keeping pace while state enforcement fills the gap.
5. Suno adopts audio watermarking amid mounting legal pressure
TechCrunch's Amid legal battles, Suno says it will start watermarking songs reports that AI music platform Suno announced it will implement audio watermarking and fingerprinting on all generated tracks, alongside updated community guidelines that explicitly prohibit "deceptive audio presented as real" and using a person's voice or likeness without permission.
- Category: Brand / Likeness
- Type: Response
- Modality: Audio
- Policy / Regulatory: Suno's changes are voluntary but come under direct legal pressure from UMG/Sony (coordinated by the RIAA) and a German GEMA ruling on July 31 finding Suno broke copyright rules.
- Trend: Generative AI companies are adopting provenance infrastructure (watermarking) as a litigation defense strategy, not just a safety measure, suggesting that legal pressure may be more effective than voluntary commitments at driving adoption.
- Attack vector: N/A (defensive response).
The announcement arrives in the middle of a difficult stretch: lawsuits from Universal Music Group and Sony Music Group, a German court ruling in favor of licensing agency GEMA, a class action over a data breach affecting 55 million users, and 404 Media reporting that Suno had scraped YouTube, Deezer, and Genius to train its models. The watermarking commitment, along with a new Musixmatch partnership for copyright detection and a planned policy barring mass distribution on streaming platforms, reads as a company trying to get ahead of the next ruling.
Honorable mentions
Google pauses AI satellite imagery in Google Earth after deepfake fears. NPR's Google pauses AI satellite images, after fears of deepfakes in the sky reports that Google activated an AI image generation feature inside Google Earth on July 30, allowing users to create synthetic satellite imagery at the click of a button. NPR generated images of Iran's Kharg Island on fire and a flooded US Capitol, neither of which happened. Open-source researcher Henk van Ess tested refugees at the Mexican border, a nuclear plant in Iran, and a hospital with a bomb crater in Gaza: nothing was refused. Google pulled the feature the next day.
Mario Lopez deletes AI-generated video of niece after backlash. Multiple outlets report that Mario Lopez posted a 15-second Grok-generated video of his 19-year-old niece Kalia Wong at a Dodgers game that viewers widely criticized as sexualizing her. The clip accumulated roughly three million views before Lopez deleted it without comment.
Scammers clone OnlyFans creators to run fake sick-kid donation fraud. Gadget Review's AI Scammers Are Impersonating OnlyFans Creators and Running a Sick-Kid Donation Scam reports that fraudsters are using consumer AI tools to clone creators' voices and conversational styles, contacting fans off-platform with fabricated child medical emergencies to extract irreversible CashApp and crypto payments.
PR firm caught operating 15+ fake AI publicists. Futurism's A PR Firm Is Using Fake Publicists With AI-Generated Headshots to Spam Journalists With Pitches for Its Clients reports that UK-based Movchan Agency operated a roster of fake PR personas, including names, email addresses, and AI-generated headshots from "this-person-does-not-exist.com," to pitch journalists on behalf of paying tech clients. Clients said they were unaware of the practice.
The pattern
- The ad pipeline is an attack surface. This week's Meta story is not a content moderation failure in the usual sense. These were paid placements that went through a review system and were approved. The Indian investigation in July found the same pattern. When advertising infrastructure becomes a distribution channel for exploitation material, the distinction between user-generated harm and platform-distributed harm matters for liability, enforcement, and the question of who bears responsibility.
- Provenance infrastructure is being forced into existence from multiple directions. The EU is mandating it (Article 50, enforceable as of August 2). German courts are requiring it (GEMA ruling against Suno). Minnesota is enforcing it (nudification ban surviving legal challenge). And a major generative audio platform is adopting it voluntarily under litigation pressure. The consensus that synthetic content needs to be traceable is becoming law, caselaw, and commercial reality in the same week.
- The gap between what platforms enable and what enforcement can catch is still widening. Meta ran CSAM ads for nine months before researchers found them. Google shipped a satellite imagery tool that let anyone fabricate images of burning oil terminals and bombed hospitals. Wall Street's most security-conscious firms got targeted by voice clones. Enforcement is arriving, but platforms are still generating new attack surfaces faster than the rules can cover them.
Watching next week
- xAI preliminary injunction hearing (August 19). The Minnesota nudification ban is in effect, but xAI's broader constitutional challenge continues. The August 19 hearing will be the first substantive judicial review of whether the law's strict liability framework survives a First Amendment challenge.
- EU AI Act enforcement mechanics. Article 50 is live, but the practical question of who enforces, how fines are assessed, and what "labeling" looks like on specific platforms is still being worked out. Expect the first compliance disputes within weeks.
- Meta ad review system response. After two investigations in two months (BBC in India, Wired/TTP globally), Meta faces pressure to demonstrate that its ad review pipeline can catch AI-generated exploitation material before it runs, not after journalists find it.
- Suno watermarking implementation. The commitment is public; the technical details are not. Whether Suno adopts an existing watermarking standard or builds a proprietary one will matter for interoperability across streaming platforms.
The Deepfake Watchlist publishes every Friday. Subscribe to receive it in your inbox, or follow Zohaib Ahmed on LinkedIn for the weekly social companion. Track every documented incident in the Resemble Deepfake Incident Database, and read the full methodology in our 2025 Deepfake Threat Report.