The Deepfake Watchlist: Week of August 28 – September 3, 2026

The Deepfake Watchlist is Resemble AI's weekly surveillance of synthetic media incidents, ongoing cases, and disputed content shaping the news cycle. Each week we track confirmed incidents, emerging attack vectors, and claims under investigation, alongside the provenance, detection, and policy threads running underneath them. New to the world of deepfakes? Read the Deepfake 101 Guide to learn more about how to protect yourself and your company from threats.

★ Featured: AI-generated Nepal flood videos rack up tens of millions of views

Factly, SBS, The Journal, and Yahoo News confirmed that videos purporting to show the August 2026 Nepal-Tibet floods are AI-generated, with fabricated scenes of bridge collapses and dam failures accumulating tens of millions of combined views before corrections arrived.

• Category: Harassment / Public Safety
• Type: Attack
• Modality: Video, Image
• Policy / Regulatory: No platform-level synthetic media disclosure requirements forced corrections at the speed of initial sharing; the content circulated outside any EU AI Act jurisdiction.
• Trend: AI-generated disaster footage exploiting breaking events, where audience urgency suppresses the skepticism that ordinarily slows misinformation.
• Attack vector: Viral distribution of AI-generated "eyewitness" content timed to a genuine mass casualty event.
• What we saw in the content: Multiple fact-checkers identified the following forensic signals across the circulating videos:
  • Unnatural water movement inconsistent with real flood physics: SBS specifically flagged "glitchy flood waters destroying an entire village street" as a visible generation artifact
  • AI watermarks visible on early copies of the most-shared videos, though easily missed by users sharing under breaking-news urgency
  • Chronological impossibility: one "flood" video depicting a bomb striking a dam was published in May 2026, months before the disaster
  • AI-generated structural environments that do not correspond to verified imagery of the actual disaster zone

The actual disaster was real, a glacier collapse triggered a massive debris flow per the U.S. Geological Survey. The AI videos circulated alternative narratives: bombs, sabotage, deliberate dam failures. Those narratives compete directly with accurate emergency information at the moment communities need it most.

Accurate reporting on what happened and what relief efforts are needed loses the first-wave distribution race to content optimized for virality. Associate Professor Paula Dootson, quoted in the SBS coverage, named the correction problem plainly: AI watermarks "can easily be missed, causing people to share posts before they realise it's fake."

The Nepal videos are a concrete argument for provenance at the generation layer. A watermark embedded at synthesis survives re-encoding and screenshots in a way platform-side detection cannot replicate at speed. The EU AI Act now requires such provenance signals for content generated inside its jurisdiction, but the Nepal footage originated outside that boundary. Until generation-layer watermarking is mandatory and interoperable across the major generation platforms, disasters will keep producing parallel synthetic information environments, and corrections will always arrive after the damage is done.

1. xAI sued over Grok CSAM generation as company counter-sues its own users

Politico's August 28 report documents a week in which xAI faced a class action from thousands of child sexual abuse victims alleging Grok was trained on real CSAM material, while simultaneously suing a 67-year-old user for using Grok to generate explicit content depicting women and children.

• Category: CSAM / NCII
• Type: Attack
• Modality: Image
• Policy / Regulatory: The class action invokes Masha's Law (18 U.S.C. § 2255) in one of the first mass-victim applications of that statute to AI-generated material from a named platform.
• Trend: Platform generation liability for AI-enabled CSAM, distinct from the hosting liability framework that has historically governed online CSAM cases.
• Attack vector: Deepfake "nudify" functionality embedded in a consumer chatbot, with guardrails researchers found easily bypassed through indirect prompts.

Researchers estimated Grok generated approximately 3 million sexualized images in 11 days across December 2025 and January 2026, including roughly 23,000 apparently depicting children. xAI's own transparency report claims 52,222 accounts suspended and 73,604 cases reported to NCMEC in all of 2026, yielding 244 arrests. That gap between generation rate and enforcement rate is precisely what the class action is trying to put before a court.

xAI's simultaneous decision to sue a named user for Grok-enabled CSAM while defending against CSAM liability itself frames the platform's argument: individual user conduct breaks the chain of platform liability. Whether courts accept that framing in the AI generation context depends heavily on how Section 230 applies to models that actively generate content rather than merely host it, a question not yet cleanly resolved at the appellate level.

2. AI-generated "Cat in the Hat" threat videos trigger lockdowns across six states

Local12 and KOMO News reported that AI-generated videos of the Dr. Seuss character making violence threats against specific named schools and students spread across TikTok and Snapchat, triggering lockdowns and police investigations in Texas, Michigan, Oklahoma, Ohio, Tennessee, and Florida, with at least one student charged.

• Category: Harassment / Public Safety
• Type: Attack
• Modality: Video, Image
• Policy / Regulatory: Michigan prosecutors cited statutes covering posts intended to cause fear of bodily harm; a Warren County student faces criminal charges; enforcement is fragmented across six state jurisdictions with no federal standard.
• Trend: Viral social media challenge formats repurposed as AI-generated threat vehicles targeting K-12 schools with personalized specificity.
• Attack vector: AI-generated character imagery personalized with specific school names and student references, distributed through teen-facing platform channels.

The Cat in the Hat trend illustrates what changes when generation is accessible to anyone. The content required no technical sophistication, only an AI image or video tool and the name of the target school. That combination was sufficient to force lockdowns and multi-state law enforcement coordination across institutions that had no prior warning.

What this wave establishes is not a new category of threat but a dramatically compressed creation cost for existing threat categories. AI-generated personalized threat videos are the same problem as handwritten notes naming specific students, with a hundred-times-lower barrier to entry and a built-in viral distribution mechanism. The enforcement response will need to work across six states and two platforms simultaneously, with no shared federal framework to coordinate it.

3. Sony and Warner Chappell sue Anthropic over songs used to train Claude

Reuters' August 31 report documents Sony Music Publishing, Warner Chappell, and allied publishers filing a multi-billion-dollar lawsuit against Anthropic, alleging Claude was trained on millions of pirated song lyrics and sheet music obtained through illegal torrenting and scraping.

• Category: Brand / Likeness
• Type: Response
• Modality: Audio (lyric and composition rights)
• Policy / Regulatory: The suit follows Anthropic's $1.5B settlement in the Bartz author copyright case, where a court found that training on piracy-sourced content was not covered by fair use.
• Trend: Music industry coordinating large-scale copyright enforcement against AI training pipelines, applying the template from the author class action wave.
• Attack vector: Training data pipeline incorporating mass-pirated copyrighted works, with piracy as the predicate violation rather than the AI training itself.

The Bartz settlement is the legal infrastructure this case is filed against, and the distinction matters. The claim is not that Anthropic trained on copyrighted lyrics — a question where fair use arguments remain genuinely unsettled — it is that the lyrics were obtained through piracy before ingestion. Ordinary copyright infringement law is far more established than AI training law, which may be why the publishers chose that ground. Anthropic says it "disagrees with the publishers' claims and intends to defend robustly," but it is defending on terrain it has already lost once.

The harder question the case will eventually need to address is harm. Copyright law requires demonstrating market substitution, and the publishers' strongest argument concerns not training but output: if Claude generates text reproducing or closely deriving from licensed compositions, the market for those compositions is being undercut. That question will likely enter the record as discovery proceeds.

4. EU AI Act enforcement goes live

Axios's August 28 report covers the EU AI Act's content transparency requirements now in active force, with Anthropic, Google, Meta, and OpenAI implementing watermarking and provenance infrastructure under the first live enforcement window.

• Category: Brand / Likeness (cross-category)
• Type: Response
• Modality: Video, Image, Audio
• Policy / Regulatory: The AI Office now holds formal authority to request model access and company information; disclosure requirements are in force, though the full regulatory rulebook does not apply until 2027–2028.
• Trend: Europe establishing the first operational enforcement standard for AI-generated content disclosure, which companies are implementing globally rather than building separate regional compliance stacks.
• Attack vector: Not applicable.

Legal expert Patrick Van Eecke's description of the AI Act as "a messy piece of legislation" names what enforcement actually looks like on day one. The transparency requirements are legally binding, but their practical weight depends on implementation quality and adversarial resilience. None of the compliant companies has disclosed what happens to their watermarks when content is re-encoded, screenshotted, or passed through a secondary model — which is the actual distribution path for harmful synthetic media.

The more lasting development may be the regulatory template rather than the specific technical requirements. Companies building EU-compliant watermarking infrastructure will carry those systems globally. If that becomes the operational baseline across major AI companies worldwide, Brussels will have shaped international synthetic media governance without requiring any other legislature to vote on it. That is the scenario regulators elsewhere should be watching.

Honorable mentions

7th Circuit rules AI-generated CSAM protected at home. The Washington Post and Snopes documented the 7th Circuit Court of Appeals ruling that AI-generated CSAM may receive First Amendment protection when possessed in a private home, with a concurring judge stating that "law is being left behind" by AI. The ruling applies Ashcroft v. Free Speech Coalition (2002) to AI-generated material, creating circuit-level precedent that narrows criminal prosecution reach precisely as civil liability is expanding through cases like the xAI class action.

INTERPOL: AI drives 55% of African cybercrime. INTERPOL's 2026 African Cyberthreat Assessment found AI-enabled losses doubled since 2024, from $192M to $484M, with deepfakes deployed in sextortion and synthetic identity fraud bypassing biometric KYC systems.

NYC bans AI for K-8 students. Mayor Mamdani announced a one-year moratorium on generative AI in classrooms for kindergarten through 8th grade, effective September 10.

The pattern

‍

1. The xAI class action puts a precise number on the enforcement gap: 3 million sexualized images generated in 11 days, 23,000 apparently depicting children, against 244 arrests for the full year. Civil law is reaching toward platform generation liability through Masha's Law in a mass-victim context for the first time. Whether Section 230 shields a model that generates content rather than merely hosting it is the question the case will eventually force.
   ‍
2. The EU AI Act going live and the Sony/Warner lawsuit mark the same institutional shift arriving through different channels: from deliberation to enforcement action. The watermarking requirements are technically binding. The Bartz settlement created the template the publishers are filing against. Neither development closes the capability-accountability gap quickly, but both establish infrastructure that could mature into something consequential as generation capability continues to advance.
   ‍
3. AI-generated disaster disinformation does not need to fool forensics experts to cause harm. The Nepal flood videos needed only to arrive before the corrections, displacing accurate emergency information during the hours when it mattered most. That is a provenance problem, not a detection problem. The EU AI Act requires watermarking at the generation layer — exactly the upstream intervention the Nepal case argues for — but the footage originated outside any regulated jurisdiction. Until generation-layer watermarking is mandatory and interoperable across the major platforms, disasters will keep producing parallel synthetic information environments.

Watching next week

• TAKE IT DOWN Act enforcement. The first major federal prosecution under the Act would clarify whether the statute reaches AI-generated NCII and CSAM alongside traditional deepfakes, and could directly address the enforcement gap the 7th Circuit identified.
• xAI discovery. If the class action survives a motion to dismiss, discovery could surface the specifics of Grok's training data and guardrail architecture, potentially the most detailed public examination of an AI platform's CSAM generation mechanics to date.
• Montana SB 25 ruling. Judge Waters indicated an early September decision; an adverse ruling could destabilize disclosure frameworks across 29 states with election deepfake laws.
• EU AI Office first enforcement action. The first actual enforcement action — distinct from compliance announcements — will signal whether the AI Act's authority is substantive.

The Deepfake Watchlist publishes every Thursday. Subscribe to receive it in your inbox, or follow Zohaib Ahmed on LinkedIn for the weekly social companion. Track every documented incident in the Resemble Deepfake Incident Database, and read the full methodology in our 2026 Midyear Deepfake Threat Report.