The Deepfake Watchlist is Resemble AI's weekly surveillance of synthetic media incidents, ongoing cases, and disputed content shaping the news cycle. Each week we track confirmed incidents, emerging attack vectors, and claims under investigation, alongside the provenance, detection, and policy threads running underneath them. New to the world of deepfakes? Read the Deepfake 101 Guide to learn more about how to protect yourself and your company from threats.
1. PSNI victim loses £250,000 to deepfake celebrity endorsement video in layered investment fraud
BBC News's Victim loses £250k after scammers use AI video of 'well-known' celebrity reports that a resident of Ards and North Down, Northern Ireland lost £250,000 to fraudsters who used an AI-generated video of a recognized financial sector personality to establish initial credibility, then moved contact to WhatsApp and extracted funds over weeks through escalating investment demands, remote computer access requests, and encouragement to borrow.
- Category: Fraud / Impersonation
- Type: Attack
- Modality: Video
- Policy / Regulatory: The Police Service of Northern Ireland issued a public fraud warning; Superintendent Joanne Gibson advised independent verification and FCA authorization checks before investing; no new legislation is pending in response.
- Trend: Deepfake video serving as a trust-establishing opener rather than the primary fraud mechanism, with synthetic content used to move victims off-platform before the actual financial extraction begins.
- Attack vector: AI-generated celebrity endorsement video used to establish credibility → contact shifted to WhatsApp → escalating investment demands → remote computer access → victim encouraged to borrow → funds moved to criminal-controlled accounts.
Fraudsters used a synthetic celebrity video to move the victim off-platform, then closed the transaction through weeks of WhatsApp social engineering. Platform scanning for synthetic content in ads and posts doesn't reach the channel where the money actually changed hands. By the time £250,000 moved, the conversation had long since left any surface with moderation.
2. Foreign "AI slopaganda" network of 200+ Facebook pages deepfakes Australian politicians for commercial engagement
ABC News Australia's Foreign 'AI slopaganda' network targets Australian politicians with deepfakes reports that Reset Tech identified a coordinated network of 200+ Facebook pages, traced to a Sri Lanka-based influencer business, publishing AI-generated deepfake videos and sexualized imagery of Australian politicians including Prime Minister Albanese, across 1,864 globally coordinated pages with 5.9 million followers.
- Category: Political / Electoral
- Type: Attack
- Modality: Video, Image
- Policy / Regulatory: Facebook removed only 1 in 33 of the identified posts and applied AI labeling to only 1 in 3; approximately 10% of pages were monetized through Meta's own creator payment scheme.
- Trend: Foreign commercial influence operations using political deepfakes as engagement bait rather than ideological disinformation, with the platform's own monetization infrastructure as the underlying business model.
- Attack vector: Politically themed pages with names like "I Love Australia" publishing AI-generated deepfakes of named politicians, including sexualized imagery of female lawmakers, to drive engagement and collect creator payments from Meta.
Reset Tech's Rys Farthing said "This isn't foreign interference as we traditionally think of it. What we see here is foreign influence for commercial motivations." The operator isn't trying to change an election outcome; it's farming engagement from political content because political content drives clicks, and then collecting payments through Meta's own creator scheme. The 1-in-3 AI labeling rate means two-thirds of the AI-generated political content Meta encountered was served to users without any disclosure, while sexualized imagery of named female politicians circulated alongside it.
3. First federal Take It Down Act sentencing: Columbus man gets 15 years for cyberstalking and AI-generated CSAM
404 Media's First 'Take It Down Act' sentencing puts man behind bars for 15 years reports that James Strahler, 38, of Columbus, Ohio was sentenced September 9 to 15 years in federal prison for cyberstalking and producing AI-generated CSAM, in the first federal conviction and sentencing under the TAKE IT DOWN Act, which went into effect in May 2026.
- Category: CSAM / NCII
- Type: Response
- Modality: Image
- Policy / Regulatory: The first criminal conviction under the TAKE IT DOWN Act establishes what federal prosecution under the statute looks like in practice, four months after the law took effect.
- Trend: Federal synthetic media law moving from enactment to active criminal sentencing within months, compressing the window between legislation and landmark prosecution.
- Attack vector: Strahler used AI generation tools to create explicit images of minor boys from his community in sexual acts with family members and combined them with cyberstalking of multiple women through threats, extortion demands, and surveillance images.
The TAKE IT DOWN Act's first sentencing arriving the same week as the Meta CSAM ad revelations captures the scope of the same problem from two vantage points: the law reaching one individual offender with identifiable victims and a traceable FBI investigation, and a platform's commercial ad system running 350+ videos of AI-generated CSAM with no named defendant in sight.
4. Meta ran 350+ AI child sexual abuse ads as Tech Transparency Project tried to verify its own safety tools
Bloomberg and SFist's Meta caught running hundreds of ads featuring AI-generated child sexual abuse images report that the Tech Transparency Project identified 350+ video ads on Meta containing AI-generated CSAM, including ads using images of real American minors, discovered while researchers were attempting to verify Meta's advertised safety tools, with San Francisco ordering Meta to cease.
- Category: CSAM / NCII
- Type: Attack
- Modality: Video, Image
- Policy / Regulatory: San Francisco's cease-and-desist is a city-level action outside the federal CSAM enforcement framework; TTP's discovery of 250+ additional ads while testing Meta's own safety tools suggests published compliance mechanisms are not functioning as represented.
- Trend: AI-generated CSAM embedded in commercial advertising infrastructure, using the platform's paid ad review system as a distribution mechanism that bypasses organic content moderation.
- Attack vector: Video ads beginning with ordinary photos of real minors, algorithmically transforming faces into explicit sexual videos, then directing viewers to third-party AI generation apps, passing through Meta's ad review as paid placements and running to completion.
The most damning detail is not the volume of ads but the context in which they were found: researchers looking for evidence that Meta's safety tools work discovered 250 more instances of CSAM in the process. A former Meta employee described the adversarial dynamic plainly: "Within a day, maybe five days, they built a new way to avoid detection." Meta's response, noting most ads had under 200 impressions and under $5,000 in spending, is a defense that substitutes scale of audience for presence of the content. Categorically prohibited material in a platform's ad system doesn't become acceptable because it ran to small audiences.
5. Apple embeds photo authenticity verification in iPhone 18 Pro, supporting SynthID standard
TechCrunch's Apple has a new way to prove your iPhone photos aren't AI slop reports that the iPhone 18 Pro captures signed sensor data alongside each photo and processes it through Private Cloud Compute into an unalterable reference image stored in the Photos app, with support for Google's SynthID standard and developer APIs for third-party integration.
- Category: Brand / Likeness
- Type: Response
- Modality: Image
- Policy / Regulatory: Apple's SynthID support and developer APIs align with the EU AI Act's content transparency requirements and position Apple as the first major device manufacturer to build provenance verification into the camera layer of a consumer product at scale.
- Trend: Provenance infrastructure moving from enterprise tool to consumer device, with the iPhone 18 Pro making authenticity verification available through the same interface users already rely on.
- Attack vector: Not applicable, this is a provenance and detection response story.
The Apple Reference Image doesn't evaluate whether a photo looks AI-generated, which is what most detection tools do and which gets harder as generation improves. It creates a cryptographic record of what the camera sensor actually saw at the moment of capture, making it possible to verify that an image originated from a real-world event and was not altered afterward. Apple framed this as "vital for photojournalists," which understates the scope, because the consequential long-term shift is what happens when hundreds of millions of iPhones are capturing cryptographically signed images as a default.
Honorable mentions
xAI loses second bid to block Minnesota's deepfake nudification ban. Judge Donovan Frank denied xAI's preliminary injunction motion on September 4, the second denial in two months, finding the company failed to demonstrate irreparable harm while acknowledging the constitutional questions are "complex." The case proceeds to full merits review.
Ofcom sets September 30 deadline for hash matching on deepfake intimate images. Platforms must deploy automated detection or demonstrate equivalents by month's end, or face fines up to 10% of global annual revenue. The Revenge Porn Helpline estimates 369,000 women experience intimate image abuse in the UK annually.
South Korean court sentences Chinese student to 18 months for 1,100 deepfakes of colleagues. The Korea Herald reports the student used publicly available lab photos of university peers to generate non-consensual explicit imagery, in the latest in a series of institutional-setting NCII prosecutions in South Korea.
The pattern
- Two of this week's attacks share a feature that's easy to miss: neither operator had an ideological agenda. The slopaganda network wasn't trying to change an election; it was farming political content for Meta creator payments. The Co Down investment fraud used a deepfake celebrity as a commercial instrument, not a personal attack. Both treated synthetic media as business infrastructure. That categorization matters for enforcement: laws built around harassment and influence operations don't reach engagement arbitrage or investment fraud, and platform policies designed for ideological harm don't catch either.
- The first TAKE IT DOWN Act sentencing and the Meta CSAM ad story arrived in the same week, and they tell the same story from opposite ends. The law is working at the individual prosecution level, with identifiable victims and a federal investigation to support it. The platform distribution problem, 350+ ads running through Meta's commercial system with no named defendant, is the version the current enforcement model can't reach.
- The structural answer isn't a faster detector. Apple's Reference Image, embedding cryptographic provenance at the moment of capture for hundreds of millions of iPhones, changes the information architecture underneath the problem. Detection and restriction are responses to harm already in circulation. Provenance at capture is a change to what gets vouched for before content ever enters a feed.
Watching next week
- Take It Down Act's next cases. Strahler established what federal prosecution under the statute looks like; the next cases will reveal how broadly the law is being applied.
- Ofcom September 30 deadline. Which platforms confirm hash-matching compliance and whether any face initial enforcement action.
- Apple Reference Image adoption. Whether major news organizations integrate the verification tools into editorial workflows, the first real test of provenance infrastructure at the capture layer.
The Deepfake Watchlist publishes every Thursday. Subscribe to receive it in your inbox, or follow Zohaib Ahmed on LinkedIn for the weekly social companion. Track every documented incident in the Resemble Deepfake Incident Database, and read the full methodology in our 2026 Midyear Deepfake Threat Report.