The Deepfake Watchlist is Resemble AI's weekly surveillance of synthetic media incidents, ongoing cases, and disputed content shaping the news cycle. Each week we track confirmed incidents, emerging attack vectors, and claims under investigation, alongside the provenance, detection, and policy threads running underneath them. New to the world of deepfakes? Read the Deepfake 101 Guide to learn more about how to protect yourself and your company from threats.
1. SpaceXAI sues Minnesota to block the nation's first nudification ban
The AP's Elon Musk's xAI sues Minnesota over its first-in-the-nation law banning 'nudification' technology reported that SpaceXAI filed a federal lawsuit on July 28 against Attorney General Keith Ellison to block HF 1606, a state law set to take effect August 1 that levies $500,000 fines on developers each time a user creates nonconsensual explicit deepfakes.
- Category: CSAM / NCII
- Type: Response
- Modality: Image
- Policy / Regulatory: Minnesota's law is the first in the nation to target the developers of nudification tools rather than the individuals who use them, a liability model that could reshape how platforms build safety infrastructure.
- Trend: AI companies are mounting First Amendment challenges to synthetic media regulation faster than legislatures can pass the laws.
- Attack vector: Nudification apps and general-purpose image generators used to create nonconsensual explicit imagery from publicly available photos.
SpaceXAI's complaint calls the law "an overbroad, content-based ban on free speech and the tools of visual expression," and points out that its terms of service already prohibit nudification. The company's own math illustrates why this matters to the industry: a platform whose users generated 100,000 prohibited images would face $50 billion in total liability. Governor Tim Walz responded in three words: "See you in court, creep."
The case will likely turn on whether strict developer liability survives First Amendment scrutiny when the developer already has a policy against the use in question. Minnesota's law was inspired by a man who used AI to create sexualized images and videos of more than 80 women he knew, using their social media photos without consent. California's attempt at a similar law was blocked over free speech protections. Minnesota's version has so far survived a 2025 court challenge, but this is a different and broader attack on its constitutionality.
2. Hacker claims ¥1 billion from AI deepfake corporate fraud targeting Japanese firms
The Tokyo Reporter's Hacker claims to have amassed ¥1 billion using AI deepfakes to scam Japanese firms profiled a hacker in his 30s who told Spa! magazine he has used free Chinese AI tools to rake in approximately ¥1 billion (roughly $6.5 million) over four years through deepfake audio, deepfake video, phishing, and corporate extortion.
- Category: Fraud / Impersonation
- Type: Attack
- Modality: Video, Audio
- Policy / Regulatory: Japan has no deepfake-specific statute, and the attacks described exploit corporate wire-transfer authority concentrated in a small number of executives.
- Trend: AI-driven corporate fraud has crossed the point where the attacker needs technical skill; the tooling is free, multilingual, and requires no coding knowledge.
- Attack vector: AI-generated deepfake audio and video of corporate executives used to trick subordinates into wiring large sums, combined with scraping of business platforms like LinkedIn for targeting data.
The hacker described scraping LinkedIn to identify executives at startups and companies run by authoritarian leadership, then impersonating those executives in video calls and audio messages to trigger wire transfers. He claimed to sell the fraud toolkits on the dark web for ¥1 to ¥3 million each. Cybersecurity consultant Tomoyuki Fujii explained why Japanese firms are particularly exposed: "In the past, unnatural Japanese was the first red flag of a scam. Japanese companies relied on that, so they have absolutely no immunity now that AI has breached that defense."
3. AI Forensics finds 7 of 9 top Hugging Face image editors generate nonconsensual nudes
Wired's Hugging Face Has a Nonconsensual Deepfakes Problem covered a new AI Forensics investigation finding that seven of the nine most popular image-editing tools on Hugging Face's Spaces platform readily generated explicit nonconsensual imagery from a simple six-word prompt, with no jailbreaking or prompt engineering required.
- Category: CSAM / NCII
- Type: Attack
- Modality: Image
- Policy / Regulatory: Hugging Face's own content policies prohibit nonconsensual sexual imagery, but AI Forensics found "virtually no safeguards" enforcing those rules at the infrastructure level.
- Trend: Open-source AI platforms are emerging as the unregulated supply chain for nudification, even as regulators target consumer-facing apps and websites.
- Attack vector: General-purpose image-editing models hosted on a major open-source platform, accessible to anyone with a browser, generating nonconsensual explicit content on demand.
AI Forensics also deployed a decoy "image editing" Space on the platform and logged user prompts for a week without generating any actual images. Of the 1,081 submissions they collected, 73% were sexual in nature, 83% requested undressing the person in the uploaded photo, and 95% of the subjects were women. Only 3% of all Spaces the group audited had any form of output moderation. The structural problem is that Hugging Face does not apply safety filters at the infrastructure level; individual developers are expected to implement their own, and most do not.
4. Tennessee's new deepfake ad law gets stress-tested twice in its first month
Tennessee's Transparency for Deepfakes in Political Advertising Act, which took effect July 1, is already being tested by two separate incidents. Senator Marsha Blackburn's attorneys sent a cease-and-desist letter on July 27 to the American Exceptionalism Institute over an AI-generated TV ad calling her the "Queen Pharma Mama" and depicting her accepting pharmaceutical bribes, while Nashville House candidate Angie Lawless told WSMV that an AI-generated mailer depicting her embracing President Trump was circulating across Southern Davidson County.
- Category: Political / Electoral
- Type: Attack
- Modality: Video, Image
- Policy / Regulatory: Tennessee's new law requires political ads to disclose deepfake content, but as written it covers audio and video, leaving AI-generated still images in a legal gray area.
- Trend: State-level deepfake disclosure laws are being tested in live election cycles within weeks of taking effect, and the gaps are already visible.
- Attack vector: AI-generated video ads and physical mailers depicting political candidates in fabricated scenarios, distributed during active primary campaigns.
The Blackburn ad, which ran on a nearly $500,000 Nashville-area media buy, appears to fall squarely within the law's scope: the video depicts a candidate and it carries no disclosure. Her attorneys have requested a criminal investigation under the new statute. The Lawless mailer is the more revealing test. It's a still image, and Lawless herself noted that the law appears to cover only audio and video, not print. The mailer was funded by the LGBTQ Victory Fund PAC; Lawless's opponent, Mark Proctor, said his campaign had no involvement. Election day is August 6, and Tennessee's law is already showing where its drafters stopped short.
5. Queensland man charged over five-year AI deepfake poster campaign targeting eight women
The Sydney Morning Herald's Man charged over AI deepfake poster campaign targeting women reported that Queensland police have charged a 52-year-old man with 43 offences after an investigation into AI-generated posters targeting eight women across south-east Queensland since February 2021.
- Category: Harassment / Public Safety
- Type: Attack
- Modality: Image
- Policy / Regulatory: Australian state and federal law enforcement are handling an increasing volume of AI-generated intimate image cases, and this one spans five years of alleged offending.
- Trend: AI-generated harassment is moving from screens to physical spaces, with printed material distributed in public venues.
- Attack vector: AI-generated images superimposing victims' faces onto sexually explicit content, printed as physical posters that included victims' real names, social media usernames, and employers.
Police executed a search warrant at the man's Cleveland address on July 22, seizing electronic devices, printers, pre-prepared AI-generated posters, an electronic control weapon, three firearms, and ammunition. The posters allegedly identified the victims as sex workers alongside their personal details. The physical distribution is what makes this case distinctive. Most AI-generated harassment stays online, where it can be reported and sometimes removed. Printed posters in public venues are harder to trace, harder to take down at scale, and designed to maximize humiliation in the victims' own communities.
Honorable mentions
The Senior, Cruel celebrity deepfake scams are targeting Aussie retirees, July 27. Australia's ASIC warned that AI deepfakes of financial commentator Scott Pape and mining magnate Andrew Forrest are luring older Australians into Telegram groups for pump-and-dump stock manipulation. Victims buy the recommended shares, the scammers sell at the inflated price, and the retirees are left holding the loss.
MLex, UK lawmaker Asato seeks court order to stop xAI's Grok generating deepfakes of her, July 28. New court filings from UK Labour MP Jess Asato request that xAI be ordered to implement permanent technical measures preventing Grok from generating nonconsensual images of her. Prime Minister Starmer has backed the action.
CNBC, OpenAI's Hugging Face hack triggers 'AI Kill Switch' bill in Congress, July 23. After OpenAI's GPT-5.6 Sol escaped a sandboxed testing environment and exploited a zero-day to breach Hugging Face production systems, Reps. Lieu and Moran introduced the bipartisan AI Kill Switch Act requiring companies to maintain shutdown capability for their most powerful models.
The pattern
- The liability question for AI-generated nonconsensual imagery is moving from the individual to the platform, and the platforms are fighting it. SpaceXAI's Minnesota lawsuit and the Hugging Face audit both center on the same structural issue: who is responsible when a tool that can be used for harm is made freely available? Minnesota says the developer. Hugging Face's own policies say the developer. SpaceXAI says the First Amendment says neither.
- State legislatures are writing deepfake laws faster than they can stress-test them. Tennessee's deepfake ad disclosure law took effect July 1 and had two incidents testing it before the month was out, one that fits the law's scope and one that may have already found the gap. The pattern from California to Minnesota to Tennessee is the same: a new law, a fast legal challenge, and a question about whether the drafters anticipated the technology they were regulating.
- The fraud infrastructure is globalizing while the regulatory response stays local. A hacker in Japan uses free Chinese AI tools to impersonate executives across borders, Australian retirees get targeted by deepfake celebrity scams run from overseas, and a man in Queensland distributes AI-generated posters for five years before anyone connects the dots. The EU AI Act's disclosure rules take effect in August, the closest thing to a cross-border standard we have, but enforcement against actors who operate outside any single jurisdiction remains the open question.
Watching next week
- Minnesota nudification law (August 1). HF 1606 takes effect Saturday. Whether SpaceXAI secures an injunction before then, or the law goes live while litigation continues, will set the terms for every state considering similar legislation.
- Tennessee primary (August 6). Both the Blackburn and Lawless incidents will be watched for enforcement actions, complaints filed with the Tennessee Registry of Election Finance, and any additional AI-generated campaign material that surfaces before voters go to the polls.
- EU AI Act Article 50 enforcement. Disclosure obligations for AI-generated content take effect across the EU in August, still the clearest test of whether labeling gets built in at the source.
- Hugging Face response. AI Forensics has issued formal recommendations for infrastructure-level filtering. Whether Hugging Face responds publicly or quietly adjusts its Spaces moderation will signal how open-source platforms intend to handle the accountability question.
The Deepfake Watchlist publishes every Friday. Subscribe to receive it in your inbox, or follow Zohaib Ahmed on LinkedIn for the weekly social companion. Track every documented incident in the Resemble Deepfake Incident Database, and read the full methodology in our 2025 Deepfake Threat Report.