LLM-Based Social Engineering Scams
OpenAI reportedly took down a Cambodian social engineering operation that leveraged ChatGPT to scale various scams, including romance, investment, gambling, and impersonation, by crafting believable, long-form conversations.
According to a report, OpenAI took down a Cambodian social engineering operation that utilized ChatGPT as a force multiplier for a scam-as-a-service model. This operation was described as a diversified criminal enterprise simultaneously running romance, investment, gambling, and impersonation scams. The perpetrators leveraged generative AI, specifically ChatGPT, to craft believable, long-form conversations across multiple scam types, thereby reducing the human effort required to maintain fake identities and scale their social engineering efforts. The primary innovation was in spearphishing for information via AI-generated personas. The report notes that the primary indicator of such scams is behavioral, such as unnatural fluency in multiple scam narratives from a single source or language patterns that are "too perfect" for a non-native speaker. Any platform allowing LLM API access is considered a potential vector, as scammers use the models as tools rather than exploiting vulnerabilities.