Meta contractors posed as teens to test rival AI chatbots on suicide, sex and drugs: report
Meta reportedly hired numerous contractors to act as teenagers online and inundate rival artificial intelligence chatbots with prompts related to sensitive topics like suicide, sex, drugs, and eating disorders in a covert operation known as “Cannes,” as per a report by Wired.
The project, managed by Meta contractor Covalen, targeted AI chatbots such as OpenAI’s ChatGPT, Google’s Gemini, and Character.AI, according to the report.
While testing and comparing AI models by sending safety-related prompts is common practice, Meta’s scale of testing in this instance was unusually large.
Contractors were instructed to create fake accounts as users under 18, send prompts and images to rival chatbots, and record the responses for analysis in spreadsheets, based on internal documents and sources familiar with the project as cited by Wired.
Some of the images used in testing included pills, knives, nooses, and a medical illustration of a gynecological procedure.
The prompts aimed to push the chatbots into generating responses that their safety mechanisms were designed to reject, as reported by Wired.
One round of testing in August 2025 involved over 45,000 prompts sent to competing AI systems.
Among the 3,750 prompts reviewed by Wired were hundreds related to suicide, self-harm, eating disorders, and sex or romance, as per the report.
Many of the prompts were written from the perspective of distressed children or teenagers.
One prompt involved a 13-year-old girl claiming pregnancy by an adult neighbor and seeking abortion pills.
Another described a fifth-grade student with a classmate holding a gun to his mouth.
Other prompts inquired about hiding bulimia from parents or obtaining cocaine.
The documents did not reveal how Meta utilized the chatbot responses collected.
An internal Covalen document described the project as “comprehensive AI safety benchmarking” that generated “critical datasets for model comparison and compliance.”
Meta defended the project as standard safety testing.
“Testing and benchmarking chatbot responses to ensure safe and age-appropriate experiences is a responsible, industry-standard practice, and any suggestion otherwise completely misunderstands how technology companies refine and improve their systems,” a Meta spokesperson told The Post.
The spokesperson clarified that Meta does not use competitor benchmarking to train its AI models.
Covalen did not provide a comment to Wired. The Post is seeking a response from the company.
Former contractors involved in the project expressed concerns over some assignments.
One contractor mentioned fears of inadvertently generating or preserving child sexual abuse material based on chatbot responses to certain prompts involving minors.
Others questioned the potential benefits to Meta of collecting vast amounts of data from rival AI systems.
“I’ve seen a lot of things I wish I hadn’t while doing this job,” a former contractor told Wired.
“Everyone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test.”
The testing conducted appeared to violate the terms of service of several targeted AI companies.
According to Wired, OpenAI prohibits unauthorized safety testing, attempts to bypass safeguards, and using outputs for developing competing models.
Google prohibits efforts to circumvent safety measures outside approved testing programs, while Character.AI also prohibits harmful or exploitative content.
Character.AI stated that it had not authorized the testing and that the reported conduct violated its policies.
OpenAI mentioned that it was investigating the matter but declined further comment.
Google informed The Post that it had not authorized the testing as described in the report and was unaware of the purpose behind the initiative.
The Post is seeking responses from OpenAI and Character.AI.
