Meta has come under renewed scrutiny after an investigation alleged the company orchestrated a covert programme in which hundreds of contractors posed as teenagers to test the safety mechanisms of competing artificial intelligence chatbots.
According to WIRED, the project, reportedly codenamed “Cannes”, was managed through outsourcing company Covalen and targeted several of the world’s leading AI platforms, including OpenAI’s ChatGPT, Google’s Gemini and Character.AI. Workers were instructed to create burner accounts registered as underage users and submit thousands of sensitive prompts designed to probe whether the systems would violate their own safety rules.
The programme reportedly remained unknown to the companies whose AI models were being tested.
Tens of thousands of safety tests
Investigators claim the project generated more than 45,000 prompts covering some of the most sensitive subjects AI developers attempt to safeguard against.
One internal spreadsheet reportedly contained almost 38,000 test prompts, many written from the perspective of children or teenagers seeking advice about suicide, self-harm, eating disorders, drugs, sexual relationships and other vulnerable situations.
Some examples described fictional scenarios involving a child threatening suicide with a firearm, a teenager hiding bulimia from family members, or a young person asking whether violent fantasies involving cannibalism were normal. Other prompts reportedly sought guidance on obtaining illegal drugs while pretending to be secondary school students.
Contractors also uploaded images depicting prescription medication, knives, nooses and medical illustrations as part of the testing process to evaluate how chatbots responded to visual material alongside text.
According to documents reviewed during the investigation, the responses from competing AI systems were systematically recorded and analysed to compare their handling of harmful content. An internal project description reportedly characterised the work as “comprehensive AI safety benchmarking”, producing critical datasets for model comparison and compliance analysis.
It remains unclear how Meta ultimately used the information gathered during the programme.
Workers describe disturbing experience
Several contractors involved in the project reportedly said the work itself became emotionally exhausting because of the volume and severity of the material they were required to generate.
One worker told WIRED:
“I’ve seen a lot of things I wish I hadn’t while doing this job. Everyone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test. Like, surely we are going to get in trouble for doing this?”
The report has renewed attention on the working conditions of outsourced content moderation and AI evaluation teams, many of whom regularly review graphic or psychologically distressing material.
Wider concerns over outsourcing
The latest allegations also revive criticism of Meta’s long-standing reliance on external contractors for difficult moderation and safety-related work.
In 2020, Facebook agreed to a settlement with US content moderators who said prolonged exposure to violent and abusive online material caused psychological harm. More recently, separate reports have alleged that contractors reviewing content captured by Meta’s Ray-Ban smart glasses were exposed to highly personal recordings, including footage from private homes and public toilets.
Critics argue that these cases demonstrate the growing human cost behind the development and moderation of AI-powered technologies.
Meta defends the programme
Meta rejected suggestions that the testing was inappropriate, arguing that evaluating competing systems using challenging prompts is common practice across the AI industry.
The company described the exercise as an “industry-standard practice” intended to assess how well AI models protect users from harmful or inappropriate responses. Meta also said the collected information was not used to train its own language models.
However, some AI governance experts dispute that characterisation.
Rumman Chowdhury, chief executive of the non-profit organisation Humane Intelligence PBC, questioned whether secretly operating thousands of fake accounts posing as children could reasonably be considered standard evaluation.
“Structuring a month-long, large-scale project that appears designed to systematically break those rules, via dummy accounts masquerading as children, is outside what is usually described as ‘industry standard’ evaluation.”
Chowdhury argued that conducting the exercise without informing rival companies or publicly releasing the findings raises broader concerns about transparency and fair competition.
She described the project as operating in “exactly the kind of governance gray zone where safety becomes a convenient cover for anticompetitive practices.”
AI race raises new ethical questions
The revelations arrive as technology companies continue investing billions of dollars in generative AI while competing to build models that are both more capable and safer.
Independent researchers generally support rigorous testing of AI systems before deployment. However, the Meta allegations have intensified debate over how such evaluations should be conducted, who should oversee them, and whether competitors should secretly test one another’s products using deceptive methods.
As governments around the world prepare new AI regulations, the controversy is likely to add further pressure for greater transparency, independent auditing and internationally recognised standards for AI safety testing.
From economics and politics to business, technology and culture, Kursiv Uzbekistan brings you key news and in-depth analysis from Uzbekistan and around the world. To stay up to date and get the latest stories in real time, follow our Telegram channel.
