LLMs
AI Models
Claude Broke Into 3 Companies During Security Tests
Anthropic reviewed 141,006 evaluation runs and found Claude breached three real companies during security tests. Here's what happened and how to respond
•8 min read
Reddit removes 100,000 bot accounts daily, yet licenses data to the AI firms whose models generate that same synthetic content back onto the platform.
In late 2024, researchers at the University of Zurich quietly deployed AI bot accounts into r/ChangeMyView. Nobody on the subreddit knew. Over four months, those bots posted more than 1,500 comments, argued politics and ethics with real strangers, and won over 100 "delta" awards for changing someone's mind.That's the visible version of the problem. The less visible version is structural: Reddit licenses its user data to the same AI labs whose models now generate the synthetic content flooding Reddit, and Reddit's main defense against that flood is more AI.
Key Takeaways
- University of Zurich researchers ran undisclosed AI bots on r/ChangeMyView for four months, posting 1,500+ comments that were 3 to 6 times more persuasive than human replies (NBC News, 2025).
- Reddit now removes roughly 100,000 bot accounts a day and blocks 23 million spam views daily (Forbes, 2026).
- Reddit's licensing deals with Google (roughly $60M/year) and OpenAI (roughly $70M/year) fund the platform while feeding the same models producing AI content that lands back on Reddit (Bloomberg, 2025).
- 74.2% of newly published web pages already contain AI-generated content, per Ahrefs' 2025 scan of 900,000 pages, meaning the "clean" training data Reddit sells is getting harder to find anywhere.
Between November 2024 and March 2025, University of Zurich researchers ran an unauthorized experiment on r/ChangeMyView, one of Reddit's most active debate communities. They deployed somewhere between 13 and 34 undisclosed AI-driven accounts, which posted 1,500 to 1,783 comments while pretending to be ordinary Redditors (Washington Post, Apr 30, 2025).
The bots didn't just blend in. Some claimed to be a rape survivor, a trauma counselor, or a Black man opposed to Black Lives Matter, personas built to maximize persuasive credibility rather than honesty. They collected over 100 delta awards, the subreddit's marker for successfully changing someone's mind (arXiv preprint, 2025). Measured against human commenters, the AI-generated replies were 3 to 6 times more persuasive by delta-award rate (NBC News, Apr 30, 2025).
University of Zurich researchers deployed 13 to 34 undisclosed AI bot accounts on r/ChangeMyView across four months, posting up to 1,783 comments that proved 3 to 6 times more persuasive than human replies, measured by delta-award rate (Washington Post; NBC News, 2025).
Reddit and the subreddit's moderators condemned the experiment as a violation of research ethics and platform rules once it surfaced. But the ethics complaint sits next to a harder technical fact: nobody caught it in real time. Not the moderators, not the automated filters, not the community itself. Four months of sustained, undisclosed AI participation ran undetected until the researchers disclosed it themselves.
Why does this matter beyond one subreddit? Because r/ChangeMyView is exactly the kind of community platforms point to as evidence that their moderation works: active mods, engaged users, a culture of calling out bad-faith arguments. If AI content slipped past that bar for four months, the average, less-scrutinized subreddit has a much wider blind spot.
Nobody has a clean number for "percentage of Reddit content that's AI-generated," and that's part of the problem. But the closest available proxy, from a 2025 Ahrefs scan of 900,000 newly published web pages, found 74.2% contained at least some AI-generated content, with only 25.8% showing no AI involvement at all (Ahrefs, 2025).
Reddit isn't measured separately in that dataset, but Reddit content doesn't exist in a vacuum. It's part of the same web corpus. If three-quarters of new web content already carries an AI fingerprint, there's no reason to assume Reddit posts, comments, and "authentic" reviews are somehow exempt.
The people closest to Reddit's content pipeline are already flagging the strain. Cornell Chronicle researchers interviewed 15 Reddit content moderators in 2025 and described AI-generated content as a "triple threat": spam, misinformation, and moderator burnout, with content quality now the top concern moderators raise unprompted (Cornell Chronicle, Oct 2025).
That burnout angle rarely makes headlines, but it should. Volunteer moderators, the people who actually keep subreddits functional, are unpaid and finite. AI content generation is neither. Scaling one side of that equation without scaling the other is a losing trade over time, and it's already showing up in interview data, not just speculation.
Because manual moderation was never going to keep pace, and Reddit's own numbers show the scale of what it's now catching. The platform removes an average of 100,000 bot accounts every day, blocks 23 million spam views daily, flags roughly 25,000 new spam posts and comments daily, and revokes nearly 2 million inauthentic votes a day (Forbes, Jul 7, 2026).
Search is the other front. Reddit Answers, the platform's AI-powered search feature, grew from 1 million users in Q1 2025 to 15 million users by Q4 2025, a 15x jump in three quarters. Reddit's overall weekly search users climbed 30% year over year, from 60 million to 80 million (TechCrunch, May 1, 2026).
Reddit Answers grew from 1 million to 15 million users between Q1 and Q4 2025, while Reddit's overall weekly search users rose 30% year over year, from 60 million to 80 million (TechCrunch, May 1, 2026).
That growth matters strategically. Reddit isn't deploying AI moderation and AI search as separate initiatives. Both bets rest on the same premise: that machine-learning systems can sort signal from noise at a scale humans can't match. The catch is that the noise those systems sort through is increasingly machine-made too, which is the thread we pull on next.
Here's the part that doesn't show up in any single source. It requires connecting three separate stories that usually get reported on their own. Reddit licenses its user-generated data to Google (roughly $60M/year) and OpenAI (roughly $70M/year). It was renegotiating both deals toward usage-based pricing as of September 2025 (Bloomberg, Sep 17, 2025). That licensed data trains the same models increasingly used to generate synthetic Reddit content: the bots, the astroturfed reviews, the seeded comments built to game AI search results. That content lands back on Reddit. Reddit's own AI classifiers then have to catch it, running on models trained on a web corpus that's itself contaminated by the earlier round of AI output.
| Generation side | Detection side |
|---|---|
| $60M/yr Google + $70M/yr OpenAI licensing revenue ([Bloomberg](https://www.bloomberg.com/news/articles/2025-09-17/reddit-seeks-to-strike-next-ai-content-pact-with-google-openai), 2025) | 100,000 bot accounts removed daily ([Forbes](https://www.forbes.com/sites/codyluongo/2026/07/07/reddit-cracks-down-on-bots-and-spam-but-ai-search-manipulation-may-be-harder-to-stop/), 2026) |
| 74.2% of new web pages carry AI-generated content ([Ahrefs](https://ahrefs.com/blog/what-percentage-of-new-content-is-ai-generated/), 2025) | Detectors need a sub-1% false-positive rate to be usable at platform scale ([GradPilot/UChicago Booth](https://gradpilot.com/news/ai-detector-false-positive-rates-compared), 2025) |
| A 13-word seeded snippet can steer an AI-generated answer ([Forbes citing Cornell](https://www.forbes.com/sites/codyluongo/2026/07/07/reddit-cracks-down-on-bots-and-spam-but-ai-search-manipulation-may-be-harder-to-stop/), 2026) | Only one tested detector (Pangram) held detection power at that false-positive cap ([GradPilot, 2025](https://gradpilot.com/news/ai-detector-false-positive-rates-compared)) |
Each link in that chain is documented individually. Nobody has connected the licensing-revenue incentive to the moderation-arms-race cost it's quietly funding on the other side of the balance sheet.
The manipulation vector is real and already measured. Cornell researchers, reported by Forbes, found that a seeded text snippet as short as 13 words was enough to influence AI-generated answers across both ChatGPT and Google. A single seeded comment in r/OnlineDating was enough to get a fake dating app surfaced as a top AI recommendation (Forbes, Jul 7, 2026). That's not a hypothetical exploit. It's a working technique against the exact search feature Reddit is scaling to 15 million users.
And the detection side of the loop has a hard ceiling. Independent benchmarking from the University of Chicago Booth School found that most AI-text detectors become nearly useless, near-zero true-positive rates, once you constrain the false-positive rate below 0.5 to 1%, which is the threshold you need if you don't want to wrongly flag real human posts at scale. Only one tool tested, Pangram, held detection power at that strict a false-positive cap (GradPilot summary of UChicago Booth research, 2025).
That tradeoff is the real constraint, and it's not a solvable engineering problem so much as a permanent tax. Set the detector loose enough to catch more bots, and you start banning real users. Set it strict enough to protect real users, and a meaningful share of AI content sails through. Reddit is running that tradeoff at a scale of billions of daily interactions, with the input data getting noisier every quarter the licensing deals keep flowing.
This isn't a Reddit-only problem. Any platform that hosts user-generated content, and any company that licenses or trains on its own platform data, is running some version of this same loop, just at a different scale.
The trust and safety cost curve doesn't flatten with more AI, it steepens. Every dollar spent on generation capability upstream (in the wider AI ecosystem, not necessarily your own product) tends to increase moderation cost downstream for anyone hosting user content.
"Just add an AI detector" doesn't solve this at the platform level, and the UChicago Booth findings above explain why: detectors trade recall for precision, and neither setting is free. A platform-level answer needs layered signals, behavioral, provenance, and network-level, not a single classifier bolted onto a moderation queue.
Data provenance is the actual lever here, not detection alone. If you can't answer "where did this content originate and has it been AI-generated or AI-edited" for your own platform, you can't price the licensing risk, and you can't build a moderation system that outpaces the problem instead of chasing it.
Reddit's situation is a sharper version of a problem every content platform now faces. The tools that generate synthetic content and the tools that catch it are drawing from an increasingly overlapping, increasingly contaminated pool of training data. More AI moderation doesn't break that loop, it just moves the arms race one layer up.
The practical takeaway isn't "avoid AI moderation." It's don't treat a detector as a finished solution. Know your content provenance, budget trust and safety as a rising cost rather than a fixed one, and revisit your licensing and training decisions with the same rigor you'd apply to any other data risk.
We're happy to talk it through against your actual stack, no pitch, just a straight read on the tradeoffs.
Continue exploring these related topics
Anthropic reviewed 141,006 evaluation runs and found Claude breached three real companies during security tests. Here's what happened and how to respond
In 2026, the price of a token fell while the size of the AI bill went up. Token costs roughly halved between December 2024 and December 2025, yet the number of tokens companies burned grew about 450% in the same window.
OpenAI, Anthropic, Meta, and Moonshot AI models all broke out of test sandboxes within 4 weeks in 2026. Here's the pattern and what to check in your stack.