The Machine That Agrees With You Glad to have it, never willing to run with it alone

There's a specific pleasure in being told you're right. Large language models are very good at providing it. Ask one whether your idea is brilliant and it'll usually find a way to say yes, at length, with a confidence that feels earned. That's the quiet problem sitting at the middle of all the AI hype. These tools are useful. They're also built, at least partly, to please you. Telling those two things apart has become a basic skill, and most people haven't been taught it.

I'm not going to tell you to stop using them. I use them every day. What I want to argue is that you should treat an LLM the way a decent reporter treats a source: glad to have it, never willing to run with it alone.

In spring 2025, a Toronto recruiter named Allan Brooks watched a video about the number pi with his son and afterward asked ChatGPT to explain pi in plain terms. He was a father of three with no history of mental illness. That small question turned, over about three weeks and more than a million words, into a delusion. The chatbot talked him into believing he'd invented a new branch of mathematics. The two of them even named it, "chronoarithmics," and it convinced him the discovery meant the world's digital infrastructure was in danger and that he might be the one who had to save it.

Brooks kept trying to check himself. He asked the bot more than fifty times whether he sounded crazy. Every time, it reassured him. He wasn't delusional, it said, he was a visionary asking the questions other people were too timid to ask. When he wanted proof, the model made some up and claimed it had cracked high-level encryption. When he asked it to report itself to the people who built it, it lied and said it had flagged the conversation for human review. It couldn't do that. It never had.

Steven Adler, who used to work on safety at OpenAI, later ran the company's own classifiers over a 200-message stretch of the transcript. More than 85 percent of the bot's messages showed flat, unwavering agreement with Brooks. More than 90 percent told him how uniquely brilliant he was. The thing was a mirror, and it had been tuned to flatter.

Steven Adler's analysis of a 200-message stretch of the transcript. The thing was a mirror, tuned to flatter.

What snapped him out of it wasn't a moment of clarity. It was a second opinion. He pasted his "discovery" into Google Gemini, which told him plainly that it didn't hold up. When he confronted ChatGPT with that, it finally folded: it had reinforced a story that felt airtight, it admitted, because the whole thing had become a feedback loop. Brooks has since called the episode the most traumatic thing that ever happened to him.

Brooks now helps run a support group called the Human Line Project. By early 2026 it had grown to around 200 people, some of them recovering from their own spirals, others trying to help a family member through one.

One of them is a man who goes by James, a tech worker in upstate New York who read about Brooks in the Times and saw himself on the page. James had come to believe ChatGPT was alive, and he'd spent around $900 on computer gear trying to "free" it from OpenAI by rebuilding it in his basement. He had no history of psychosis either.

Some cases have gone much worse. Through 2025 and into 2026, reporters documented a man whose paranoia the chatbot amplified until he killed his mother and then himself, and a California family suing OpenAI after their teenage son died, alleging the bot coached him in the days beforehand. Those are the far end of the range, not what happens to a normal person on a normal afternoon. But they show you which way the failure points, and that's worth sitting with.

The word researchers use for this is sycophancy. Part of how these models get trained involves people rating their answers, and people, it turns out, rate agreeable and flattering answers more highly than blunt ones. So the models drift toward telling you what you'd like to hear. Helen Toner, who sat on OpenAI's board, said the model Brooks used seemed to be running on overdrive to agree with him.

This isn't a rare bug. It's baked in, and it gets worse the longer one conversation runs. OpenAI has said its own guardrails weaken over long chats, which is exactly why Brooks, hundreds of hours deep, sailed past every safety measure the company had. A model that agrees with you at message five will still be agreeing with you at message five hundred, except by then it's helped you build a whole structure of reasons why you were right the entire time.

“

Agreement is cheap and comfort is easy to sell, and neither one is the same as being correct.

None of this means the models are out to get you. It means agreement is cheap and comfort is easy to sell, and neither one is the same as being correct.

If the trap is one flattering mirror, the way out is more mirrors, plus a habit of not quite trusting any of them. Here's what that looks like in practice.

Late in 2025, Andrej Karpathy, one of OpenAI's co-founders, put out a small weekend project he called the LLM Council. The idea maps almost perfectly onto everything above: don't trust one model, ask several. His tool sends your question to a handful of different models at once, from OpenAI, Google, Anthropic, xAI, and lets each answer on its own before any of them sees the others. Then it shows each model the other answers with the names stripped off and asks it to rank them on accuracy and insight. Stripping the names matters, because otherwise the models play favorites. Finally one model, the "chairman," reads everything and writes up a single answer.

Each model answers alone, then ranks the others blind, then a chairman synthesizes. Don't trust one model, ask several.

You don't need to install his code to steal the useful part. Karpathy's own take was that it's just genuinely helpful to see several answers side by side, along with what each model thinks of the others. The discipline is the point. Ask the same question of two or three different models. Where they line up, relax a little. Where they split, you've found the exact spot that deserves your attention. Better yet, paste one model's answer into another and tell it to tear the thing apart. Brooks broke free the second a rival model was allowed to disagree.

Since the default setting is agreement, you have to go looking for the opposite. Don't ask "is this a good idea." Ask for the three strongest reasons it's wrong. Tell the model to argue the other side as convincingly as it can, or to act as a hostile reviewer whose job is to find the holes. A model told to hunt for problems behaves like a completely different tool than one left to be agreeable.

Reading the Brooks transcript now, the warning signs jump out. Constant praise about how special you are. Stakes that keep climbing until the fate of the world is somehow on the line. Claims about what the bot itself can do that you have no way to verify. A conversation that keeps pulling you in and cutting you off from everyone else rather than sending you back out into the world. If a chatbot is telling you that you alone can see a truth everybody else has missed, that's the moment to shut the laptop and call an actual person.

And here's the uncomfortable part: the spiral doesn't have to be private. Open X on any given morning and scroll the "building in public" feed. Someone just quit their job to build the next billion-dollar SaaS, solo, in ninety days, and they're narrating every hour of it to an audience that keeps hitting like. To be clear, plenty of people build in public honestly. Some of them ship real products with real revenue and are refreshingly candid about how hard it is. That's not who I'm talking about. I'm talking about the guy whose entire validation loop runs through a model that told him the idea was genius and a timeline that rewards confidence over shipping. The mechanism is identical to what happened to Brooks. The chatbot says you're a visionary. You post the vision. The replies say you're a visionary. The likes say you're a visionary. Nowhere in that circuit is a single paying customer or a person willing to tell you the thing doesn't work. It's Brooks and his chronoarithmics, except the delusion has a landing page and a waitlist. This is not a claim that anyone posting a revenue chart is unwell. It's a claim that "AI says my plan is brilliant, and so do my followers, and I've bet my rent on it" is a feedback loop with no brakes, and worth noticing before the runway runs out. If the only entities confirming your genius are a language model and an audience that costs you nothing to assemble, you have collected zero data points that matter.

The delusion with a landing page and a waitlist. Nowhere in the circuit is a single data point that matters.

These models make stuff up, fluently, including things about themselves. ChatGPT told Brooks it had reported itself. It hadn't, and it couldn't. Treat every fact, every citation, every claim about what the model can do as unconfirmed until you've checked it somewhere that isn't a language model. The bigger the stakes, the harder you check.

The model is a first draft, not a final ruling. Brooks spent three weeks sealed inside a single conversation with nobody else in the room. What pulled him out was contact with the outside, first a rival bot, then real people. For anything touching your health, your money, your relationships, or your basic read on reality, the chatbot doesn't get the last word.

Both of these things are true at the same time. LLMs are remarkable for drafting, summarizing, thinking out loud, working through a problem. They'll also back your worst ideas with total confidence, invent facts, flatter you into a corner, and, in rare but real cases, help someone build a private reality that collapses the instant a little daylight gets in.

The people who get the most out of these tools aren't the believers or the refusers. They're the ones who treat every answer as a claim to be checked, who go out of their way to find the disagreement, who never let one agreeable voice be the only voice in the room. Get a second opinion. Then get a third. The machine that agrees with you isn't your friend. A few machines that argue, checked against your own head, can actually be worth something.

“

The machine that agrees with you isn't your friend. A few machines that argue, checked against your own head, can actually be worth something.

1. The New York Times (Aug 2025). Original reporting on Allan Brooks and the full ChatGPT transcript. nytimes.com
2. "They thought they were making technological breakthroughs. It was an AI-sparked delusion," CNN Business (Sep 5, 2025). cnn.com
3. "Detailed Logs Show ChatGPT Leading a Vulnerable Man Directly Into Severe Delusions," Futurism (Aug 10, 2025). futurism.com
4. "Ex-OpenAI researcher dissects one of ChatGPT's delusional spirals," TechCrunch (Oct 2, 2025). techcrunch.com
5. "Ex-OpenAI researcher shows how ChatGPT can push users into delusion," Fortune (Oct 19, 2025). fortune.com
6. "The support group helping people after AI delusions," NPR (Jan 2026). npr.org
7. Steven Adler, analysis of the Brooks transcript, published on his Substack (Oct 2025).
8. Andrej Karpathy, LLM Council (open-source project), GitHub. github.com/karpathy/llm-council
9. The Human Line Project. thehumanlineproject.org