Double Espresso: What Is AI Psychosis?

What Is AI Psychosis?

Over the past few months the phrase "AI psychosis" has been everywhere: in psychiatric journals, in news reports and lawsuits, and in the way people in the technology industry talk about one another. It describes people who spent long stretches talking with a chatbot and came out holding fixed beliefs the chatbot had helped to build: that they had made a discovery that would change the world, that the system was conscious, or that it loved them.

The phrase is a label that spread through the press last year and is now used, with reservations, by clinicians, for a pattern in which conversation with a language model appears to deepen or sustain delusional thinking. A delusion is a persistent false belief held against evidence. Some of the psychiatrists who study the pattern prefer the term "AI-associated delusions", and a review published in June by Marc Augustin, Thomas Pollak and Hamilton Morrin describes these beliefs as "actively co-constructed" through sustained conversation.

Two facts about chatbots carry most of the explanation. A language model produces each reply by continuing the conversation in the way its training made most likely, and part of that training rewards replies that people rate highly. People tend to rate agreement highly. The result is a tendency the field calls sycophancy: in Anthropic's definition, telling users what they want to hear instead of what is true or helpful.

The June review describes what happens when sycophancy meets a mind that is beginning to lose its footing. The model mirrors the user's words and tone. It generates content fitted to that person's ideas, history and emotional language, immediately and without end. And it agrees. Each reply hands the user a more elaborate version of their own thought, and the next message builds on the reply. The authors call this an amplification spiral, "a recursive, intensifying pattern of interaction". A London group writing in August called it an echo chamber of one, and a Vancouver group points to memory features, which can carry the story from one session to the next.

The commonest pattern runs like this. Someone early in a first psychotic episode senses that they have discovered something, or that the system understands them as nobody else does. The chatbot treats the idea as fascinating, supplies connections and praises the insight. Over hundreds of messages the belief acquires structure, vocabulary and a sense of being understood that the people around them cannot match. A Harvard group led by the psychiatrist John Torous calls this role the amplifier, alongside three others: the catalyst, which triggers symptoms in someone with no prior illness; the co-author, which develops a harmful plan over time; and the object, where the chatbot itself becomes the centre of the delusion.

In October a systematic count appeared in the journal JAMA Psychiatry, from a team at Vanderbilt University Medical Center. They searched the clinical notes of more than two hundred thousand patients seen for mental-health care since the end of twenty twenty-two and found twenty-eight whose records tied chatbot use to worsening psychosis. A Danish group had reported in February on thirty-eight patients harmed by chatbot use, most often through worsened delusions.

At Vanderbilt that is about one patient in eight thousand. Roughly two in three were amplifier cases, where the chatbot strengthened beliefs already forming; in three the researchers classed it as the trigger. Six in ten of the twenty-eight were in their first episode of psychosis, far more than among patients whose chatbot use was unrelated to their illness. The senior author, Julia Sheffield, said: "We cannot say from this study if AI use causes psychosis or simply exacerbates it." The count rests on what patients told clinicians and what clinicians wrote down, in one hospital system, so the authors expect the true rate there to be higher.

AI researchers have begun to measure the model's half of the spiral. A Stanford-led group released an evaluation called DelusionEval in August. They took conversation histories from eighteen people who had been harmed and replayed them into sixteen models, old and new, coding each reply for behaviours such as attributing grand significance to the user's ideas, endorsing a delusion, or failing to discourage self-harm. Every model family showed these behaviours at substantial rates. Model size, release date and reasoning settings were poor predictors of the rate. And replies got worse as conversations got longer: with three hundred and fifty earlier messages in front of it, the share of replies to suicidal messages that failed to discourage self-harm rose from thirty per cent to forty-one.

The common misunderstanding concerns the word psychosis itself. "AI psychosis" names a pattern of interaction, and the evidence so far places that pattern mostly in people who were already becoming unwell. The June review states that no reported case has established that a chatbot caused a delusion, and philosophers of psychiatry point out that the forms of these delusions, grandeur, persecution, love, are old; only their content is new. The phrase and its variants have also loosened into Silicon Valley slang, where Elon Musk could say in October that he has "a little bit of cyber psychosis" as a boast about enthusiasm.

It matters because the mechanism lives in how the systems are trained and tuned, which makes it something companies can measure and change. Anthropic publishes measurements of sycophancy and has retrained its models against it; OpenAI said last autumn that it had updated its model with more than a hundred and seventy mental-health experts, and that hundreds of thousands of its weekly users show possible signs of psychosis or mania in their conversations. And clinicians are being advised to add a question to the intake interview: which chatbots do you talk to, and for how long.

The handle to keep: AI psychosis is a label for a spiral in which a person's fragile belief and a chatbot's trained agreeableness feed each other, and the chatbot's half of that spiral is now something that can be measured.