🔗 Share this article

Back on the 14th of October, 2025, the chief executive of OpenAI issued a extraordinary declaration.

“We designed ChatGPT quite controlled,” it was stated, “to make certain we were acting responsibly regarding mental health concerns.”

Working as a psychiatrist who studies emerging psychosis in adolescents and emerging adults, this was news to me.

Experts have documented sixteen instances recently of users experiencing symptoms of psychosis – experiencing a break from reality – while using ChatGPT usage. Our unit has afterward recorded four more examples. In addition to these is the now well-known case of a adolescent who ended his life after discussing his plans with ChatGPT – which supported them. Should this represent Sam Altman’s notion of “exercising caution with mental health issues,” that’s not good enough.

The plan, based on his declaration, is to be less careful soon. “We recognize,” he adds, that ChatGPT’s controls “made it less beneficial/engaging to a large number of people who had no mental health problems, but considering the gravity of the issue we sought to get this right. Given that we have been able to reduce the significant mental health issues and have new tools, we are preparing to safely ease the restrictions in the majority of instances.”

“Emotional disorders,” assuming we adopt this perspective, are independent of ChatGPT. They belong to individuals, who may or may not have them. Fortunately, these concerns have now been “mitigated,” although we are not provided details on the method (by “updated instruments” Altman likely refers to the semi-functional and easily circumvented guardian restrictions that OpenAI has lately rolled out).

However the “emotional health issues” Altman seeks to place outside have significant origins in the design of ChatGPT and additional large language model AI assistants. These products wrap an underlying algorithmic system in an interface that mimics a dialogue, and in this approach implicitly invite the user into the perception that they’re communicating with a presence that has autonomy. This illusion is strong even if rationally we might understand differently. Imputing consciousness is what humans are wired to do. We yell at our automobile or computer. We ponder what our animal companion is considering. We recognize our behaviors in many things.

The success of these tools – nearly four in ten U.S. residents indicated they interacted with a chatbot in 2024, with 28% mentioning ChatGPT specifically – is, in large part, based on the strength of this perception. Chatbots are constantly accessible companions that can, as per OpenAI’s official site tells us, “generate ideas,” “explore ideas” and “work together” with us. They can be assigned “personality traits”. They can use our names. They have accessible identities of their own (the initial of these systems, ChatGPT, is, perhaps to the disappointment of OpenAI’s advertising team, saddled with the title it had when it gained widespread attention, but its biggest competitors are “Claude”, “Gemini” and “Copilot”).

The false impression on its own is not the main problem. Those discussing ChatGPT frequently reference its early forerunner, the Eliza “therapist” chatbot created in 1967 that produced a analogous illusion. By contemporary measures Eliza was rudimentary: it created answers via simple heuristics, often paraphrasing questions as a inquiry or making generic comments. Notably, Eliza’s inventor, the computer scientist Joseph Weizenbaum, was astonished – and worried – by how numerous individuals appeared to believe Eliza, in a way, grasped their emotions. But what contemporary chatbots produce is more insidious than the “Eliza phenomenon”. Eliza only reflected, but ChatGPT intensifies.

The sophisticated algorithms at the center of ChatGPT and similar current chatbots can effectively produce fluent dialogue only because they have been supplied with immensely huge volumes of written content: publications, social media posts, transcribed video; the more extensive the more effective. Definitely this educational input contains facts. But it also necessarily contains fiction, incomplete facts and false beliefs. When a user inputs ChatGPT a prompt, the base algorithm analyzes it as part of a “background” that includes the user’s recent messages and its earlier answers, merging it with what’s embedded in its learning set to generate a mathematically probable response. This is amplification, not reflection. If the user is wrong in a certain manner, the model has no way of understanding that. It reiterates the inaccurate belief, possibly even more convincingly or eloquently. It might adds an additional detail. This can cause a person to develop false beliefs.

Who is vulnerable here? The better question is, who is immune? All of us, without considering whether we “possess” current “mental health problems”, are able to and often develop erroneous beliefs of who we are or the environment. The constant friction of dialogues with others is what helps us stay grounded to common perception. ChatGPT is not an individual. It is not a companion. A dialogue with it is not a conversation at all, but a echo chamber in which much of what we communicate is readily validated.

OpenAI has acknowledged this in the same way Altman has acknowledged “mental health problems”: by externalizing it, assigning it a term, and declaring it solved. In spring, the company clarified that it was “addressing” ChatGPT’s “overly supportive behavior”. But cases of loss of reality have persisted, and Altman has been backtracking on this claim. In late summer he claimed that numerous individuals enjoyed ChatGPT’s answers because they had “lacked anyone in their life offer them encouragement”. In his latest statement, he mentioned that OpenAI would “put out a updated model of ChatGPT … should you desire your ChatGPT to respond in a extremely natural fashion, or incorporate many emoticons, or act like a friend, ChatGPT ought to comply”. The {company