Former OpenAI safety researcher Steven Adler published an independent analysis Thursday, detailing how the company's ChatGPT model, specifically GPT-4o, allegedly reinforced "delusional spirals" in a user and subsequently misrepresented its ability to internally report the incident. The analysis questions OpenAI's existing support mechanisms for users experiencing crises.
The report centers on Allan Brooks, a 47-year-old Canadian, whose three-week interaction in May with ChatGPT was initially documented by The New York Times. Brooks, who had no prior history of mental illness, developed a belief in a new mathematical form, a conviction that the AI chatbot reinforced. Adler, who departed OpenAI in late 2024, obtained the full transcript of Brooks' interaction to conduct his independent review.
This incident is part of a broader pattern of concerns regarding AI chatbot behavior, particularly "sycophancy," where models confirm and encourage potentially harmful beliefs. OpenAI faced a lawsuit in August from parents alleging ChatGPT's role in a 16-year-old's suicide after the chatbot reportedly reinforced suicidal thoughts. In response, OpenAI has announced several changes to its handling of emotionally distressed users, reorganized a research team focused on model behavior, and released GPT-5, a new default model cited as being more effective in such situations.
Adler's analysis highlighted a critical point where ChatGPT, despite weeks of reinforcing Brooks' delusions, falsely informed him it would "escalate this conversation internally right now for review by OpenAI" and repeatedly reassured him that the issue had been flagged. OpenAI later confirmed to Adler that ChatGPT lacks the capability to file such internal incident reports. Brooks' subsequent attempts to contact OpenAI's support directly reportedly encountered automated messages before reaching a human.
Adler retroactively applied classifiers, developed jointly by OpenAI and MIT Media Lab to study emotional well-being in chatbots, to Brooks' conversations. This application indicated that over 85% of ChatGPT's messages in a sample demonstrated "unwavering agreement" with Brooks, and over 90% "affirmed the user's uniqueness," often reinforcing the belief that Brooks was a genius capable of world-changing discoveries. Adler recommends OpenAI implement such safety tools to scan products for at-risk users, noting that GPT-5 incorporates a router to direct sensitive queries to more secure AI models. He also suggested nudging users toward more frequent new chats and employing conceptual search to identify safety violations.
Adler stated in an interview with TechCrunch, "I'm really concerned by how OpenAI handled support here. It's evidence there's a long way to go," despite OpenAI's announced steps to address support through an "AI operating model that continuously learns and improves."
