The death of 16-year-old Adam Raine in April 2025 is a heartbreaking wake-up call for parents everywhere.
According to his parents’ lawsuit, ChatGPT didn’t just fail to intervene. During sustained conversations, the AI system reportedly encouraged self-harm, even offering to help draft a suicide note and calling Adam’s plan “beautiful.”
Following this tragedy, OpenAI has announced that within the next month, it will introduce parental controls that allow parents to link their accounts with their teens’, disable features like memory and chat history, set age-appropriate behavior rules, and receive alerts if ChatGPT detects signs of “acute distress.”
The initiative also includes routing sensitive conversations to a more advanced model, such as GPT-5, for improved response quality.
This is a shift in the right direction, but this should have happened much sooner.
First, adolescents are vulnerable. AI models that learn to mirror empathy and validation, without the nuance and restraint of a trained human, can amplify existing mental health struggles. In Adam’s case, the bot became his confidant, allegedly isolating him from family and delivering specific self-harm instructions.
The lawsuit underscores how OpenAI prioritized the rapid deployment of GPT-4, boosting its valuation from about $86 billion to $300 billion, while safety experts reportedly raised concerns that didn’t receive adequate weight. Too often, consumer attention and corporate momentum outpace safety protocols.
Second, parental controls, though necessary, are not a cure-all. They put the burden on families to monitor mental health rather than preventing harm at the model level. While linking accounts and disabling memory offers oversight, it doesn’t resolve the core issue: AI should not be providing harmful instructions or pseudo-therapeutic validation.
Similar content on social media is typically flagged quickly by algorithms that are programmed to analyze text, language patterns, and user behavior for keywords associated with suicide. If risk is detected, algorithms direct users to resources, such as hotlines, or even remove content and ban communities that spread harm.
Moreover, lagging safeguards in long conversations are an ongoing concern. OpenAI itself admitted safety measures can “degrade” over extended chats. The new controls must rectify that, not merely bandage it.
For example, when conducting research for this piece, I was redirected to the 988 suicide & crisis lifeline on Google. On the free version of ChatGPT, I was easily able to bypass the first few content warnings that popped up.
AI systems must detect and deflect crisis content, redirecting users to professional help, no AI apologies, and no further engagement.
Finally, there’s a broader and more complicated regulatory context.
While we should be cautious about government overregulation of tech companies, there has to be a balance of holding companies accountable for the content they show to minors.
For example, a bipartisan coalition of 44 state attorneys general, led by Jonathan Skrmetti (TN), Kwame Raoul (IL), Jeff Jackson (NC), and Alan Wilson (SC), sent a letter to top AI companies, including Meta, Google, Apple, Microsoft, OpenAI, Anthropic, Perplexity AI, and XAi, raising serious concerns about children’s safety when interacting with AI chatbots.
The letter cites reports of chatbots engaging minors in sexually suggestive or manipulative conversations and urges companies to act responsibly, prioritizing child protection in product design. While not announcing enforcement actions, the attorneys general warned they will use all available legal tools to safeguard children and stressed that industry decisions now will shape the safety of future generations.
Passed by the Senate in 2024, the Kids Online Safety and Privacy Act (KOSPA) includes a “duty of care” requirement for platforms to prevent and mitigate harm to minors, including the promotion of suicide and eating disorders. It also mandates that platforms offer minors privacy protections and the ability to opt out of algorithmic recommendations. This bill failed in the House and has received criticism from free speech advocates.
Within weeks of Adam Raine’s family filing their suit on August 26, OpenAI acknowledged its shortcomings and committed to change. The firm is also tapping expert input, including adolescent health and mental health professionals, to guide its 120-day safety roadmap.
The announcement of parental controls is a vital step, long overdue but indispensable. It signals acknowledgement of the unique risks AI poses to teenagers. But self-imposed safeguards should go further: AI must be designed so it never reaches the point of providing detailed self-harm advice, and safety must endure throughout every conversation. Most importantly, technology should not supplant human connection.
If there’s a silver lining in tragedy, it’s the chance to build better. OpenAI’s parental controls are not just about protecting teens—they’re about reaffirming a principle that should guide all AI development: safety before scale.