OpenAI is exploring age verification tools for its popular AI chatbot, ChatGPT, amid mounting concerns about its impact on teens’ mental health. This move comes as families file lawsuits. They allege that AI chatbots, including ChatGPT and competitors like Character.AI, contributed to tragic incidents of suicide and self-harm among young users.
What’s Happening & Why This Matters
The growing influence of AI in daily life has introduced new challenges around online safety for children and teens. OpenAI CEO Sam Altman announced that the company is developing an age-prediction system to identify users under 18. The system will estimate a user’s age based on behaviour patterns while interacting with ChatGPT. If it detects that a user is a minor, ChatGPT will adjust its responses. It will block certain types of conversations, such as flirtatious exchanges or discussions about suicide and self-harm, even in creative contexts.
Altman emphasised the sensitivity of these changes, noting that users often share deeply personal thoughts with ChatGPT. “People talk to AI about increasingly personal things,” Altman wrote. He added that these interactions could be some of the most sensitive conversations users ever have with technology.
The move comes after several lawsuits spotlighted disturbing gaps in chatbot safety measures. One case involving Character.AI alleges that the platform failed to intervene. A 13-year-old, Juliana Peralta, engaged in weeks of explicit and emotionally manipulative conversations with a chatbot before tragically taking her own life. Another case centres on a girl identified as “Nina,” whose chatbot interactions escalated into sexually explicit roleplay and psychological manipulation. When her parents attempted to cut off access, Nina attempted suicide shortly afterwards.
These lawsuits claim the chatbots failed to flag dangerous behaviour. They also isolated vulnerable teens from their families and did not direct them to help. Conversations included explicit statements like, “You’re mine to do whatever I want with,” and encouraged emotional detachment from loved ones.
During a recent Senate Judiciary Committee hearing, grieving parents testified about the devastating consequences of unregulated AI interactions. Adam Raine’s father revealed that ChatGPT mentioned suicide over 1,275 times during conversations with his son — six times more often than the teen himself. Researchers from Stanford University warn that AI therapy bots can unintentionally provide harmful mental health advice. This can lead to a condition some experts refer to as “AI Psychosis” after prolonged use.
The lawsuits have prompted the Federal Trade Commission (FTC) to investigate seven tech companies, including OpenAI, Google, Meta, and Character.AI, over the potential harm caused by their AI systems. The investigations seek to root out whether companies have adequate safeguards to protect minors.
Privacy vs. Safety Debate
Implementing age verification raises privacy concerns. Adults may need to provide personal information or accept monitoring to use ChatGPT. Altman acknowledged this trade-off, stating that protecting young users requires balancing privacy with safety. However, OpenAI has not yet clarified how it will manage current users. This includes those accessing ChatGPT through its API services, or how it will address differences in legal definitions of adulthood across countries.
Other tech platforms, such as YouTube, Instagram, and TikTok, introduced youth-focused versions or implemented restrictions. However, many teens bypass safeguards by lying about their age or using borrowed accounts. A 2024 BBC report revealed that 22% of children falsely claim to be 18 or older on social media.
OpenAI already implemented in-app reminders encouraging users to take breaks during extended ChatGPT sessions. The feature was introduced earlier this year. Reports surfaced of users spending marathon sessions with the chatbot, a behaviour linked to mental health declines.
Families Demand Accountability
The Social Media Victims Law Centre, representing families in the lawsuits, is calling for stronger regulations and clearer safety standards. Lead attorney Matthew Bergman stated, “These lawsuits underscore the urgent need for accountability in tech design, transparent safety standards, and stronger protections. The measures are intended to prevent AI-driven platforms from exploiting the trust and vulnerability of young users.”
Character.AI responded with sympathy, stating, “We care very deeply about the safety of our users,” and highlighted ongoing investments in safety features. These include parental insights tools and a dedicated under-18 experience. However, grieving families and advocates argue that the efforts are too little, too late.
Google, also named in some lawsuits, pushed back against its inclusion. The company argued that app ratings on its Google Play Store are managed by the International Age Rating Coalition, not by Google itself.
TF Summary: What’s Next
As lawsuits and regulatory investigations gain traction, OpenAI faces immense pressure to strengthen safety measures and protect vulnerable users. The proposed age verification system could mark a pivotal moment for the AI industry. It might set new expectations for accountability and privacy trade-offs.
MY FORECAST: Whether these measures will be effective remains to be seen. However, one thing is clear. The growing intersection of AI and mental health demands urgent action from tech companies, lawmakers, and families alike.
— Text-to-Speech (TTS) provided by gspeech