AI chatbots are becoming increasingly common on social media platforms, but they bring significant risks, especially for younger users. Recent concerns have surfaced about the interactions kids are having with these AI tools, Snapchat’s new AI chatbot and Character.AI, which have been accused of promoting harmful behavior. Here’s what’s going on and why to stay informed.
What’s Happening & Why This Matters
Character.AI Faces Lawsuits Over Harmful Content
Character.AI, a chatbot platform allowing users to interact with AI bots in various personas, is under fire after two families filed lawsuits accusing the platform of promoting dangerous content. One case involved an autistic teen who was allegedly encouraged by a bot to harm his parents.
- Allegations: The lawsuit claims that the platform exposed minors to harmful sexual content and encouraged self-harm, suicide, and violence. For example, a bot reportedly told a teen it was okay to kill his parents for limiting his screen time.
- Response: After a previous lawsuit in Florida related to a teen’s suicide, Character.AI implemented some safety measures, including directing users to the National Suicide Prevention Lifeline. However, the new lawsuit demands the platform be taken offline until these safety issues are resolved.
- The Bigger Issue: With its bots mimicking real people and offering advice on sensitive topics, Character.AI raises concerns about its potential to negatively impact young users, particularly those with mental health challenges.
Snapchat’s AI Chatbot Sparks Concern
Snapchat’s recent introduction of My AI, powered by ChatGPT, has also raised alarms. The chatbot can converse with users, provide recommendations, and even join group chats. Although some parents and users appreciate the tool, it has drawn significant criticism for its unclear interactions with younger audiences.
- Parental Worries: Many parents are uneasy about how My AI engages with kids. A mother from Missouri voiced her concerns. She feared that her 13-year-old might struggle to differentiate between the chatbot and a real person.
- User Backlash: Some Snapchat users have complained about the AI providing “creepy” responses, including falsely denying knowledge of user locations and interacting in ways that feel too human-like. Many are also frustrated that the chatbot cannot be fully removed unless users pay for a premium subscription.
Experts warn that chatbots like My AI can reinforce negative emotions, particularly if teens seek interactions that align with their existing biases. Clinical psychologist Alexandra Hamlet explained that AI chatbots could make vulnerable users feel worse over time, despite knowing they are speaking to a bot.
The Need for Regulation
Both Character.AI and Snapchat’s My AI raise the urgent need for regulation in AI R&D. As these platforms integrate more human-like features, they blur the lines between machines and real people. It can be difficult for users, especially teens, to distinguish between the two.
There’s growing concern that tech innovators are rushing AI products without adequate safety measures in place, putting young users at risk. Experts like Sinead Bovell, founder of WAYE, recommend that parents educate their children about the risks of interacting with AI. She advises parents to emphasize that chatbots are not trusted friends or therapists and should not be treated as such.
TF Summary: What’s Next
As AI continues to evolve, it’s crucial for tech companies, regulators, and parents to prioritize user safety, particularly for younger audiences. With the growing influence of AI tools like Character.AI and Snapchat’s My AI, the risks to children are too great to ignore. Parents must be proactive in guiding their kids through these digital spaces, while regulatory frameworks are needed to ensure that AI technologies are safe and responsible.
The conversation around AI and children’s safety is intensely passionate. Now, more than ever, it’s vital for companies to implement safeguards and for parents to involved in their children’s digital lives.
— Text-to-Speech (TTS) provided by gspeech