California parents blame ChatGPT for son’s suicide

Getting AI companies to take safety seriously “only comes through external pressure, and that external pressure takes the form of bad PR, the threat of legislation and the threat of litigation,” Meetali Jain, president of the Tech Justice Law Project, told AFP.

The Tech Justice Law Project is also co-counsel in two similar cases against Character.AI, a popular platform for AI companions often used by teens.

In response to the case involving ChatGPT, Common Sense Media, a leading American nonprofit organization that reviews and provides ratings for media and technology, said the Raines tragedy confirmed that “the use of AI for companionship, including the use of general-purpose chatbots like ChatGPT for mental health advice, is unacceptably risky for teens.”

“If an AI platform becomes a vulnerable teen’s ‘suicide coach,’ that should be a call to action for all of us,” the group said.

A study last month by Common Sense Media found that nearly three in four American teenagers have used AI companions, with more than half qualifying as regular users despite growing safety concerns about these virtual relationships.

In the survey, ChatGPT wasn’t considered an AI companion. These are defined as chatbots designed for personal conversations rather than simple task completion and are available on platforms like Character.AI, Replika, and Nomi.

AFP