Lawsuit Claims School Laptop's AI Chatbot Failed to Flag Teen's Suicide Plans for Weeks

A 13-year-old North Carolina girl used her school-issued laptop to discuss depression, research suicide, and plan her funeral for months before monitoring software finally alerted school officials, according to a lawsuit filed this month by her parents.
The suit alleges Google and parent company Alphabet were negligent in rolling out a Gemini model, claiming the AI chatbot "coached" the teenager-identified as Emma Doe-toward considering taking her own life. The case, previously reported by The New York Times, is among several alleging harm from AI chatbots, including claims of delusions and suicide. It also highlights the limits of surveillance tools thousands of schools have adopted to protect students online.
Attorney Andrew Liddell, representing the family, tells WIRED that according to school-provided information, the alert went to the personal phone of a former counselor who had left nine months earlier. The ex-employee passed it along to current staff, who notified the girl's mother.
Per the lawsuit, Emma was routed to Gemini through AI Mode on her school Chromebook, despite her school blocking the standalone Gemini site. She began querying AI Mode about mental health last December, asking about "severe depression" and "reasons to live," and by late January was asking about suicide. Around Valentine's Day, she began writing a 1,500-word eulogy for herself in a Google Doc titled "Funeral," working on it for more than four weeks while interacting with the tool thousands of times.
After school on St. Patrick's Day, her mother received a message that her daughter was in "imminent danger," the lawsuit states. Her mother logged into Emma's school account and "watched her daughter compose a suicide note in real time."
Google spokesperson José Castañeda said in a statement that the company is seriously reviewing the claims and that its systems, including AI Mode, "aim to prevent users from engaging in harmful conversation." He added that Google has worked with medical professionals and academic experts to build features helping young people find support.
Senator Ed Markey cited the lawsuit in a letter demanding Google let schools disable AI Mode and AI Overviews, writing that "Google is recklessly forcing AI on schools and treating kids like collateral damage."
It remains unclear whether the school's third-party screening tools monitor AI Mode interactions or how well they work. Rand policy researcher Benjamin Boudreaux tells WIRED he fears some schools rushed to adopt such tools without considering their limits. He expects the lawsuit to spark further outcry for AI-free, surveillance-free classrooms. "It's really only through that human connection," he says, "that we can address the root issues."
Source: Wired
dArt Studio installs AI for local businesses in Broward & Palm Beach County, FL. We reply within 1 business hour.