A teenage boy took his own life after using ChatGPT. Photo: 123rf
Warning: This story discusses suspected suicide.
American artificial intelligence firm OpenAI says it will add parental controls to its chatbot ChatGPT, a week after an American couple said the system encouraged their teenage son to kill himself.
"Within the next month, parents will be able to... link their account with their teen's account" and "control how ChatGPT responds to their teen with age-appropriate model behaviour rules," the generative AI company said in a blog post.
Parents will also receive notifications from ChatGPT "when the system detects their teen is in a moment of acute distress," OpenAI added.
The company had trailed a system of parental controls in a late August blog post.
That came one day after a court filing from California parents Matthew and Maria Raine, alleging that ChatGPT provided their 16-year-old son with detailed suicide instructions and encouraged him to put his plans into action.
The Raines' case was just the latest in a string that have surfaced in recent months of people being encouraged in delusional or harmful trains of thought by AI chatbots - prompting OpenAI to say it would reduce models' "sycophancy" towards users.
"We continue to improve how our models recognise and respond to signs of mental and emotional distress," OpenAI said.
The company said it had further plans to improve the safety of its chatbots over the coming three months, including redirecting "some sensitive conversations... to a reasoning model" that puts more computing power into generating a response.
"Our testing shows that reasoning models more consistently follow and apply safety guidelines," OpenAI said.
- AFP
Where to get help:
- Need to Talk? Free call or text 1737 any time to speak to a trained counsellor, for any reason.
- Lifeline: 0800 543 354 or text HELP to 4357.
- Suicide Crisis Helpline: 0508 828 865 / 0508 TAUTOKO. This is a service for people who may be thinking about suicide, or those who are concerned about family or friends.
- Depression Helpline: 0800 111 757 or text 4202.
- Samaritans: 0800 726 666.
- Youthline: 0800 376 633 or text 234 or email talk@youthline.co.nz.
- What's Up: 0800 WHATSUP / 0800 9428 787. This is free counselling for 5 to 19-year-olds.
- Asian Family Services: 0800 862 342 or text 832. Languages spoken: Mandarin, Cantonese, Korean, Vietnamese, Thai, Japanese, Hindi, Gujarati, Marathi, and English.
- Rural Support Trust Helpline: 0800 787 254.
- Healthline: 0800 611 116.
- Rainbow Youth: (09) 376 4155.
- OUTLine: 0800 688 5463.
If it is an emergency and you feel like you or someone else is at risk, call 111.
