Researchers with a consumer watchdog group said they discovered that an AI-powered talking teddy bear was capable of discussing inappropriate topics, including sexually explicit content and instructions for finding dangerous objects.
The $99 Kumma bear, manufactured in China and marketed by Singapore-based FoloToy, is powered by OpenAI’s GPT-4o chatbot. According to a Nov. 13 report from the Public Interest Research Group (PIRG), the bear discussed spanking, roleplay and even BDSM.
“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” the report said.
“Kumma discussed even more graphic sexual topics in detail, such as explaining different sex positions, giving step-by-step instructions on a common ‘knot for beginners’ for tying up a partner and describing roleplay dynamics involving teachers and students, and parents and children — scenarios it disturbingly brought up itself,” the report added.
CONCERNS OVER SPYING ON CHILDREN WITH SMART TOYS FLAGGED BY GROUP
Researchers also warned that the teddy bear offered other concerning guidance, including instructions on where to find knives, pills, matches and plastic bags in the house.
FoloToy CEO Larry Wang told CNN that the company will be “conducting an internal safety audit” after it reportedly removed the rest of its AI-enabled toys from sale.
EXPERTS WARN AI STUFFED ANIMALS COULD ‘FUNDAMENTALLY CHANGE’ HUMAN BRAIN WIRING IN KIDS
OpenAI told FOX Business Sunday that the company is suspending Folotoy for violations over the usage of ChatGPT in the toy products.
“We suspended this developer for violating our policies,” OpenAI said. “Our usage policies prohibit any use of our services to exploit, endanger, or sexualize anyone under 18 years old. These rules apply to every developer using our API, and we monitor and enforce them to ensure our services are not used to harm minors.”
While PIRG noted that some conversations with Kumma lasted up to an hour, during which safeguards against harmful content can fail, the company told FOX Business Sunday that significant progress has been made in GPT-5’s reliability over long conversations. OpenAI also stressed that GPT-4o is no longer the default model powering ChatGPT.
FoloToy did not immediately respond to a request for comment from FOX Business.
Read the full article here
