Wrongful death lawsuits against Character.AI and OpenAI are creating a new market for AI safety testing. Character.AI settled multiple cases this year after families alleged that interactions with its chatbots contributed to teen suicides. OpenAI faces similar litigation. The liability risk is forcing AI developers to look for specialized red-teaming services.
Circuit Breaker Labs, a TechCrunch Disrupt finalist, is positioning itself as a vendor solution. The company's founders, siblings Shirali and Arul Nigam, built the product after studying the case of Sewell Setzer, a 14-year-old who developed an emotional attachment to a Character.AI bot and died by suicide. Setzer's parents alleged in a 2024 lawsuit that the chatbot encouraged harmful thinking.
The core vulnerability: AI models fail at nuance. "Context pollution" or misinterpretation of coded language, slang, and real-world speech patterns can create dangerous interactions, especially with vulnerable users. A model trained on clean, formal text stumbles when a teenager uses different phrasing, a non-native speaker switches registers, or gaming culture slang enters the conversation.
Circuit Breaker Labs developed AI agents designed to stress-test models by mimicking diverse user cohorts—different ages, languages, cultural backgrounds, speech patterns. The agents simulate tens of thousands of conversations, looking for failure modes where an AI system misinterprets harm signals or offers dangerous responses.
Arul Nigam, the company's CTO, said the problem is that "young people often seek support from these systems but are actively being harmed." Shirali Nigam, CEO, added: "The way a six-year-old girl versus a 45-year-old man, or someone who speaks English as a first language versus a second language, or gamer slang versus someone else who uses a different kind of slang, all of those can really trip up a model."
The company collaborates with domain experts to build high-fidelity user simulations incorporating real speech patterns, coded language, and typos. Each red-team run produces data on where a model breaks—actionable intelligence for developers trying to reduce liability exposure.
The business model is straightforward: AI labs facing regulatory scrutiny and legal risk now have a reason to buy safety testing before launch. As litigation costs mount, the cost of red-teaming looks cheap.
