TikTok excluded approximately 15 million U.S. users from an algorithmic safety update in early 2022 as part of an A/B test designed to measure its effect on daily active users and overall platform stickiness, according to a confidential internal document obtained by Bloomberg on Aug. 4.
Among the users excluded from the safety update was 16-year-old Chase Nasca, who died by suicide in Feb. 2022. Because he was part of the control group, TikTok's strategies for disrupting repetitive content recommendations did not take effect on his account by design, according to the internal report.
An analysis of 7,563 videos served to Nasca in his final two weeks showed 73 percent contained content related to mental health struggles, suicide awareness, loneliness, relationships, or sadness. TikTok did not review his viewing history immediately following his death. The investigation into his account began in March 2023, after Bloomberg initiated an inquiry into how the platform affected young users.
TikTok had recognized as early as 2021 that its algorithm could expose users to excessive amounts of troubling material. At that time, the company announced it was testing methods to interrupt repetitive recommendations, noting that a series of similar videos could create a harmful viewing experience even if individual videos complied with platform rules.
The company's solution introduced the updated recommendation system to 90 percent of its U.S. users by early 2022, leaving 10 percent on the older algorithm. This allowed TikTok to conduct a backtest comparing outcomes between the two groups.
TikTok told Bloomberg that the experiment aimed to confirm the effectiveness of its safety changes. The company said it has since refined its testing practices but did not specify what those refinements entail.
The tension between optimizing for engagement metrics and protecting user safety underpins the economics of social media platforms. Daily active users and stickiness directly drive advertising revenue. For TikTok, the tradeoff was quantifiable: the company needed proof that the safety feature would not cannibalize growth.
