Documents reveal that TikTok withheld a safety fix from millions of users.

      In 2021, TikTok modified its algorithm to prevent users from being inundated with harmful content. However, the safer version was not made available to all users; about 10% of U.S. users, which amounted to roughly 15 million people at that time, remained on the previous version. The company aimed to assess whether the safer feed was less addictive, according to a confidential document disclosed by Bloomberg.

      The document indicates that the control group included Chase Nasca, a 16-year-old from New York, whose account was part of the test. It states that “TikTok’s filter bubble prevention strategies did not take effect on this user by design.” As a result, his feed exposed him to a continuous stream of content related to suicide, self-harm, and depression. Tragically, he died by suicide in February 2022.

      Details from the document reveal that the experiment was an A/B test, also known as a “backtest,” a typical method used to compare two groups to evaluate the impact of a change. Bloomberg reports that the safety system itself was the aspect being tested, with TikTok analyzing whether the safer feed diminished "stickiness," a metric of interest in Wall Street analysis. The test was conducted over a period of up to a year.

      The 14-page document was created in March 2023 and is under court seal, as reported by Bloomberg. It was compiled only after Businessweek inquired about safety measures for teens. The review examined 7,563 videos that Chase viewed in his last two weeks and found that 73% contained themes of sadness or mental health issues that did not violate TikTok's guidelines. Almost 10% of the content did breach those guidelines.

      The document made two recommendations: it suggested that the control group should be reduced to under 1%, and that the test duration should be limited to seven days rather than extending up to a year. Nonetheless, it pointed out that a 1% group would still result in “at least 1.8 million US users” lacking the update. TikTok stated that the 2022 test was a standard procedure and that it has since improved its testing methods.

      The phrase “by design” has significant implications in legal contexts. Numerous lawsuits claim that social networking platforms are inherently dangerous, a contention that companies like TikTok reject. In March, a jury in California found both Google’s YouTube and Meta’s Instagram negligent for employing addictive features targeting children, resulting in a $6 million award to one plaintiff. The Chase document emerged during this broader legal battle.

      In a statement to Bloomberg, TikTok expressed its sorrow for any family that has suffered a tragic loss. A spokesperson affirmed that the company is “deeply committed to the safety and well-being of users,” particularly teenagers, and invests significantly in detection and enforcement. TikTok and Snap reached settlements right before the California trial, with TikTok finalizing three additional settlements on Monday.

      Two weeks following the internal review, TikTok CEO Shou Chew testified before Congress. He emphasized that “Safety and wellness, particularly for teens, is a core priority for TikTok.” One congressman, Gus Bilirakis, pointed to Chase’s parents in the audience, stating, “your company destroyed their lives.” Another, Gary Palmer, remarked that every time Chew mentioned “transparency,” it seemed to convey “deception.”

      In written responses dated May 2023, Congress asked TikTok how many user suicides it was aware of. The company's reply claimed it “does not believe that use of the platform has caused users to commit suicide.” Bloomberg notes that this response was given two months after TikTok had reviewed Chase’s account and did not reference the backtest.

      In the broader context, a former leader of TikTok's trust and safety team, who was present during the test, informed Bloomberg that he was unaware of it. He acknowledged that while A/B tests are standard in the industry, this specific test was flawed due to the size of the control group, which should have excluded minors and certain types of content, such as suicide videos. TikTok did not address these concerns directly.

      Chase’s parents filed a wrongful death lawsuit against TikTok in 2023. A judge dismissed the case last year, concluding that Section 230 provided protection for the company. An appeal is currently underway. This case is part of a growing examination of the responsibilities surrounding young users and social media, with courts evaluating potential liabilities and governments considering restrictions on access for minors.

      This article discusses suicide. If you or someone you know is in crisis, support is available. You can find a global list of helplines at befrienders.org, and in the U.S., you can call or text 988.