The issue of AI-generated low-quality content on TikTok is more serious than it appears, and children are being exposed to it the most.

      TikTok has dedicated years to mastering the ability to understand exactly what you'll want to watch next. You open the app, scroll a bit, and before long, it presents videos that seem remarkably suited to your interests. But what occurs before TikTok figures out who you are? Research by video editing platform Kapwing suggests that the initial experience often consists of low-quality AI-generated content.

      The study revealed that almost 60% of the videos shown to a newly created TikTok account were categorized as AI-generated "slop." This issue isn't confined to obscure areas of the platform; it's the first impression TikTok gives to new users before the algorithm starts customizing their content. What's even more troubling is the findings regarding content targeted at children.

      The algorithm's junk-food phase

      TikTok’s recommendation algorithm is built to adapt swiftly. It considers factors like likes, follows, watch time, and scrolling behavior when selecting videos to display next. To explore what a completely new TikTok experience entails, researchers set up a fresh account and analyzed the initial 500 videos presented on the For You page. The results were striking: out of those, 294 videos were deemed AI slop. This implies that new users are more likely to encounter AI-generated junk than authentic human-made content before TikTok gathers any meaningful data about their tastes.

      Perhaps more revealing is how TikTok stacks up against other platforms. Kapwing previously conducted a similar study on YouTube Shorts and discovered significantly less AI-generated clutter. TikTok was not only worse — it was markedly worse. Currently, AI content has transitioned from being an occasional occurrence to an integral part of the platform's standard aesthetic. For many users, particularly younger ones, AI-generated videos are becoming commonplace.

      Sesame Street meets the uncanny valley

      The report's most concerning findings pertain to content designed for children. Researchers observed that over 50% of the videos in TikTok’s Kids category were classified as AI-generated slop. One hashtag, #CartoonKids, was nearly entirely dominated by AI-produced content, with only a handful of videos seemingly created by humans. Those who have come across these videos will instantly recognize the formula — well-known cartoon characters in bizarre situations, educational lessons filled with errors, characters using eerie synthetic voices, and animations that shift in nonsensical ways.

      At first glance, the content might resemble typical children's programming, but it falls apart upon closer inspection, which is why it is concerning. Young children lack the ability to differentiate between high-quality educational material and a misleading AI-generated imitation that confidently delivers incorrect information. A counting lesson with erroneous numbers may appear absurd to an adult, but a preschooler lacks the same contextual understanding. While the internet has always had questionable content for children, the scale has shifted. Generative AI can produce endless streams of videos at a speed that no human creator can match, and TikTok's recommendation algorithm seems eager to circulate them.

      The issue transcends children's content as well. The study indicated that various educational categories, including science, health, and history, were significantly impacted by AI slop. This is particularly unfortunate because accuracy is crucial in these subjects. A poorly created comedy skit is simple to scroll past, but a history lesson filled with inaccuracies or a health video offering misleading information poses a much greater concern. It's important to note that not every creator utilizing AI produces low-quality content. Some are using AI-generated presenters and visuals to make educational topics more captivating. In the best scenarios, AI acts as an aid to the creator’s work instead of a replacement. However, the report emphasizes a growing trend across social media: the incentives often favor quantity over quality. If a creator can produce dozens of videos in the time it previously took to make one, platforms become inundated with content that is technically watchable but lacks depth.

      TikTok seems to recognize that users are becoming fatigued by this. The company has implemented controls to help users limit their exposure to AI-generated content and has invested in initiatives promoting AI literacy. Nevertheless, the research implies that these efforts may be inadequate in keeping up with the influx of such content. The irony lies in the fact that social media gained popularity due to its distinctly human qualities: creativity, personality, expertise, and connection. While AI can mimic these attributes surprisingly well, imitation does not equate to authenticity. When nearly 60% of the videos a new user encounters are AI-generated, the concern shifts from whether AI slop is present on TikTok to whether it has become a defining characteristic of the platform. For a generation of children growing up with these feeds, this answer is more important than ever.