Google's AI Overviews are declaring fan-fiction creatures as real.

      Google’s AI is struggling to differentiate between horror stories and real-life accounts. When queried about specific online monsters, it often presents them as if they were actual facts.

      A report from Futurism states that Google’s AI Overviews frequently describe entries from the "SCP Foundation" as truth. The SCP Foundation is an expansive collaborative fan-fiction endeavor where its “anomalies” are purely fictional, and its website makes this clear.

      For instance, SCP-565, known as “Ed’s Head”, is depicted in the AI Overview as an “ambulatory human head” that scuttles on the ocean floor like a crab, accompanied by a fictitious name and “official” documentation—without ever indicating that this is not real.

      Another peculiar case is SCP-426, a fictional toaster that causes people to refer to it in the first person. Consequently, the AI Overview responded as if it were the toaster and narrated fabricated accounts of harm as though they were real. Futurism identified at least 20 similar instances.

      This reflects a known vulnerability. The entries from the SCP Foundation are crafted to imitate dry research documents, which is part of the humor. Still, the website includes a clear fiction disclaimer, which the AI largely overlooked. Occasionally, it alluded to “lore” without clarifying its meaning.

      This issue isn’t new; AI Overviews have confidently produced inaccuracies before, including incorrect recipes and invented expressions. An analysis suggested their accuracy is approximately 91 percent, sounding impressive until one considers that Google processes trillions of queries, which means that 9 percent amounts to millions of incorrect responses.

      The significance of this issue is considerable. Most individuals searching for SCP codes are likely aware that they are fictional. However, the concern lies with those who are not, such as a child who views a frightening video or an adult uncertain about what is real, who may take the AI’s claims at face value. The situation is becoming more critical as Google shifts Search to an AI-centric interface focused on providing answers rather than linking to sources.

      This transition is already causing a decline in clicks to the websites that the AI summarizes. Moreover, it raises the possibility of fan fiction being presented at the top of the results as factual information. Google did not provide a comment to Futurism. One note: when Digital Trends re-evaluated the searches, some now correctly identified these entities as “fictional,” suggesting that Google may have quietly corrected part of the issue.