Google's AI Overviews are identifying fan-fiction creatures as real.

      Google's AI struggles with distinguishing horror tales from reality. When queried about specific online creatures, it often describes them as if they were factual.

      As reported by Futurism, Google's AI Overviews frequently present information from the “SCP Foundation” as if it were true, despite the fact that the SCP Foundation is a large collaborative fan-fiction project, explicitly labeled as fictional on its website.

      Consider SCP-565, known as “Ed’s Head.” When searched on Google, the AI Overview described it as an “ambulatory human head” that moves along the seabed like a crab, complete with a deceased person's name and “official” records. The AI did not clarify that this is purely fictional.

      Another peculiar case involves SCP-426, a fictional toaster that causes individuals to refer to it in the first person. Consequently, the AI Overview responded as though it were the toaster, recounting fabricated instances of harm as if they were real. Overall, Futurism identified at least 20 such examples.

      This issue is not new. AI Overviews have previously provided misleading information, ranging from bizarre recipes to fictitious phrases. An analysis estimated their accuracy at approximately 91 percent, a seemingly high figure, but considering Google's handling of trillions of queries, that 9 percent translates into millions of incorrect responses.

      The implications are significant. Most individuals searching for SCP codes understand they are fictional; however, the concern lies with others. A child who views a frightening video or an adult uncertain about what is real may simply accept the AI’s assertions. This concern grows as Google shifts its Search to an AI-centered approach that offers answers instead of links.

      This transition is already putting pressure on the open web, with AI Overviews contributing to a notable decrease in clicks to the websites they summarize. The possibility of a tool presenting fan-fiction as fact at the top of search results exacerbates the situation. Google did not respond to Futurism's inquiry. However, when Digital Trends rechecked the searches, some now correctly identified the entities as “fictional,” indicating that Google may have remedied part of the issue.