Meta's AI detector fails to identify half of its own altered images.

      The Meta AI detector claims to identify Meta's own fakes, but when images are cropped, over half of them evade detection. This tool was intended to address the deepfake issue, rather than being an illustration of it. Recently, Meta showcased an image detector alongside Muse Image, its most advanced image generator, asserting that it could recognize anything generated by the model, even after alterations.

      However, Reuters conducted a test and discovered that after generating 40 images with Muse Image, cropping them, and reintroducing them, the detector failed to identify more than half of them.

      The simplicity of cropping exposed its flaws. The data tells the tale; Reuters reported that the tool approved all 40 original AI images. When those images were cropped to about one-third or one-half of their size, the detector failed to flag 55% of them. A simple crop, a common action before sharing images, was enough to eliminate the signal that the detector relies on.

      The signal is a watermark known as Content Seal, an invisible marker integrated into every image produced by Muse Image. On its own website, Meta asserts that the Meta AI detector can recognize its images even after cropping, but the Reuters analysis shows this claim only holds true to a certain extent.

      When asked about these results, Meta noted that the detector is still in preview mode. They stated that the watermark is designed to endure common edits, but that signal “may be lost if an image is heavily cropped.” This encapsulates the core contradiction: the mark is intended to be durable, yet a typical internet edit can easily erase it.

      Meta is not the only one facing this challenge. Both Google and OpenAI have cautioned that their detection tools are not foolproof against alterations made to images. Watermarking is the preferred solution in the industry for synthetic media, and many significant labs are adopting some variant of it.

      Google's SynthID recently exposed a prominent deepfake, supporting the efficacy of the technology, while Meta's failure raises concerns about relying solely on it.

      Researchers have highlighted this vulnerability for some time. Siwei Lyu, a computer science professor at the University at Buffalo who specializes in image forensics, stated that watermark methods perform well only if the mark remains intact. The issue arises with what follows. He explained to Reuters, “Any modification that removes or weakens the embedded signal, such as cropping, resizing, heavy compression, or editing, may reduce their effectiveness.”

      Some argue that the standard shouldn’t be perfection. Sarah Barrington, an AI researcher at UC Berkeley, compared watermarking to security measures that succeed in thwarting most threats without eliminating all of them. “Even if we catch only 90%, that’s still a significant improvement from 0,” she remarked. Both viewpoints can coexist.

      A detector that overlooks 55% of slightly edited images is far from achieving the 90% mark and fuels a burgeoning market for AI detection that still lacks reliability.

      The timing of this issue is critical. The United States is approaching a midterm election year, and platforms are preparing for a surge of AI-generated fakes aimed at voters. Governments, including South Korea, are also drafting punitive legislation against misleading content.

      In March, Meta’s own Oversight Board urged the company to take stronger action against deceptive AI and to enhance detection efforts. Four months later, the primary detector struggles to reliably detect Meta’s own outputs once cropping occurs.

      Nonetheless, this does not render Content Seal ineffective. A tool that labels new, unaltered images still increases the difficulty of presenting a fake, and Meta has plans to expand this system to video content. However, it challenges the notion that a watermark serves as a definitive solution rather than merely a hurdle.

      Those most likely to remove a signal are precisely the individuals the detector aims to thwart. In the realm of synthetic media, much like in educational settings, detection often lags behind advancements. Current evidence suggests that catching up may require nothing more than a simple crop.