Meta's AI detector overlooks 50% of its own cropped forgeries.

      The Meta AI detector claims to identify Meta’s own fakes, but simply cropping an image allows over half of them to bypass detection. This tool was intended to address the deepfake issue, not demonstrate its limitations. Recently, Meta showcased an image detector alongside Muse Image, its most advanced image generator, assuring that the tool would recognize anything produced by the model, even after modifications.

      However, a test conducted by Reuters revealed a different story. After generating 40 images with Muse Image, cropping them, and resubmitting them, the detector failed to identify more than half of the modified images.

      The issue with cropping is evident in the statistics. Reuters discovered that the tool authenticated every one of the original 40 AI-generated images. Yet, after cropping those same images to about a third or half of their original size, it failed to flag 55% of them. A simple crop, a common practice before sharing online, was sufficient to eliminate the key signal the detector relies on.

      The heart of the EU tech scene has seen the latest updates, including a story from our founder Boris and some questionable AI art. It's free and delivered weekly to your inbox; sign up today!

      The significant signal in question is a watermark. Meta refers to this as Content Seal, an invisible marker integrated into every image produced by Muse Image. On its website, Meta claims that the Meta AI detector can identify its images even after cropping, but the Reuters analysis indicates that this promise only holds to a certain extent.

      In response to the findings, Meta explained that the detector is still in preview mode. The company stated that the watermark is designed to endure common edits but noted that “the signal may be lost if an image is heavily cropped.” This illustrates the tension in one concise sentence: while the mark is intended to be durable, even the simplest online edit can remove it.

      Meta is not the only company facing this challenge. Google and OpenAI have acknowledged that their detection tools are not foolproof against image alterations. Watermarking has become the preferred solution for synthetic media, with each major lab developing its own variant.

      Google's alternative, SynthID, recently disproved a well-known deepfake, showcasing the potential of the technology. However, Meta’s recent misstep raises concerns about relying solely on it.

      The limitations of a watermark have been recognized by researchers for some time. Siwei Lyu, a computer science professor at the University at Buffalo who studies image forensics, noted that watermarking methods are effective as long as the mark remains intact. The complication arises with subsequent modifications. “Any alteration that removes or weakens the embedded signal, such as cropping, resizing, heavy compression, or editing, may diminish their effectiveness,” he explained to Reuters.

      Some experts argue that perfection should not be the expectation. Sarah Barrington, an AI researcher at UC Berkeley, compared watermarking to security systems that successfully intercept most threats without stopping every one of them. “Even if we catch only 90%, that’s still a significant improvement from 0,” she said. Both perspectives can coexist.

      A detector that fails to recognize 55% of lightly modified images falls far below 90% effectiveness and contributes to a growing demand for AI detection that fails to assure reliability.

      The timing of these shortcomings is significant. The United States is approaching a midterm election year, and platforms are preparing for an influx of AI-generated fakes targeting voters. Governments, including South Korea, are also enacting strict laws against misleading content.

      In March, Meta’s Oversight Board urged the company to enhance its measures against deceptive AI and to invest in more robust detection methods. Four months later, the primary detector remains unable to reliably detect Meta’s own outputs when cropped.

      Despite these issues, Content Seal is not rendered useless. A tool that identifies fresh, unedited images still increases the difficulty of disseminating fakes, and Meta has plans to extend this system to video. However, it undermines the notion that a watermark is a complete solution rather than just a hurdle.

      Those most likely to alter a signal are precisely the individuals the detector is meant to thwart. In synthetic media, as in educational settings, detection technology often lags behind, and current evidence suggests that catching up can be as simple as cropping.