Google Earth's AI misstep was short-lived, enduring just a single day. It was a story marked by perilous naivety.

      Technology giants seem to be out of touch or perhaps moving too quickly in their deployment of AI. Google has integrated its Gemini AI across its entire software suite, from Android to widely used productivity tools like Gmail. AI is pervasive, and while not all implementations are negative, some come across as contrived.

      In its latest experiment with AI, Google focused on Google Earth, allowing users to utilize its Nano Banana 2 AI image generator to create images that could be placed on the map. On the surface, it seems like an intriguing concept. For instance, imagining what the Colosseum might look like in your neighborhood could be fun.

      However, it raises questions when such images might include a nuclear reactor in an adversarial country or a tank in a conflict zone. Unfortunately, this is exactly what Google Earth produced without restraint.

      So, what’s the concern?

      Let’s first revisit the initial intentions behind this feature. Google aimed to offer a new capability:

      “For the first time, you can generate custom images using Google Earth’s satellite, aerial, and 3D imagery in conjunction with Nano Banana, which creates concepts grounded in the real world. Just zoom into a location in Google Earth on the web, tap ‘create image,’ and enter whatever you want to visualize.”

      The issue is that AI isn’t always a suitable addition.

      In fact, Google’s AI has faced misuse by various malicious actors in the recent past. A report submitted to Australia’s eSafety Commission highlighted the exploitation of Gemini AI to create deepfake content related to terrorism and child exploitation. Earlier this year, Google’s Cloud unit revealed instances of Gemini being used by cybercriminals for malware and surveillance activities.

      Just recently, Google filed a lawsuit against a network linked to China for utilizing Gemini to conduct large-scale phishing operations. Even a regular user can easily manipulate Google’s AI image generator into creating something disturbing, like a Nazi-themed Spongebob. The situation with OpenAI’s ChatGPT image model is similar.

      Google ventured too close to risky ground with its latest AI feature in Google Earth, which lasted only a day. Henk van Ess, an expert in online research, tested the new AI capability and quickly critiqued it in his newsletter, Digital Digging:

      "Tonight I typed just one sentence into Google Earth and placed refugees near the Mexican border. Then I put a nuclear plant in Iran. Then I created a fatal crash scene on a street in Amsterdam. All with Google’s own satellite imagery beneath. What is Google doing?"

      There are many facets to this issue.

      This was not the end of the complications. The Nano Banana engine simply complied with controversial image generation requests without any resistance. "Nothing was rejected, nothing was moderated, and there was no suggestion to try a different prompt," van Ess noted.

      Just one day after its launch, Google decided to retract the Nano Banana image generation feature after users created prompts that resulted in depictions of a plane hitting a Manhattan skyscraper, a bombing in Moscow, a nuclear reactor in Iran, and refugees at the US–Mexico border.

      Let’s give Google some leeway here; AI image generators typically aim to avoid inappropriate content like sexual imagery, weaponry, and other harmful requests. It struggles with seemingly benign prompts that can trigger significant international tensions. Situating a nuclear reactor in Iran or a bomb crater in a Gaza hospital reflects such "innocent failures," if I can put it that way.

      But Google’s AI oversight didn’t stop there. The AI-generated images produced by Google Earth passed through checkers without triggering any alarms. The BBC also tested the tool, which generated controversial scenarios such as Russian tanks on Ukrainian soil, the Eiffel Tower collapsed, and the great pyramids of Egypt engulfed by a sinkhole.

      Alarmingly, research indicated that it was possible to deceive Google’s Gemini into endorsing these fake images as authentic. Third-party AI checker tools similarly failed in numerous cases. To clarify, all AI-generated images from Gemini (and Nano Banana) are imbued with an invisible SynthID watermark (developed by Google DeepMind) and a visible watermark as well.

      This isn’t even my primary concern. The internet is filled with strikingly realistic AI-generated content, with many users unaware that it is fabricated. Even if they suspect something is off, they often have no way to validate its authenticity or identify its AI origins. In many instances, users lack the motivation to fact-check questionable images.

      "Journalists might verify that image… and report on it. But not everyone will do that," AI researcher Henry Ajder shared with the BBC. After speaking to three fact-checkers from the Poynter-backed International Fact-Checking Network (IFCN), they highlighted a significant digital literacy divide. Often, individuals simply lack a reason to investigate a synthetic image's true source and flag it publicly.

      This was indeed a misstep.

      Ultimately, Google’s attempt to let users express their creativity through this AI feature revealed a shocking lack of awareness