Google has filed a lawsuit against a Chinese cybercrime group that utilized Gemini to create phishing websites and send out 2.5 million fraudulent messages.

      **TL;DR** Google is suing a Chinese cybercrime group named Outsider Enterprise, which utilized Gemini AI to create phishing sites and sent 2.5 million scam texts within two weeks. The FBI is involved in the situation.

      On Friday, Google initiated legal action to dismantle the operations of a Chinese cybercrime group known as Outsider Enterprise. This group employed AI, including Google’s own Gemini, to produce phishing websites and disseminate fraudulent text messages that pretended to be from Google and other brands. They distributed 2.5 million scam texts to Android users over a span of two weeks.

      The operation used 9,000 fake websites and 1 million counterfeit web domains aimed at stealing passwords and credit card information. Google reported that the group has financially defrauded "hundreds of thousands of victims," with losses "estimated in the millions." In just two weeks in May, Android users flagged 55,000 spam texts, averaging more than two complaints per minute.

      A particularly concerning detail in the court documents reveals that Outsider Enterprise members urged one another to utilize Gemini for creating customized code for phishing websites, which was then incorporated into the group’s software and transformed into active scam sites. The group exploited Google’s AI to develop tools targeting Google users.

      They communicated through Telegram and circulated "phishing kits" that empowered lesser criminals to initiate fake text campaigns imitating trusted brands. Google claims it employs "AI-powered tools to combat AI-driven scams," intercepting over 10 billion scam messages monthly through its detection systems.

      Google stated it is collaborating with the FBI, which is undertaking unspecified law enforcement actions. The company is also partnering with AT&T, T-Mobile, and Verizon to prevent the scam texts from reaching users. The FBI did not respond to a request for comment.

      The lawsuit is civil, not criminal, implying that Google aims to eliminate the infrastructure rather than pursue imprisonment of individuals. This type of action is increasingly adopted by Big Tech companies to combat cybercrime operations when law enforcement is slow or unresponsive, especially when the offenders operate in areas beyond the reach of Western authorities.

      The breadth of the operation highlights how inexpensive AI-driven scams have become. Creating thousands of believable phishing pages previously necessitated skilled developers. Now, a Telegram group can leverage a cutting-edge AI model to generate code and deploy it at scale. AI is lowering the cost of attack tools across all categories, from vulnerability discovery to social engineering. Outsider Enterprise exemplifies this distribution model: large-scale fraud accomplished using readily available AI.

      Google indicated that it will persist in investing in AI-driven scam detection and encouraged users to activate its spam protection features on Android. However, the troubling reality is that the same company providing the AI is now suing those who exploited it for criminal purposes. The security of AI agents is not simply a matter of product features. Given the potency and accessibility of these tools, the conflict between creators and malicious users is likely everlasting.