Code generated by AI was delivered with controls but bypassed the verification process.

      A recent penetration test conducted at a financial services firm, which manages billions in client assets, has uncovered a significant flaw in AI-generated code. What’s noteworthy is that the code successfully addressed certain issues. Sygnia, the incident response firm, examined a customer onboarding application largely developed with Claude. This application processed government-issued IDs, identity verification data, payment details, and Social Security numbers. The security flaw allowed one applicant to access another applicant's records.

      What actually malfunctioned

      The application tackled a legitimate issue where individuals start an application, leave, and later return before fully creating an account. Therefore, the system needed to restore their progress without requiring a password. Claude devised a temporary access model to facilitate this. The flaw originated in how the model determined eligibility for an access token. According to Sygnia, possession of an applicant’s globally unique identifier (GUID) was deemed adequate proof to issue or restore access tokens for that applicant.

      “As a result, the GUID effectively became a bearer secret,” Sygnia noted in its findings. A GUID identifies a record but does not confirm that the person holding it possesses control over the session, device, identity, or the verified contact method associated with it. Consequently, anyone with another applicant’s GUID could access names, contact information, application statuses, financial data, identity verification information, payment details, Social Security numbers, and records of co-applicants—individuals whose information was stored in the system because someone included their names.

      The controls were in place

      What makes this discovery particularly unusual is that it diverges from the typical narrative surrounding AI coding failures. The implementation was not lacking in security measures. Sygnia described a comprehensive strategy that included temporary tokens, expiration protocols, rate limiting, audit logs, session restoration, and suspicious activity detection. Each of these measures is credible, yet none addresses the key question the system needed to resolve: “What must be proven prior to the backend issuing or restoring an access token for an applicant?”

      The vulnerability was present before token issuance—specifically when the system decided if the requester was eligible for a token. “The problem was not a lack of security controls,” Sygnia wrote. “The issue lay in the fact that the controls were designed around the wrong decision of trust.” A short-lived access token can still cause harm if it goes to the incorrect user. Although audit logs can assist in investigations afterward, they provide no preventive measures.

      Why scanners overlook this

      Static application security testing tools focus on identifying insecure coding practices and unsafe data flows, and this flaw did not fall into those categories. The issue was structural, stemming from an incorrect assumption about trust, expressed in code that adhered to well-known framework standards and passed basic evaluations.

      “Working code does not equate to secure code,” stated Zach Mead, the Sygnia penetration tester who conducted the assessment. “AI-generated code can compile correctly, maintain familiar conventions, and pass fundamental checks, while still operating on misguided assumptions regarding trust boundaries, authorization, state, ownership, or third-party integrations.” His advice to security teams is straightforward: consider AI-generated results as untrustworthy until verified.

      An LLM found what an LLM created

      This oversight was not manually caught by Sygnia; instead, the firm employed a language model to analyze sections of the client’s codebase in search of trust-boundary issues and business-logic flaws. The model directly identified the applicant-token problem. Sygnia summarized this point sharply, revealing that a flaw in security architecture was uncovered through a review of vibe-coded code.

      This observation has implications for both sides; if a defender can utilize a model to pinpoint missing trust boundaries, so too can anyone with access to leaked source code or stolen implementation notes. Cisco has been applying open-weight models for bug detection for a similar rationale, showcasing one capability with dual applications.

      The tempo problem

      This research parallels another Sygnia case, reinforcing the argument. In a cloud breach investigated by the firm, the initial indicator was not a specific exploit; rather, it was the rapid pace of operations. Credential discovery, cloud enumeration, source-code review, CI/CD probing, database access, and operational disruptions occurred simultaneously, resembling activities by multiple operators rather than a single individual. However, it was one person working with external agents.

      When these cases are analyzed together, the pattern is evident. AI is expediting the process of code generation, flaw identification, and the speed at which a vulnerability can evolve into a target. We have tracked four distinct attacks stemming from a single underlying flaw.

      What Sygnia advises clients to do

      The recommendation is not to prohibit the tools. Such bans only drive usage to personal accounts, which remain unseen. Instead, the firm suggests that pull requests generated or significantly altered by AI should be marked, specifying the tool used and flagging any work related to authentication, payments, data exports, or regulated client information.

      The most precise guidance pertains to prompts. Simply requesting “