Code generated by AI was delivered with controls but bypassed the verification.

      A penetration test conducted at a financial services company that oversees billions in client assets has uncovered a significant vulnerability in AI-generated code. Notably, the code also had elements that were effective.

      Sygnia, the incident response firm, evaluated a customer onboarding application primarily developed using Claude. This application managed sensitive information such as government-issued IDs, identity verification data, payment details, and Social Security numbers. The discovered flaw allowed one applicant to access another applicant's records.

      What went wrong

      The application addressed a genuine issue: users often start an application, step away, and then return before creating an account. Hence, the system needed to restore their progress without requiring a password. Claude devised a temporary access model for this purpose. The vulnerability arose in the model's way of determining who warranted a token.

      According to Sygnia's research, possessing an applicant's globally unique identifier (GUID) was deemed adequate proof to issue or restore an access token for that applicant.

      “Effectively, the GUID became a bearer secret,” wrote Sygnia. A GUID serves to identify a record but doesn't confirm that its holder controls the session, device, identity, or verified contact channel associated with it.

      Anyone with a different applicant’s GUID could access names, contact details, application statuses, financial information, identity verification data, payment details, Social Security numbers, and co-applicant records. Co-applicants are noteworthy as they are individuals whose data is in the system due to someone else naming them.

      The security measures were in place

      What makes this finding unusual, distinguishing it from typical AI coding failures, is that the implementation was secure. Sygnia outlines a framework that included temporary tokens, expiration, rate limiting, audit logging, session restoration, and suspicious activity detection. Each of these controls is valid, yet none addresses the core question the system needed to solve.

      Researchers posed the question: “What needs to be established before the backend issues or restores an applicant's access token?” The vulnerability was present before issuance, at the moment when the system decided if the requester was entitled to a token.

      Sygnia noted, “The problem was not that the system lacked security controls; rather, the problem was that the controls were centered around the wrong trust decision.” Even a momentary token can cause harm if it ends up with the wrong individual. While audit logs can assist in investigations afterward, they do not prevent breaches.

      Why conventional tools overlook this

      Static application security testing tools focus on identifying insecure coding patterns and unsafe data flows, but this flaw was not about these issues. It was an architectural flaw grounded in an incorrect assumption about trust, embedded in code that adhered to standard framework conventions and passed basic checks.

      Zach Mead, the Sygnia penetration tester who conducted the evaluation, stated, “Working code is not synonymous with secure code.” He cautioned that “AI-generated code might compile, adhere to familiar conventions, and pass basic checks while still harboring flawed assumptions about trust boundaries, authorization, state, ownership, or integrations with third parties.” He advised security teams to regard AI-generated outputs as untrusted until they are validated.

      An LLM identified what an LLM created

      Sygnia didn't identify this issue manually; rather, it utilized a language model to analyze sections of the client’s codebase, searching for problems related to trust boundaries and business logic flaws. The model pinpointed the applicant-token issue directly.

      Sygnia succinctly summarized this finding, noting that a security architectural flaw characterized by "vibe-coding" was unearthed through a similar code review approach. This discovery has implications in both directions; if defenders can query a model about missing trust boundaries, so can anyone in possession of compromised source code or stolen implementation notes.

      Cisco has been using open-weight models for bug hunting for this very reason; the same capability can serve two different purposes.

      The acceleration issue

      This research coexists with another Sygnia case, further reinforcing the argument being made. In a cloud breach case the firm examined, the initial indication was not a breach itself but rather the speed of activities. Credential discovery, cloud enumeration, source-code reviews, CI/CD probing, database access, and operational disruptions all happened simultaneously, suggesting multiple operators, when in fact it was one individual employing agents.

      Together, these cases illustrate a clear trend: AI is hastening not just how code is written but also how flaws are discovered and how quickly vulnerabilities can be exploited. We have already recognized four separate attacks exploiting the same underlying flaw.

      What Sygnia advises clients to do

      Rather than banning AI tools, which might push usage into private accounts where oversight is lost, the firm recommends flagging pull requests that were either generated or heavily modified by AI, specifying the tool used, and marking any work related to authentication, payments, data exports, or regulated client information.

      The most tangible advice pertains to prompts. A prompt requesting “temporary access tokens for applicants” is incomplete. A more secure prompt would