Cloudflare Precursor monitors entire visits to identify bots.

      For the first time, bots account for more than half of all internet traffic. Cloudflare Precursor, the company’s latest tool, shifts its focus from verifying identities at the entrance to monitoring visitor behavior once they are inside.

      The internet has recently reached an unusual milestone, with bots generating more web requests than actual users. According to Cloudflare, automated traffic constitutes roughly 57% of all activity on the web.

      This change sets the stage for a product that Cloudflare unveiled on Monday. Named Cloudflare Precursor, this tool alters the way the internet differentiates between humans and machines.

      Monitoring the entire visit, not just the entrance

      Traditional defenses function like a bouncer who checks one ID at the entrance. A CAPTCHA prompts users to confirm they are human once, allowing them to proceed afterward. Modern bots have become adept at simulating that single validation moment.

      Precursor adopts a different approach. It runs within the browser, tracking an entire session. This encompasses mouse movements, scrolling patterns, typing rhythm, clipboard activity, and the duration a page remains visible. While faking a single click may be easy, mimicking an entire human visit poses a significant engineering challenge.

      “Conventional security measures only evaluate a single instance, but modern bots have developed the capability to deceive enough to gain entry,” stated Dane Knecht, Cloudflare’s chief technology officer. He noted that the area between login and checkout remains unclear, and Precursor aims to illuminate it.

      Cloudflare asserts that the tool prioritizes privacy. It captures behavioral patterns rather than specific content, measuring typing as rhythm and cadence without recording actual keystrokes. It activates with a single click and requires no coding modifications.

      Categorizing machines based on their intentions

      Precursor is part of a broader reassessment that also includes a focus on beneficial bots.

      Not all automation poses a threat. Cloudflare now categorizes AI traffic into three groups: search bots that index pages for future inquiries, agent bots that act in real-time on behalf of a user, and training bots that assimilate content for modeling.

      Starting from September 15, new websites using Cloudflare will block Training and Agent bots by default on ad-supported pages while allowing Search bots to pass through. The reasoning is financial; search bots drive traffic back to the site, while the others typically do not. This represents another evolution in Cloudflare's approach, which has already mandated that AI crawlers either pay publishers or face restrictions.

      Additionally, the company is implementing a way for sites to dictate how bots can use their content: they can choose to allow storing nothing, indexing and linking back, or summarizing and reproducing. A new database called BotBase will catalog all known crawlers. This initiative builds on Cloudflare’s prior efforts to establish a privacy-centered anti-bot standard in collaboration with major browsers.

      Trust that you can both carry and lose

      The most challenging aspect is that the bot accessing your site may not originate from the company that created it. Cloudflare encourages operators to self-identify using an existing web header so that a site can permit “OpenAI” and maintain that distinction even through various intermediaries.

      The loss of trusted status across over 20% of web domains managed by Cloudflare, according to the company, acts as a significant deterrent. This concept is a gentler version of initiatives like Estonia’s proposal to assign ID numbers to every AI agent. The stakes increase as agents start to engage in shopping and financial transactions on our behalf. It also resonates with the movement allowing publishers to opt out of AI without disappearing from search results.

      The infrastructure is evolving as well

      The adjustments extend beyond a single provider. On the same day, the Internet Engineering Task Force announced a new HTTP method called QUERY, as reported by the Register. This addition endows complex searches with their own verb, making them safe and cacheable, rather than requiring them to disguise themselves as data-altering requests.

      Cloudflare and Akamai engineers collaborated on this standard. This theme of adaptation is evident across the board; the foundational structures of the web were originally designed for human interactions, but are now being subtly restructured to accommodate a landscape where the majority of visitors are machines.