Cloudflare and OpenAI are collaborating on a pilot project to enhance the freshness of AI search results.
Cloudflare has dedicated the past year to fortifying websites against AI crawlers, and now it aims to assist one in crawling more effectively. On Wednesday, the company announced it is collaborating with OpenAI on a research pilot project to determine if its network data can enhance the accuracy of AI search results.
Cloudflare monitors over 20% of the internet, which enables it to observe real-time changes in web pages and user traffic patterns. This pilot will provide OpenAI's search and answer system with key signals, including content freshness and traffic quality.
The importance of freshness
The reliability of AI responses depends heavily on the pages they are drawn from. If an index is outdated, chatbots may return outdated or incorrect information. OpenAI seeks to leverage Cloudflare’s live web insights to ensure that ChatGPT identifies new content more quickly.
In addition, OpenAI has established content agreements to enrich ChatGPT. However, this pilot aims to address a distinct challenge: content discovery rather than licensing.
“Having current information is crucial for providing accurate responses to ChatGPT users,” stated Nick Ryder, OpenAI’s VP of research. He described the test as a method for more efficient content discovery.
A significant shift
This initiative appears to contrast with Cloudflare's recent actions. Last year, the company allowed websites to prevent AI crawlers from accessing their content unless AI companies compensated publishers. It also developed a browser protocol to distinguish between humans and bots, promoting the principle: your content, your rules.
This pilot, however, takes a different approach. Instead of restricting access to content, Cloudflare is now facilitating AI engines’ access to it. CEO Matthew Prince framed this as a matter of efficiency, aiming to “enhance AI search efficiency and help users obtain quality answers more rapidly.”
Why it matters
The search landscape is transitioning from link-based to answer-oriented results. Publishers are already concerned about losing traffic to chatbots that summarize their content—a concern that has led regulators to enforce opt-out measures. Cloudflare now occupies a unique position, operating on both sides of the issue. It provides tools to block crawlers while also exploring the mechanisms to support them. The outcome of this pilot could influence who has access to the open web and the conditions under which that access is granted.
Other articles
NATO selects Accenture and Leonardo for a €200M secure cloud project.
NATO has finalized a seven-year contract worth approximately €200 million with Accenture and Leonardo to establish a secure cloud infrastructure, known as the Protected Business Network, for the Alliance.
Perplexity is discreetly developing a coding tool powered by AI.
Perplexity has developed an internal coding tool referred to as "Teammate," which may compete with Cursor and Claude Code, according to Business Insider. The launch has not yet been confirmed.
Gartner's research indicates that customers favor ChatGPT over company chatbots.
According to Gartner, customers are three times more inclined to utilize third-party GenAI services like ChatGPT for support compared to company chatbots, as the returns on AI investments are falling short of expectations.
ZML's complimentary AI server operates on any leading chip.
France's ZML has launched ZML/LLMD, a free inference server that operates open-source AI at high speed on Nvidia, AMD, Google, Intel, and Apple processors.
Mark Cuban: Lovable and Replit have the potential to outlast AI labs.
During the RAISE Summit, Mark Cuban expressed that vibe-coding tools such as Lovable and Replit can endure in the market against Anthropic and OpenAI by focusing on controlling the workflow rather than merely the code.
ZML's complimentary AI server operates on all major chipsets.
France's ZML has launched ZML/LLMD, a free inference server that operates open-source AI at high speed on Nvidia, AMD, Google, Intel, and Apple processors.
Cloudflare and OpenAI are collaborating on a pilot project to enhance the freshness of AI search results.
Cloudflare is collaborating with OpenAI to integrate real-time network signals into ChatGPT's search, marking a shift for a company that was established to prevent AI crawlers.