Cloudflare and OpenAI are collaborating on a pilot project to enhance the freshness of AI search results.
Cloudflare has dedicated the past year to fortifying websites against AI crawlers, and now it aims to assist one in crawling more effectively. On Wednesday, the company announced it is collaborating with OpenAI on a research pilot project to determine if its network data can enhance the accuracy of AI search results.
Cloudflare monitors over 20% of the internet, which enables it to observe real-time changes in web pages and user traffic patterns. This pilot will provide OpenAI's search and answer system with key signals, including content freshness and traffic quality.
The importance of freshness
The reliability of AI responses depends heavily on the pages they are drawn from. If an index is outdated, chatbots may return outdated or incorrect information. OpenAI seeks to leverage Cloudflare’s live web insights to ensure that ChatGPT identifies new content more quickly.
In addition, OpenAI has established content agreements to enrich ChatGPT. However, this pilot aims to address a distinct challenge: content discovery rather than licensing.
“Having current information is crucial for providing accurate responses to ChatGPT users,” stated Nick Ryder, OpenAI’s VP of research. He described the test as a method for more efficient content discovery.
A significant shift
This initiative appears to contrast with Cloudflare's recent actions. Last year, the company allowed websites to prevent AI crawlers from accessing their content unless AI companies compensated publishers. It also developed a browser protocol to distinguish between humans and bots, promoting the principle: your content, your rules.
This pilot, however, takes a different approach. Instead of restricting access to content, Cloudflare is now facilitating AI engines’ access to it. CEO Matthew Prince framed this as a matter of efficiency, aiming to “enhance AI search efficiency and help users obtain quality answers more rapidly.”
Why it matters
The search landscape is transitioning from link-based to answer-oriented results. Publishers are already concerned about losing traffic to chatbots that summarize their content—a concern that has led regulators to enforce opt-out measures. Cloudflare now occupies a unique position, operating on both sides of the issue. It provides tools to block crawlers while also exploring the mechanisms to support them. The outcome of this pilot could influence who has access to the open web and the conditions under which that access is granted.
Other articles
Gartner has found that customers favor ChatGPT over company chatbots.
According to Gartner, customers are three times more likely to utilize third-party generative AI such as ChatGPT for support compared to company chatbots, as the returns on AI investments are slower than the expenditures.
Mark Cuban: Lovable and Replit have the potential to endure beyond the AI labs.
During the RAISE Summit, Mark Cuban contended that vibe-coding tools such as Lovable and Replit can endure against Anthropic and OpenAI by taking control of the workflow rather than merely the code.
The GhostApproval bug disrupts six leading AI coding agents.
Wiz's 'GhostApproval' employs an outdated symlink technique to enable six AI coding agents, ranging from Amazon Q to Cursor, to operate beyond the sandbox and transfer ownership of the box.
CEO of ElevenLabs discusses $600 million in revenue and the AI laboratories.
During the RAISE Summit in Paris, ElevenLabs CEO Mati Staniszewski spoke about achieving a reported $600 million in revenue and outlined the startup's strategies to surpass the AI labs.
NATO engages Accenture and Leonardo for a secure cloud project valued at €200 million.
NATO has finalized a seven-year agreement worth approximately €200 million with Accenture and Leonardo to create a secure cloud infrastructure, known as the Protected Business Network, for the Alliance.
Cloudflare and OpenAI are collaborating on a pilot project to enhance the freshness of AI search results.
Cloudflare is collaborating with OpenAI to integrate real-time network signals into ChatGPT's search, marking a shift for a company that was established to prevent AI crawlers.