Cloudflare and OpenAI are collaborating on a project to enhance the freshness of AI search results.
Cloudflare has spent the past year equipping websites to defend against AI crawlers. Now, the company aims to assist one in improving its crawling capabilities. On Wednesday, it announced a research pilot in collaboration with OpenAI to explore whether its network data can enhance the accuracy of AI search results.
Cloudflare covers more than 20% of the internet, providing it with insights into real-time changes to web pages and user traffic patterns. This pilot will relay signals, such as content freshness and traffic quality, to OpenAI's search and response system.
The significance of freshness
The quality of AI responses depends on the underlying pages. When an index is outdated, a chatbot can disseminate outdated or inaccurate information. OpenAI seeks to utilize Cloudflare's live web insights to help ChatGPT uncover new content more promptly.
In addition, OpenAI has established content partnerships to enrich ChatGPT. However, this pilot addresses a different challenge—it focuses on discovery rather than licensing.
"Current information is vital for providing accurate answers to ChatGPT users," stated Nick Ryder, OpenAI’s Vice President of Research. He characterized the trial as a means to streamline content discovery.
A noteworthy shift
This initiative seems unusual considering Cloudflare’s recent actions. Last year, it allowed websites to block AI crawlers unless the AI companies compensated publishers, and it developed a browser protocol to distinguish between humans and bots. Its message was clear: your content, your rules.
This pilot represents a different approach. Instead of restricting access to content, Cloudflare is now facilitating AI systems to access it. CEO Matthew Prince described it as a method to enhance efficiency, aiming to “make AI search more efficient and assist people in obtaining quality answers more quickly.”
The implications
The landscape of search is evolving from traditional links to direct answers. Publishers are already concerned about losing traffic as chatbots summarize their content, a concern that has led regulators to mandate opt-outs. Cloudflare now occupies a dual role, providing tools to block crawlers and testing systems to support them. The outcome of this pilot could influence who interacts with the open web and the conditions under which they do so.
Other articles
Perplexity is discreetly developing a coding tool powered by AI.
Perplexity has developed an internal coding tool referred to as "Teammate," which may compete with Cursor and Claude Code, according to Business Insider. The launch has not yet been confirmed.
Just 6% of managers in the UK believe that Gen Z is prepared for the workforce.
A CMI report reveals that 45% of UK Gen Z believe they are prepared for work, yet only 6% of managers concur, and among those managers, 82% were elevated to their positions without any training.
The GhostApproval bug disrupts six leading AI coding agents.
Wiz's 'GhostApproval' employs an outdated symlink technique to enable six AI coding agents, ranging from Amazon Q to Cursor, to operate beyond the sandbox and transfer ownership of the box.
CEO of ElevenLabs discusses $600 million in revenue and the AI laboratories.
During the RAISE Summit in Paris, ElevenLabs CEO Mati Staniszewski spoke about achieving a reported $600 million in revenue and outlined the startup's strategies to surpass the AI labs.
Mark Cuban: Lovable and Replit have the potential to outlast AI labs.
During the RAISE Summit, Mark Cuban expressed that vibe-coding tools such as Lovable and Replit can endure in the market against Anthropic and OpenAI by focusing on controlling the workflow rather than merely the code.
Merely 6% of managers in the UK believe that Gen Z is prepared for the workforce.
A CMI report reveals that 45% of Gen Z in the UK believe they are prepared for the workplace, yet only 6% of managers concur with this assessment. Additionally, 82% of these managers were promoted without receiving any training.
Cloudflare and OpenAI are collaborating on a project to enhance the freshness of AI search results.
Cloudflare is collaborating with OpenAI to integrate real-time network signals into ChatGPT's search, marking a shift for a company that was founded on preventing AI crawlers.