Nadella's Reverse Information Paradox: The unseen cost of AI

      Satya Nadella, CEO of Microsoft, asserts that every company utilizing AI is essentially paying for it twice: once with cash and again with the information they provide to make the technology effective. He refers to this as the Reverse Information Paradox, while also acknowledging that his own company contributed to creating this predicament.

      Nadella offers a cautionary message to those purchasing AI solutions: you are making a twofold payment, with the second payment being your most valuable assets. In a lengthy essay on X that garnered 10 million views, he elaborated on this concept, which he describes as sharp, somewhat complex, and somewhat awkward coming from him.

      The premise is that you make one payment in cash and another in secrets. This idea is inspired by Nobel economist Kenneth Arrow's original paradox, which centered on the seller's dilemma. To sell information, one often must disclose it, which raises the question of why anyone would pay for something that is now exposed.

      Nadella turns this on its head. He argues that in the realm of AI, the burden lies with the buyer. To truly enhance a model's effectiveness, proprietary knowledge must be input. The more accurate you want the model to be, the more information you need to provide. Thus, you pay with money and again with something even more valuable: the insights that define your business. He stated, “The seller learns more and more about you as you use what you purchased,” while “you learn very little about what the seller is learning in return.”

      The nuanced aspect of his argument lies in where this knowledge escapes. It's not through glaring security breaches but rather through what he terms “exhaust”: the prompts you create, the tools used by your agents, and especially the adjustments you make when the model makes errors. Each correction enhances the model’s learning. “It’s the kind of knowledge a competitor could never buy,” Nadella wrote, “and the kind that leaks almost imperceptibly: trace by trace, correction by correction, eval by eval.” His conclusion is straightforward: if learning is unidirectional, then the financial flow follows suit, favoring the AI owner over the knowledge provider.

      The twist is that Microsoft is the one delivering this message. The company has invested billions into OpenAI and operates ChatGPT through Azure. Its Copilot assistant is designed to integrate deeply with a company’s emails, files, and communications. In a 2024 survey, nearly half of data leaders reported that they had restricted or halted the use of Copilot due to these concerns, as highlighted by the Register.

      Nadella does acknowledge the double standards present within his own organization. AI labs seek appropriate rights to utilize public data for training while simultaneously limiting customers’ use of model outputs. His point is valid, yet he is also promoting a solution.

      Nadella proposes a robust “trust boundary” surrounding a company's data, evaluations, and memory, ensuring nothing crosses it without permission, including “intelligence exhaust.” He references a sentiment from Palantir’s Alex Karp about wanting to control the means of production.

      His recommendations encompass five points: own your evaluations, create learning environments within your own tenant boundary, ensure the orchestration layer remains independent of any single model, and allow everything to build upon itself. Microsoft offers products that facilitate each of these aspects.

      Despite the promotional angle, the fundamental message remains clear: this is the same leader who has criticized the AI titans he played a role in establishing. The leading AI labs continue to accumulate significant knowledge from other businesses, while the firms providing that information, for the time being, are doing so without compensation.