IBM invests $240 million in affordable, open-source inference to compete with the hyperscalers.

      Nvidia's Blackwell systems are set to be deployed on IBM Cloud through a multi-year partnership with the startup Together AI, with the belief that businesses now prioritize the cost of AI operation over the prestige of the underlying model. IBM has concluded that the financial opportunity in artificial intelligence encompasses not just the creation of the most advanced models, but also delivering them at a lower cost, backing this idea with a $240 million investment.

      They have entered into a long-term agreement with San Francisco-based Together AI to establish a significant AI inference cluster on IBM Cloud, targeting enterprises aiming to reduce their AI expenses. This cluster will operate on Nvidia's HGX B300 systems, based on the Blackwell architecture that Nvidia promotes specifically for inference, integrated with the company's Spectrum-X Ethernet networking. This approach aligns with a trend where specialists believe that real profitability in AI lies in cost-effective inference rather than just in larger training processes.

      Together AI is an intriguing collaborator, valued at $8.3 billion as of July. Its platform enables businesses to train and operate workloads on open-source models like DeepSeek, MiniMax, and Kimi, positioning itself as a more affordable and adaptable alternative to mainstream closed systems. The company reportedly processes around 400 trillion tokens monthly, highlighting the significant volume of inference traffic and explaining why a traditional player like IBM seeks to capture this activity on its own cloud.

      Inference, which involves responding to queries post-model training, has become a major demand driver for computing resources, spurring a rush of investment and talent into this area. Nebius recently acquired a 20-person team focused on inference optimization for $643 million, a move justified only if one believes that reducing the cost per token is where profit margins reside.

      Another aspect of this situation is the gradual migration of enterprises towards open-source solutions. Companies are eager to lower their AI expenditures, and open models are gaining substantial traction within large organizations. Additionally, concerns over cybersecurity related to proprietary models from firms like Anthropic, OpenAI, and Meta have led some companies to prefer running inspectable and self-hosted solutions. For institutions like banks or hospitals, the ability to manage sensitive data on their own infrastructure is often a critical requirement.

      For IBM, offering cost-effective, open-source inference presents an opportunity to compete effectively. Given that it was never positioned to overpower Amazon, Microsoft, or Google in pure cloud scale, promoting IBM Cloud as the affordable platform for running open models enables it to compete on economic grounds rather than on size, aligning well with a growing European demand for infrastructure independent of a single American corporation.

      This trend is evident in other regions as well. The initiative to establish trustworthy and controllable inference capacity for enterprises and governments has led to the emergence of companies like TensorX, which secured €8 million to develop sovereign AI inference for Europe using Nvidia Blackwell. This scenario emphasizes that the same technology supporting IBM's strategy is being tied into a broader discussion regarding ownership of the infrastructure.

      The situation indeed resembles a gold rush, and IBM is aware of it. While the debate around open versus closed models in enterprise AI often focuses on capabilities, the critical conflict is now centered on pricing. IBM's investment in a $240 million cluster filled with Blackwell chips signifies its preference to sell the tools rather than merely the vision of advanced models. Whether affordable, open inference will prove as enduring and lucrative as its advocates anticipate remains a key gamble for the entire industry, with IBM's $240 million cluster being one of the largest bets made so far.