IBM invests $240 million in affordable, open-source inference to compete with hyperscalers.

      Nvidia's Blackwell systems will be deployed on IBM Cloud through a multi-year partnership with the startup Together AI, with the assumption that businesses prioritize the cost of AI operations over the prestige of the models utilized. IBM believes that the real value in artificial intelligence now lies not just in creating the most advanced models but in running them efficiently and has invested $240 million to support this belief.

      The agreement with Together AI, a San Francisco-based startup, will establish a large-scale AI inference cluster on IBM Cloud, targeting enterprises looking to reduce their AI expenses. This cluster will operate using Nvidia HGX B300 systems based on the Blackwell architecture, which Nvidia promotes for its inference capabilities, enhanced by the company’s Spectrum-X Ethernet networking. This approach is being pursued by many specialists who believe that the true profitability in AI comes from cost-effective inference rather than increasingly extensive training processes.

      Together AI is an intriguing choice for this partnership; valued at $8.3 billion as of July, it offers a platform that allows businesses to train and execute workloads on open-source models, such as DeepSeek, MiniMax, and Kimi. It positions itself as a more affordable and adaptable option compared to the closed, proprietary systems that are often in the spotlight.

      With the reported capacity to handle around 400 trillion tokens monthly, Together AI exemplifies the substantial demand for inference capabilities, and this is likely why a traditional company like IBM is eager to share in this traffic on its cloud.

      Inference, which involves responding to queries after a model has been trained, has emerged as a significant driver of computing resource demand, prompting an influx of investments and talents in this area. Nebius, for instance, recently invested $643 million to acquire a small team focused on inference optimization, a move that reflects the belief that reducing costs per token is where profitability lies.

      Additionally, the trend towards open source is slowly gaining traction among enterprises that aim to lower their AI expenses. Open models are increasingly being adopted within large organizations, partly due to security concerns regarding closed models from companies like Anthropic, OpenAI, and Meta. Some organizations prefer operating systems they can inspect and manage themselves, particularly when it comes to sensitive data that must remain within controlled infrastructure, such as banks or hospitals.

      For IBM, facilitating inexpensive, open-source inference presents an opportunity to compete effectively. The company recognizes it cannot match the sheer cloud scale of Amazon, Microsoft, or Google, so by branding IBM Cloud as a cost-effective platform for running open models, it can compete on economic terms rather than just size. This strategy aligns well with a growing demand in Europe for infrastructure that isn’t solely dependent on one American corporation.

      This demand is evident in other initiatives as well. The drive to create reliable and controllable inference capacity for enterprises and governments has led to startups like TensorX, which secured €8 million to develop sovereign AI inference for Europe using Nvidia Blackwell, highlighting that the same technology supporting IBM’s efforts is part of a broader discussion on infrastructure ownership.

      There is a sense of urgency surrounding this movement, and IBM is well aware of it. The competition between open and closed models in enterprise AI has frequently been framed as a matter of capability. However, the more significant struggle is evolving into a contest over pricing, and IBM's investment in a $240 million cluster of Blackwell chips is its way of indicating a preference to provide the tools rather than just the sophisticated promises.

      Ultimately, whether inexpensive, open inference becomes as enduring and lucrative as its proponents anticipate is a gamble that the entire sector is quietly taking, with IBM’s $240 million cluster representing one of the more significant bets in this arena.