Nvidia’s CUDA advantage encounters its initial significant challenge: artificial intelligence.

      The common narrative surrounding Nvidia centers on hardware: it offers the fastest chips, which are in limited supply and priced at a premium. However, the more crucial aspect is software. For the past two decades, Nvidia’s true competitive edge has been CUDA. This technology converts its silicon into a platform that developers can utilize for AI development. That advantage is currently being put to the test.

      CUDA, which stands for Compute Unified Device Architecture, required several years to develop. It includes pre-built code and debugging tools, allowing numerous chips to collaboratively train a model. The emerging threat is straightforward: AI coding agents capable of autonomously writing low-level software.

      Jeremy Nixon, a former researcher at Google Brain, established the startup Infinity. He shared with Business Insider that his team utilized agents to recreate software similar to CUDA for the chip company D-Matrix in approximately 10 hours. He presented this as evidence that one of Nvidia's significant advantages is being challenged.

      Two competitive edges, both under pressure

      CUDA's first strength lies in its software, while its second stems from everything built upon it. Millions of lines of proprietary code and workflows make transitioning to a competitor's chip slow and expensive. In the past, Amazon's documents identified CUDA as a significant obstacle to adopting its own AI chips.

      Agents chip away at the first advantage, and the pressure comes not just from startups. Companies like Google, Amazon, and Microsoft have invested years in developing software for their own chips. OpenAI and Anthropic have demonstrated models capable of generating system code. The founder of DeepSeek stated that coding agents, along with its unique programming language, significantly simplify the development of AI software.

      Nvidia does not deny this trend; instead, it embraces it. The company asserts that developers increasingly rely on CUDA’s libraries each year and that it also employs coding agents to expedite CUDA’s development. In other words, the lock-in effect might flex but not break.

      The inference challenge

      A more pressing concern is a transformation in the purpose of AI chips. As the industry shifts from training models to executing them, new priorities emerge. Buyers are increasingly focused on cost-effective AI performance rather than maximum peak performance. This shift favors software capable of operating across various chips instead of software tied to a specific vendor.

      According to Marshall Choy of the Korean chip startup Rebellions, CUDA “is no longer a factor” in inference. He views it as an open-source opportunity. This aligns with the pursuits of other inference competitors, including developments in optical chips, networking silicon, and Alibaba’s open-source alternative to CUDA.

      Chris Lattner, whose startup Modular creates chip-agnostic AI software, notes that CUDA’s age is a double-edged sword. He suggests it carries years of legacy from its gaming roots, akin to “Microsoft Windows trying to fit onto a phone.” Wall Street has taken notice, with analysts interpreting Nvidia's stagnant stock over the past year as a partial sign of resistance against its moat.

      The moat may just shift

      An alternative perspective is that agents may merely shift the moat rather than eliminate it. Generated code must still be verified and optimized, which remains one of CUDA’s strongest ecosystems. Bing Xu, whose last chip-software startup was acquired by Nvidia, contends that verification could become the next competitive advantage.

      “Agents can produce a substantial amount of code quickly, but verification poses the biggest challenge,” Xu remarked. Lattner is even more direct, suggesting that the excitement surrounding agents is “very overblown.” He emphasizes that writing code is merely a fraction of software development; the real challenge lies in fine-tuning it for production. Furthermore, chip software is a specialized domain with few public examples available for agents to learn from.

      Thus, the clear takeaway is not that CUDA is declining; rather, for the first time, the shield protecting Nvidia has a believable expiration date. Agents can accomplish in hours what once took dedicated teams years to complete. Inference favors those who liberate buyers from a single technology stack. Ultimately, whether the moat withstands pressure hinges on one critical factor: rivals must close the gap faster than Nvidia, which is “not sleeping,” can create a new advantage.