AWS GPU prices rise by 20% due to memory shortages.

      Renting an AI chip is increasingly resembling booking a hotel in a fully booked city. You pay to secure the room, and the rates continue to escalate. On AWS, this just happened again.

      Amazon Web Services has increased the prices for EC2 Capacity Blocks for machine learning by about 20% starting in July. This change was first reported by Business Insider, and AWS has confirmed it. This service allows companies to reserve Nvidia GPUs in advance, ensuring that lengthy training runs continue without interruption.

      This marks the second price hike in six months, with AWS having already raised the same prices by around 15% in January. When combined, the cost of securing this computing power has surged significantly since the start of the year. AWS indicated that the prices fluctuate “periodically based on supply and demand.”

      The increase is specific rather than widespread. It affects one purchasing option: the reserved blocks preferred by serious AI teams that are training or refining large models. According to AWS, other options maintain fixed prices, which the company claims will remain stable. The price increase did not affect Trainium, Amazon’s proprietary AI chip, as per The Information.

      The extent of this increase is important due to the larger implications. As the largest cloud provider globally, AWS hosts numerous AI services. When the prices for its highest-tier compute rise, the effects reverberate to the start-ups and enterprises utilizing these services. AWS characterized this decision as evidence of the escalating demand for GPUs.

      The reason the cloud could not maintain low costs

      The notable shift to observe is where the constraints have now developed. For the past two years, the limitation on AI was mainly software and expertise. Currently, it is physical resources. The bottleneck now lies with high-bandwidth memory, the chips that accompany AI processors, and there is a finite supply of it. As Business Insider highlighted, the limitation on AI has transitioned from software to hardware, and creating hardware requires years.

      This situation is direct and unforgiving. A shortage of memory results in fewer GPUs. With fewer GPUs, there are fewer data centers. “With a cap on how much memory can be produced, there’s a cap on GPU production, which in turn limits the number of data centers that can be constructed,” noted Peter Berezin, chief economist at BCA Research, on X.

      Such scarcity gives cloud providers leverage. When GPU availability is limited, customers have few options, allowing AWS, Microsoft, Google, and Oracle to pass increased costs directly onto them. The shortage raises their operational expenses, Berezin pointed out, but it also maintains demand that exceeds supply, granting them pricing authority over who can access computing resources.

      A widespread price increase

      Amazon is not the only company experiencing this trend, and that is significant. Apple recently raised prices across its Mac and iPad lineups, attributing the hike to memory costs. Xbox has done the same. Elon Musk referred to the rise in memory prices as the largest increase he has witnessed in any product. Now, these same memory costs are reflecting in cloud billing.

      On the flip side of the shortage is a financial boon. The rising AWS GPU prices have elevated the market valuations of memory manufacturers Micron and SK Hynix to record levels. Investors are betting that the high-bandwidth memory shortage will keep the market constrained, causing prices to remain elevated for years.

      For AWS customers, the implication is clear. The lowest cost for AI computing is now behind them, and pressing the reserve button has become more expensive. The lingering question is how far this trend will extend. If a 15% increase led to another 20% rise in six months, the teams developing the most ambitious models are left uncertain about the cost of their next reservation.