The energy consumption of AI was already troubling. Studies indicate that AI agents are over a hundred times more energy-intensive.

      The rapidly increasing electricity consumption in the AI industry has become a significant concern for governments, utility companies, and tech firms. A recent study indicates that the next generation of artificial intelligence may exacerbate this issue considerably.

      Researchers from the Korea Advanced Institute of Science and Technology (KAIST) have released what they refer to as the first thorough analysis of the energy expenses associated with AI agents—systems that can reason, plan, and autonomously perform tasks. Their research reveals that these agents can use up to 136.5 times more energy per query than traditional generative AI models, prompting new inquiries into whether the infrastructure for future AI can cope with the impending demand.

      Enhanced AI comes with significantly higher electricity costs

      In contrast to standard chatbots that provide a single answer to a query, AI agents repeatedly engage large language models (LLMs), browse the internet, run code, use calculators, and interact with other software to address complicated tasks. While these abilities enhance their utility for research, programming, and workplace automation, they also demand far greater computing resources.

      Led by Professor Minsoo Rhu from KAIST’s School of Electrical Engineering, the research team classified AI agents as a distinct category of data center workload, assessing their computational needs in real-life situations.

      The findings were remarkable. The team discovered that AI agents could increase response latency by as much as 153.7 times when compared to traditional chain-of-thought reasoning. Surprisingly, the costly GPUs that drive these tasks remained idle for up to 54.5 percent of the execution time while awaiting external tools to complete their jobs. This means that the hardware continues to utilize power even when it's not actively engaged in AI computation.

      Energy consumption scales in a similarly dramatic fashion. Operating an AI agent that employs a 70-billion-parameter language model, comparable to current commercial AI systems, used an average of 348.41 watt-hours per query—approximately 136.5 times more than a typical chatbot responding to a simple question.

      To gauge the broader impact, the researchers envisioned a future where AI agents manage 13.7 billion requests daily, roughly aligning with Google's daily search volume. In this scenario, AI infrastructure would require about 198.9 gigawatts of electricity, nearly half of the average power consumed across the United States, far surpassing the capacity of existing AI data centers.

      The hidden price of intelligence

      These findings come at a time when companies like OpenAI, Google, Microsoft, and Anthropic increasingly invest in agentic AI, touted as the next significant advancement beyond conversational chatbots. However, the study asserts that merely enhancing AI models is insufficient. Future advancements will hinge equally on more efficient semiconductors, improved GPU usage, smarter data center design, and expanded power infrastructure.

      Professor Rhu notes that the research illustrates a shift in AI competitiveness from developing "smarter AI" to creating more efficient AI. The team believes that upcoming AI advancements will necessitate a co-design approach, whereby models, AI chips, servers, and energy systems are optimized together to keep operating costs sustainable and ensure AI can scale effectively.

      The paper, titled “The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective,” was presented at the IEEE International Symposium on High-Performance Computer Architecture (HPCA) earlier this year. The researchers have also made their AI agent benchmarks available as open-source, aiming to foster further efforts in minimizing one of AI’s quickest-growing—and often overlooked—expenses: electricity.