EY developed an 'AI router' to prevent their AI expenses from escalating.

      EY has developed a system it refers to as an “AI router,” designed to direct each task to the most cost-effective model capable of managing it, as a means of controlling its escalating AI expenses. According to Business Insider, this tool addresses a growing issue in corporate IT: the costs associated with AI token consumption. The concept is based on straightforward arbitrage.

      Not every request requires the most powerful and expensive model, so the router efficiently allocates simpler tasks to less costly models while reserving the high-end ones for more complex challenges, thereby reducing costs without noticeably affecting output.

      EY has a vested interest in monitoring expenses. The firm allocates over $1 billion annually to AI, operates around 1,000 AI agents, and has recorded a 30% increase in AI-related consulting revenue, a level where token expenses become significant, particularly as some of the most AI-focused companies spend thousands of dollars per employee monthly.

      Research by EY indicates that this concern is widespread. In their recent AI Pulse survey, which involved 534 senior business leaders in the US, 82% expressed worries regarding token usage costs, with 98% of those utilizing token-based tools indicating that expenses had led them to reconsider their strategies.

      However, many organizations lack insight into their spending. Only 64% of the surveyed firms actively monitor token usage against budgetary limits, implying that a significant portion is investing in AI without clear oversight, which can lead to substantial billing surprises in the industry.

      The prevailing sentiment has transitioned from excess to moderation. Dan Diasio, EY’s global AI consulting leader, expressed it succinctly: “’AI saves time’ is no longer a sufficient justification when costs rise and remain unclear,” reflecting a shift from prioritizing adoption at any cost to seeking value with transparency.

      The economics of tokens are indeed perplexing. Although the price per token has decreased as models have become cheaper, corporate AI expenses have tripled due to agentic tools requiring significantly more tokens than traditional single chatbot prompts.

      This paradox is precisely what the router aims to resolve. By aligning each task with the least expensive model that can fulfill it, companies can continue to leverage AI while preventing expenses from spiraling, which is exactly what EY aspires to demonstrate on its own scale.

      EY is not alone in this endeavor. Over the past two years, the industry has encouraged employees to utilize AI extensively, a trend referred to as tokenmaxxing, but is now shifting towards implementing budgets and controls, with companies like Atlassian and Amazon leading the charge.

      For a consultancy, the router also serves as a marketing tool. EY provides AI consulting to other businesses, so a system that effectively manages its own costs serves as a showcase of its capability to assist clients in achieving similar outcomes.

      The survey aligns with this observation. Approximately 76% of leaders told EY that readily available software no longer fulfills their requirements, and 91% now consider developing AI tools internally as essential, a trend that benefits companies offering the expertise needed for such development.

      However, the challenge lies in the complexity of in-house development. Almost 75% of leaders surveyed by EY indicated that their AI projects were slowing down progress, with one-third citing risks related to shadow IT and governance, exposing the challenges behind the promise of a streamlined router.

      Ultimately, what this narrative signifies is a shift in focus. The initial phase of the AI boom concentrated on whether a tool functioned properly; the subsequent phase, to which EY’s router belongs, emphasizes cost-effectiveness and whether the benefits justify the expenses.

      For the time being, EY’s strategy is to create its own tracking system. A firm investing a billion dollars annually in AI recognizes that diligent monitoring of every token is the key to sustaining its expenditures, representing not a retreat from AI but a clear indication of its extensive utilization.