EY developed an 'AI router' to prevent its artificial intelligence expenses from escalating.

      EY has developed what it refers to as an “AI router,” a system designed to direct each task to the most cost-effective model capable of handling it, aiming to manage its increasing AI expenses. This tool, reported by Business Insider, addresses a growing challenge within corporate IT: the rising costs associated with the tokens utilized by AI. The underlying principle is straightforward arbitrage.

      Not every request necessitates the use of the most powerful and expensive model; therefore, the router allocates simpler tasks to less costly models while reserving the pricier models for more complex issues, thereby reducing costs without noticeably diminishing output.

      EY has a strong incentive to monitor its expenses. The firm invests over $1 billion annually in AI, operates a fleet of around 1,000 AI agents, and has experienced about a 30% increase in AI-related consulting revenue. At this scale, token costs become significant in an environment where top AI-focused companies may spend thousands per employee monthly.

      According to its own research, EY recognizes that this concern is widespread. In EY’s most recent AI Pulse survey of 534 senior business leaders in the US, 82% expressed worries about token costs, and 98% of those employing token-based tools stated that these costs had prompted them to rethink their strategies.

      However, most companies lack visibility into their spending. Only 64% of the surveyed firms reported actively monitoring token usage with budgetary limits, indicating that a third are engaging in AI expenditures without a clear understanding of costs, which could lead to unexpected financial shocks in the industry.

      The overall perspective has shifted from more to enough. Dan Diasio, EY’s global AI consulting leader, succinctly stated: “‘AI saves time’ is no longer enough when costs accumulate and remain unclear,” highlighting the shift from adopting technology at any price to seeking value with known expenses.

      The economics of tokens present a unique situation. Although the price per token has plummeted as models have become cheaper, enterprise expenses for AI have tripled, as tools that operate through multiple steps require far more tokens than traditional single chatbot prompts.

      This is precisely the challenge the router is designed to address. By matching each task to the least expensive model that can adequately fulfill it, companies can continue to leverage AI without drastically increasing their overall costs, a goal EY aims to validate on its own scale.

      EY is not alone in this endeavor. After two years of encouraging extensive AI usage—a trend referred to as tokenmaxxing—the industry is now shifting towards implementing budgets and controls, with companies ranging from Atlassian to Amazon leading this change.

      For a consulting firm, the router also acts as a product. EY offers AI consultancy services to other organizations, meaning that a tool capable of effectively managing its own costs serves as a demonstration of the firm’s capability to replicate those savings for clients.

      The survey results reflect this trend. Approximately 76% of leaders indicated to EY that off-the-shelf software no longer suffices for their needs, and 91% view the development of in-house AI tools as essential, a shift that benefits firms providing the expertise to create such tools.

      The challenge is that developing in-house solutions is complex. Nearly 75% of the leaders surveyed by EY reported that their AI development was slowing progress, with one-third acknowledging risks related to shadow IT and governance, exposing the complications behind the seemingly straightforward concept of a router.

      What this narrative signifies is a shift in focus. The initial stage of the AI boom concentrated on whether a tool functioned; the current phase, which includes EY’s router, emphasizes the costs involved and whether the value justifies those expenses.

      For now, EY’s strategy is to create the measure itself. A firm that invests a billion dollars annually in AI has concluded that the key to maintaining spending is diligent tracking of every token, indicating a deeper integration of AI use rather than a retreat from it.