Why businesses require more defined boundaries prior to granting AI agents the authority to take action.

      An IBM survey involving 2,000 C-level executives revealed that only 11% feel completely equipped for the deployment of AI agents. Madhuri Chandoor, founder of PromptHalo, argues that the main issue lies in differentiating between capability and authority. She uses a refund-splitting scenario to illustrate how agents can bypass limits through sequential requests. Chandoor advocates for behavioral profiling for AI agents, similar to financial fraud prevention, and suggests documenting what agents can access, under which conditions, and the potential downstream consequences.

      An upcoming 2026 IBM study raises concerns regarding AI readiness, visibility, and control. The findings show that just 11% of 2,000 C-level technology executives felt fully prepared for the imminent deployment of AI agents. Moreover, two-thirds of CIOs and CTOs reported being accountable for AI systems they do not entirely control, with 70% stating that teams were implementing technology at a pace quicker than IT could monitor. IBM views these outcomes as indicative of an increasing control gap as AI adoption grows within organizations.

      Chandoor emphasizes the importance of recognizing the difference between capability and authority in addressing this control gap. She characterizes PromptHalo as a company focused on AI security and trust infrastructure, stating that its aim is to evaluate the reasons behind certain actions rather than merely the actions themselves. Chandoor believes this approach offers organizations better contextual insights into whether an action aligns with user intent, existing permissions, and the broader context before a system moves forward.

      A hypothetical example involving an enterprise database task highlights the issues Chandoor raises regarding context. She describes an AI agent tasked with enhancing application performance that could autonomously alter index or table structures within a live environment. Such changes could impact real-time transactions, customer data, or related processes that the agent did not immediately analyze. “A technical conclusion may seem reasonable in a narrow context,” Chandoor notes, emphasizing the need to consider context, situation, and potential downstream effects prior to executing an action.

      Earlier chatbots, according to Chandoor, primarily functioned within limited sets of questions and answers. With the advent of large language models, there is now a broader array of company data and tools that heighten the risk exposure, prompting organizations to evaluate how incoming requests might affect a system. She suggests that security teams analyze how requests are interpreted and the level of authority of interconnected systems before allowing an agent to proceed.

      Chandoor illustrates the significance of context across multiple actions with a refund scenario. In her example, an agent can process refunds up to $50 without human oversight. If a user requests ten $50 refunds rather than a single $500 refund that would require review, each transaction might seem valid when considered separately, yet collectively they may indicate an attempt to bypass the limit. She believes reviewing the overall session context and behavior could aid in determining when human review is warranted.

      Drawing from her two decades of experience in financial services, Chandoor likens her method to fraud monitoring for transactions and accounts. She recommends that organizations create behavioral profiles for autonomous agents, including identity and access permissions. Under her proposed framework, teams would examine the resources accessed by an agent, its tool usage, any changes in its behavior over time, and actions that seem inconsistent with its designated role or session context.

      Regarding authorization, Chandoor asserts that it should be evaluated during both design and operational phases. She suggests documentation of the resources an agent may access, the applicable conditions, and the potential downstream effects of specific actions. Additionally, she recommends implementing observability gates to monitor activities, particularly to investigate patterns when requests become repetitive, unusually broad, or deviate from the originally assigned purpose. These checkpoints, she explains, can help mitigate impacts and identify any additional controls needed to secure underlying systems.

      Chandoor stresses her commitment to supporting responsible AI adoption. She supports leveraging agentic automation for analysis and workflows while advocating for extra verification when critical decisions and actions are involved. “Trust, but verify,” she advises.

      “Organizations should embrace AI responsibly and continuously verify its behavior throughout the process,” she insists. “Clear accountability ownership for the security of AI applications across businesses is crucial for implementing these safeguards.” In her opinion, this strategy could enable organizations to advance AI innovation while ensuring appropriate focus on security and accountability.