As AI moves from answering questions to acting, organizations face a harder test of responsibility.
Imagine opening your business dashboard on a Monday morning and discovering that your AI agent has issued a customer a $2,000 refund.
The customer had a legitimate complaint. The agent reviewed the account, found the billing error, and calculated the correct amount. It even sent a thoughtful apology.
There is just one problem: refunds above $200 require a manager’s approval.
The AI got the answer right and still crossed a boundary that mattered.
This hypothetical situation captures a problem that deserves more attention as organizations adopt agentic AI. We have spent years asking whether AI outputs are accurate, fair, and reliable. Those questions remain essential. But when a system can send messages, update records, approve requests, or transfer money, accuracy alone cannot determine whether it should proceed.
We also need to ask who gave it permission.
That question is becoming increasingly practical. On October 15, the Financial Stability Board, an international body that coordinates financial authorities, published its final sound practices for responsible AI adoption by financial institutions. The report emphasizes clear responsibilities, ongoing risk assessment, and effective human oversight. The same day, ServiceNow announced AI Passport, a solution intended to control and verify what AI agents are authorized to do during operation. These are different responses to a shared concern: how to preserve accountability as AI takes on more work.
For a business leader, the appeal of an AI agent is easy to understand. A customer issue that once took three departments to resolve might be resolved in minutes. An employee could spend less time copying information between systems and more time addressing difficult cases. Faster service can be a real benefit to the people waiting for help.
The difficulty is that a broad instruction such as “resolve this customer’s problem” leaves many decisions unstated.
May the agent offer compensation? How much? Can it change the customer’s contract? What happens when the customer disputes the evidence? At what point must someone with the appropriate authority take over?
People often navigate these boundaries through experience, workplace relationships, and an understanding of what their role permits. Even then, organizations use approval limits, access controls, and escalation procedures because good intentions do not prevent costly mistakes.
An AI agent needs those boundaries made explicit and enforced in the systems it uses. Writing “seek approval when necessary” in an instruction is a starting point. It becomes a dependable control only when the organization defines what “necessary” means and ensures that the agent cannot simply continue past that point.
The refund example makes the distinction clear. If the limit is $200, the system should route the $2,000 refund to an authorized person before money moves. That person needs enough information to plan: the billing discrepancy, the relevant policy, the proposed amount, and any uncertainty in the evidence.
Otherwise, human approval risks becoming a hurried click on a screen. This is where the familiar promise of a “human in the loop” needs closer examination. A person can be present without having a meaningful opportunity to intervene. If the action has already occurred, the reviewer is dealing with the consequences. If the reviewer cannot inspect the evidence or stop the process, their involvement offers limited protection.
Meaningful oversight requires time, information, and authority.
It also requires an organization willing to treat escalation as a successful outcome. An agent that pauses at its approval limit may look slower on a performance dashboard than one that completes every request. Yet that pause could be exactly what responsible operation requires. If speed and completion rates are the only measures of success, appropriate caution can start to look like poor performance.
My work with the SAFER AI™ Protocol focuses on the moment of reliance: the point at which someone accepts an AI output or allows it to shape an action. With agentic AI, that moment may happen quickly and repeatedly across a workflow. It deserves deliberate attention.
Before a consequential action proceeds, the organization should be able to establish the scope of the task, the authority to act, the ways the system could fail, the evidence supporting the decision, and the record that will remain afterward. Those considerations connect responsible AI principles to everyday operations. They help explain why a seemingly successful transaction can still reveal a governance failure. They also keep responsibility where it belongs. When an agent exceeds its limits, saying “the AI decided” leaves the important work unfinished. Someone selected its tools, granted it access, set its approval rules, and accepted the conditions under which it could operate. Understanding those choices is essential to fixing the problem.
Organizations do not need to remove every human approval step to benefit from AI. They need to make careful decisions about which actions can proceed independently, which require review, and how those decisions will be tested as the system changes.
A useful place to begin is with one question about an agent already in use: What is the most consequential action it can take without asking anyone?
Then check whether the answer matches what leadership intended.
The refund may be correct. The customer may be happy. The workflow may be faster.
Who authorized the $2,000?
