Human approval belongs immediately before an AI agent action creates a material consequence that is difficult to reverse. Use consequence, reversibility, authority, evidence quality and time sensitivity to classify each tool call. The resulting policy should assign one of four paths: run automatically, run within a bounded rule, pause for an authorised reviewer or prohibit the action.
This is the job of an AI agent approval matrix. It replaces blanket “human in the loop” requirements with an operating policy that distinguishes reading a customer record from issuing a refund, and drafting an email from sending it.
Start with the action, not the agent
Approving an agent as a whole is too broad. The same support agent may search an approved knowledge base, draft a reply, change an account entitlement and issue a credit. Those actions do not deserve the same authority.
List the concrete tools available to the agent and write each action at the level where a side effect occurs:
- search customer records
- draft a customer response
- send the response
- update the case status
- apply an account credit
- export customer data
This action inventory prevents a common policy failure. A team labels the workflow “human reviewed” because somebody checks the final answer, while an earlier tool call has already changed a record or exposed restricted information.
OpenAI’s Agents SDK guidance on guardrails and approvals makes the boundary explicit: checks around a custom tool call need to sit beside that tool. Agent-level input and output guardrails do not cover every action in a chain.
Score five properties before assigning authority
The matrix needs enough detail to expose real risk without becoming a compliance spreadsheet nobody maintains.
Consequence
Record what changes if the action succeeds. A tool may affect money, customer communication, production systems, employee access, contractual commitments or regulated data. Rate the consequence against the specific workflow rather than the tool name. Updating a test record and updating a live customer record use the same API but create different exposure.
Reversibility
Define the recovery operation and test it. “We can undo it” is not a control until the team knows how long reversal takes, what evidence survives and whether somebody else acts on the change first.
A CRM field update may be technically reversible but commercially costly once a sales sequence reads it. A sent message cannot be recalled from the recipient’s judgement. A production deletion can be recoverable from backup while still causing downtime.
The AI agent rollback plan covers operation IDs, action ledgers, idempotent retries and compensating steps for actions that need engineered recovery.
Authority
Name the identity that executes the tool call, the permission it uses and the person authorised to approve it. Avoid a shared service account with enough access to cover every branch of the workflow.
NIST’s NCCoE project on software and AI agent identity and authorisation is examining standards-based approaches to identify agents and manage their access and actions. Its current project status is “reviewing comments”, so it signals the direction of implementation guidance rather than a finished standard.
Evidence quality
Show which records support the proposed action, when they were retrieved and where they conflict. An approval request that says “issue £500 credit?” forces the reviewer to reconstruct the case. The useful packet includes the order, policy, customer history, agent rationale, exact tool arguments and the expected state after execution.
Weak evidence should change the route. The agent can request missing context or escalate the case rather than asking a reviewer to approve a guess.
Time sensitivity
Record how long the decision can wait and what happens when nobody responds. Approval expiry must fail closed for consequential actions. The workflow should preserve its state, notify the right queue and avoid silently converting delay into permission.
Microsoft’s Agent Framework human-in-the-loop documentation shows the underlying pattern: a workflow emits a request, pauses, receives an external response and resumes with saved state. A working business process still needs an owner, response window, delegation rule and expiry path around that mechanism.
Use four execution paths
The matrix below gives teams a starting policy. Thresholds need to be set against the company’s workflow, data and risk appetite.
| Execution path | Suitable action profile | Example | Required controls |
|---|---|---|---|
| Automatic | Low consequence, reversible, authorised and supported by current evidence | Retrieve an approved internal article | User-scoped identity, source permissions, logging and rate limits |
| Conditional | Bounded consequence with deterministic limits and tested recovery | Update a case category from an approved taxonomy | Field allowlist, confidence or rule threshold, post-action verification and exception queue |
| Human approval | Material consequence, weak reversibility, elevated permission or judgement under conflicting evidence | Send an external commitment, issue a high-value refund or change production access | Authorised reviewer, evidence packet, exact arguments, expiry, recorded decision and recovery route |
| Prohibited | Authority is absent, recovery is unacceptable or the action breaches policy | Export restricted records to an unapproved destination | Tool unavailable to the agent, independent enforcement and alerting on attempted use |
The prohibited row matters. Human approval does not make every action acceptable. A reviewer cannot grant authority the organisation has not defined, and a chat confirmation should not override a technical boundary.
Put the approval gate beside the side effect
A useful approval request contains the decision the reviewer is being asked to make. Include:
- the proposed action and exact tool arguments
- the business object affected
- the executing identity and effective permission scope
- the evidence used, including timestamps and conflicts
- the expected consequence and recovery method
- the rule that triggered review
- the time limit and result of no response
The reviewer should be able to approve, reject, edit or escalate. Editing matters because real judgement rarely fits a binary button. If the reviewer changes the refund amount, recipient or access duration, record the final arguments as a new authorised action rather than treating the original request as approved.
OpenAI’s practical guide to building agents recommends rating tools by factors such as read versus write access, reversibility, account permissions and financial impact. It then uses those ratings to trigger safeguards or human escalation. The approval matrix turns that build-time advice into a policy the workflow owner can inspect and revise.
Preserve the decision when the run resumes
Long-running review creates a state problem. The evidence can change while the request is waiting. The customer may cancel the order, an access request may expire or another operator may resolve the case.
Bind the approval to:
- the workflow run and action ID
- the proposed arguments
- the evidence version or retrieval time
- the reviewer identity
- the decision time and expiry
Revalidate mutable facts before execution. If the order, permission or policy changed after approval, route the action back through review instead of replaying an old decision against a new state.
The decision record also improves evaluation. Teams can see which actions reviewers rejected, what they edited and where evidence was missing. Feed those cases into the test set and release gate described in Copilot Studio agent evaluation.
Measure review load against accepted outcomes
Approval rate alone tells you little. A low rate can mean the policy is well targeted, or that dangerous actions bypass review. A high rate can reflect appropriate caution, or an agent that saves no labour because every step waits in a queue.
Track:
- actions by execution path and consequence band
- review time and expiry rate
- rejection and edit reasons
- permission or evidence failures caught before execution
- side effects verified after execution
- reversals, disputes and incidents
- accepted outcomes per hour of reviewer effort
Connect these measures to the AI agent pilot success criteria. Expansion should depend on accepted outcomes, clean authority boundaries and tested recovery, rather than the number of tool calls completed.
Build the first matrix around one live workflow
Choose a workflow where the agent already needs to read evidence and propose a consequential action. Map every tool call, classify it, then test each route with representative cases and failure conditions.
Model Operator is implementing and validating a governed company operating layer for this kind of cross-system work. The Design Partner engagement connects permission-aware company evidence, accepted truth, approvals, audit records and workflow outcomes across agreed tools. An approval matrix is one implementation artefact inside that work, tied to the identities and source systems that enforce it.
If a live agent can write to customer, financial or operational systems and its approval policy still reads “human review required”, bring that workflow to Model Operator or email alexander@modeloperator.io. The first step is to turn the label into action-level authority the team can test.