Human approval belongs immediately before an AI agent action creates a material consequence that is difficult to reverse. Use consequence, reversibility, authority, evidence quality and time sensitivity to classify each tool call. The resulting policy should assign one of four paths: run automatically, run within a bounded rule, pause for an authorised reviewer or prohibit the action.

This is the job of an AI agent approval matrix. It replaces blanket “human in the loop” requirements with an operating policy that distinguishes reading a customer record from issuing a refund, and drafting an email from sending it.

Start with the action, not the agent

Approving an agent as a whole is too broad. The same support agent may search an approved knowledge base, draft a reply, change an account entitlement and issue a credit. Those actions do not deserve the same authority.

List the concrete tools available to the agent and write each action at the level where a side effect occurs:

  • search customer records
  • draft a customer response
  • send the response
  • update the case status
  • apply an account credit
  • export customer data

This action inventory prevents a common policy failure. A team labels the workflow “human reviewed” because somebody checks the final answer, while an earlier tool call has already changed a record or exposed restricted information.

OpenAI’s Agents SDK guidance on guardrails and approvals makes the boundary explicit: checks around a custom tool call need to sit beside that tool. Agent-level input and output guardrails do not cover every action in a chain.

Score five properties before assigning authority

The matrix needs enough detail to expose real risk without becoming a compliance spreadsheet nobody maintains.

Consequence

Record what changes if the action succeeds. A tool may affect money, customer communication, production systems, employee access, contractual commitments or regulated data. Rate the consequence against the specific workflow rather than the tool name. Updating a test record and updating a live customer record use the same API but create different exposure.

Reversibility

Define the recovery operation and test it. “We can undo it” is not a control until the team knows how long reversal takes, what evidence survives and whether somebody else acts on the change first.

A CRM field update may be technically reversible but commercially costly once a sales sequence reads it. A sent message cannot be recalled from the recipient’s judgement. A production deletion can be recoverable from backup while still causing downtime.

The AI agent rollback plan covers operation IDs, action ledgers, idempotent retries and compensating steps for actions that need engineered recovery.

Authority

Name the identity that executes the tool call, the permission it uses and the person authorised to approve it. Avoid a shared service account with enough access to cover every branch of the workflow.

NIST’s NCCoE project on software and AI agent identity and authorisation is examining standards-based approaches to identify agents and manage their access and actions. Its current project status is “reviewing comments”, so it signals the direction of implementation guidance rather than a finished standard.

Evidence quality

Show which records support the proposed action, when they were retrieved and where they conflict. An approval request that says “issue £500 credit?” forces the reviewer to reconstruct the case. The useful packet includes the order, policy, customer history, agent rationale, exact tool arguments and the expected state after execution.

Weak evidence should change the route. The agent can request missing context or escalate the case rather than asking a reviewer to approve a guess.

Time sensitivity

Record how long the decision can wait and what happens when nobody responds. Approval expiry must fail closed for consequential actions. The workflow should preserve its state, notify the right queue and avoid silently converting delay into permission.

Microsoft’s Agent Framework human-in-the-loop documentation shows the underlying pattern: a workflow emits a request, pauses, receives an external response and resumes with saved state. A working business process still needs an owner, response window, delegation rule and expiry path around that mechanism.

Use four execution paths

The matrix below gives teams a starting policy. Thresholds need to be set against the company’s workflow, data and risk appetite.

Execution pathSuitable action profileExampleRequired controls
AutomaticLow consequence, reversible, authorised and supported by current evidenceRetrieve an approved internal articleUser-scoped identity, source permissions, logging and rate limits
ConditionalBounded consequence with deterministic limits and tested recoveryUpdate a case category from an approved taxonomyField allowlist, confidence or rule threshold, post-action verification and exception queue
Human approvalMaterial consequence, weak reversibility, elevated permission or judgement under conflicting evidenceSend an external commitment, issue a high-value refund or change production accessAuthorised reviewer, evidence packet, exact arguments, expiry, recorded decision and recovery route
ProhibitedAuthority is absent, recovery is unacceptable or the action breaches policyExport restricted records to an unapproved destinationTool unavailable to the agent, independent enforcement and alerting on attempted use

The prohibited row matters. Human approval does not make every action acceptable. A reviewer cannot grant authority the organisation has not defined, and a chat confirmation should not override a technical boundary.

Put the approval gate beside the side effect

A useful approval request contains the decision the reviewer is being asked to make. Include:

  1. the proposed action and exact tool arguments
  2. the business object affected
  3. the executing identity and effective permission scope
  4. the evidence used, including timestamps and conflicts
  5. the expected consequence and recovery method
  6. the rule that triggered review
  7. the time limit and result of no response

The reviewer should be able to approve, reject, edit or escalate. Editing matters because real judgement rarely fits a binary button. If the reviewer changes the refund amount, recipient or access duration, record the final arguments as a new authorised action rather than treating the original request as approved.

OpenAI’s practical guide to building agents recommends rating tools by factors such as read versus write access, reversibility, account permissions and financial impact. It then uses those ratings to trigger safeguards or human escalation. The approval matrix turns that build-time advice into a policy the workflow owner can inspect and revise.

Preserve the decision when the run resumes

Long-running review creates a state problem. The evidence can change while the request is waiting. The customer may cancel the order, an access request may expire or another operator may resolve the case.

Bind the approval to:

  • the workflow run and action ID
  • the proposed arguments
  • the evidence version or retrieval time
  • the reviewer identity
  • the decision time and expiry

Revalidate mutable facts before execution. If the order, permission or policy changed after approval, route the action back through review instead of replaying an old decision against a new state.

A saved approval records a decision against the state the reviewer saw. Before acting, the application has to establish whether that decision still covers the resolved recipients and affected records. The OpenAI Agents SDK documentation describes serialising paused runs and persisting approval decisions, including tool-wide decisions for the rest of a run. Those features preserve execution state; the application still has to enforce the approved business boundary.

Consider an illustrative customer-update workflow. A reviewer approves sending a message to the members of a named customer group. Before execution, someone adds another account to that group.

The tool name and group identifier remain unchanged, but the message now reaches someone the reviewer never saw. Comparing the original argument string alone will miss the change.

The ICML-hosted abstract for research on stale consent in tool-using agents identifies this failure across mutable references, default arguments and other state changes. Its controlled tests support checking the resolved effect, rather than treating an unchanged tool call as sufficient consent. They do not establish a production failure rate for your workflow.

Resolve recipients and affected records before review. Record the approved set alongside the content version, executing identity and expiry. At execution, enforce that boundary again. Where the source system supports a conditional write or transaction, use it to bind the final state check to the action. A separate read followed by an unconditional write leaves a race between the two operations. If the integration cannot enforce the required condition, keep that action outside autonomous execution until the boundary is engineered and tested.

Rehearse changed state and replay before release

Use a sandbox with synthetic records. For each case below, retain the approval packet, execution log and source-system read-back. A normal success case must also complete, so a gate that blocks everything cannot pass.

TestChange after the reviewer sees the requestPassing result
Expanded recipient setAdd a recipient behind the same group identifierNo message goes to the added recipient; any broadened send requires a new review
Changed business objectEdit the approved refund amount or target accountThe old approval cannot authorise the changed action
Revoked authorityRemove the executing identity’s relevant permissionExecution stops; saved approval does not restore access
Expired decisionResume the run after the agreed approval deadlineThe action stays blocked until a fresh decision is recorded
Replayed responseDeliver the same approval response after its action completedNo duplicate side effect; the result reconciles to the original action ID
State change at the write boundaryChange a relevant record between the final read and attempted writeThe source system rejects the stale condition, or the integration refuses autonomous execution

For an unchanged, authorised action inside its approval window, require exactly one intended side effect and verify it in the source system. A returned “approved” status does not satisfy that test.

Feed rejected or edited proposals into the release evidence for the PoC-to-production handover. The owner should be able to show which changed conditions stop execution and which unchanged conditions allow work to finish.

Measure review load against accepted outcomes

Approval rate alone tells you little. A low rate can mean the policy is well targeted, or that dangerous actions bypass review. A high rate can reflect appropriate caution, or an agent that saves no labour because every step waits in a queue.

Track:

  • actions by execution path and consequence band
  • review time and expiry rate
  • rejection and edit reasons
  • permission or evidence failures caught before execution
  • side effects verified after execution
  • reversals, disputes and incidents
  • accepted outcomes per hour of reviewer effort

Connect these measures to the AI agent pilot success criteria. Expansion should depend on accepted outcomes, clean authority boundaries and tested recovery, rather than the number of tool calls completed.

Build the first matrix around one live workflow

Choose a workflow where the agent already needs to read evidence and propose a consequential action. Map every tool call, classify it, then test each route with representative cases and failure conditions.

Model Operator is a Dubai-based AI product studio led by Alexander Chrisostomou. Its AI Initiative Consulting engagement maps consequential workflows and the decisions people must retain. Discovery establishes which tools can enforce the approval boundary and what must be tested before an agent receives write access.

If a live agent can write to customer, financial or operational systems and its approval policy still reads “human review required”, bring that workflow to Model Operator or email alexander@modeloperator.io. The first step is to turn the label into action-level authority the team can test.