AI Agent Troubleshooting: Diagnose the Failure Layer First

Troubleshoot an AI agent by locating the first layer where the run diverged: context, judgement, tool execution, authority, handoff or business outcome. Inspect one complete trace, compare each layer with the expected path, fix the earliest confirmed defect and replay the same case before changing anything else.

This order prevents an expensive debugging habit: rewriting the prompt when the model received stale evidence, an API timed out or the workflow recorded output as completion. A trace narrows the search. A failure-layer diagnosis turns that trace into an engineering and operating decision.

Start with one failed run and its expected outcome

Collect a single case that a reviewer can label clearly. Record the request, the authorised evidence available at run time, the action expected, the permitted tools, the approval rule and the downstream state that would count as success.

The downstream state matters because an agent can finish its conversation without finishing the work. Anthropic’s agent evaluation guide distinguishes the transcript from the outcome: an agent may state that a flight was booked while the reservation database shows no booking. That gap appears in less dramatic workflows too. A sales agent drafts a correct follow-up but never writes it to the approved queue. A reporting agent produces a polished deck from last month’s numbers. Both runs look complete at the interface and fail in the operation.

Give the case a stable run ID and preserve enough evidence to reconstruct it. OpenAI’s Agents SDK tracing documentation captures model generations, tool calls, guardrails and handoffs as spans inside an end-to-end trace. The same documentation warns that generation and function spans can contain sensitive inputs and outputs. Trace coverage therefore needs the same access, retention and redaction discipline as the workflow it records.

Layer 1: did the agent receive the right context?

Check the evidence before judging the answer. Identify every retrieved item, its version, timestamp, permission basis and ranking position. Then compare that set with the sources an informed operator would have needed.

Context failures include:

• the authoritative document was absent from the eligible corpus;
• an obsolete policy outranked its replacement;
• retrieval returned a related passage without the decisive exception;
• an identity or metadata filter removed evidence the user was allowed to use;
• the workflow injected a summary after its underlying record had changed.

A prompt edit cannot restore a missing source. Repair the connector, source hierarchy, freshness rule, metadata or retrieval configuration, then replay the trace with the corrected evidence set.

For company workflows, record the distinction between raw evidence and accepted company truth. Two documents can both be retrievable while disagreeing on price, policy or ownership. The agent needs a defined precedence or conflict route; access to more text leaves the contradiction intact.

Layer 2: did the model apply sound judgement to that context?

Once the evidence is complete, inspect how the agent interpreted it. Look for instruction violations, unsupported inference, weak planning, premature certainty and failure to abstain when the evidence did not resolve the request.

This is the layer where prompt, model and reasoning changes belong. The repair depends on the observed defect:

• tighten the instruction when the required rule was ambiguous;
• add a worked example when a stable edge case keeps being misclassified;
• require citation to the decisive evidence when unsupported synthesis is the problem;
• introduce a planning or review step when the task contains dependent decisions;
• change the model only after representative cases show a capability constraint.

Evaluate the process alongside the final prose. OpenAI’s trace grading guide applies structured labels to the end-to-end record so teams can identify mistakes in decisions and tool calls, then compare changes across a wider set of traces. A targeted orchestration change can then face a regression test, which carries more weight than one improved screenshot.

Layer 3: did each tool execute the intended operation?

Separate tool selection, parameters, technical success and result interpretation. These are different failure modes with different owners.

Microsoft Foundry’s agent evaluator reference reflects this separation. Its process evaluators cover tool selection, input accuracy, output utilisation and call success. A single “tool failed” label hides whether the agent chose the wrong integration, supplied the wrong account ID, received a timeout or ignored a valid response.

Inspect the raw request and response for every consequential call. Verify schema validation, authentication, retry behaviour, timeout handling and idempotency. A timed-out write requires special care: the absence of a response does not prove the destination rejected the action. Blind retry can duplicate a refund, message or record.

The fix may sit in deterministic code around the model. Stable transport, validation, deduplication and write-back rules belong in conventional software where pass or fail can be verified directly. Keep model judgement for the variable part of the task.

Layer 4: was the action inside the agent’s authority?

A technically successful call can still violate the workflow. Compare the action with the user’s identity, the agent’s delegated scope, the approval threshold and the current policy state.

Check four facts:

1. Which principal authorised the run?
2. Which resources and actions were permitted for this case?
3. Which condition required human approval?
4. Did access remain valid when the action executed?

Treat approval as a state transition with evidence, timestamp and reviewer identity. A chat message saying “looks good” has limited audit value when it cannot be tied to the exact artefact or action parameters approved.

For sensitive workflows, revocation and stale-permission behaviour need explicit tests. Model Operator’s implementation direction uses permission-aware evidence, candidate and accepted truth, approval-gated actions and audit records because reliable answers and authorised actions are separate contracts.

Layer 5: did the handoff preserve state and ownership?

Multi-agent and human-in-the-loop workflows fail between components. The sending agent may complete its part while the receiving system gets an incomplete payload, an ambiguous status or no owner.

Inspect the handoff artefact directly; both components’ summaries can hide missing state. It should carry the operation ID, current state, evidence references, unresolved questions, authority boundary, acceptance criteria and named next owner. Version it when later work changes the artefact.

OpenAI tracing includes handoff spans, which helps reveal where control moved. The commercial issue lies beyond visibility: each transfer adds latency, token use and another opportunity to lose context. Collapse a handoff when the receiving specialist does not improve accepted outcomes enough to justify that coordination cost. The multi-agent coordination tax teardown provides a fuller test for that architecture decision.

Layer 6: did the run create the required business outcome?

Close the diagnosis against the system of record and the reviewer decision. Confirm that the requested state exists, the artefact was accepted, restricted data stayed protected and the economic result can be measured.

Microsoft Foundry separates system evaluation from process evaluation: task completion and adherence assess the end-to-end result, while tool evaluators inspect execution steps. Its broader observability guidance combines evaluation, production monitoring and distributed tracing across model calls, tools and service dependencies.

Use that separation in the operating scorecard:

Evidence	Question answered	Example measure
Trace	Where did execution diverge?	tool-input failure rate
Evaluation	Does the corrected system handle representative cases?	regression pass rate
Human review	Is the output usable within the workflow?	first-pass acceptance rate
System of record	Did the intended state change occur?	verified completion rate
Commercial ledger	Was the result worth the full cost?	cost per accepted outcome

An agent that passes process checks and produces no accepted value still needs product work. Review time, rework, recovery and downstream delay belong in the result, alongside model and tool spend. The AI agent unit economics guide shows how to join those costs at run level.

Turn each useful failure into a regression test

After fixing the first broken layer, replay the original case under the same acceptance rule. Then run adjacent cases that probe the boundary: an older source, a denied permission, a malformed tool response, a duplicate event and an approval rejection.

Promote the failure into a durable evaluation when it represents a plausible recurrence, expensive exception or safety risk. Store:

Teams using Microsoft’s platform can apply that replay loop through Copilot Studio agent evaluation, which supports repeatable test sets, case-level result inspection and run comparison.

• the initial conditions and authorised evidence;
• the expected outcome and prohibited outcomes;
• the trace fields needed for diagnosis;
• a deterministic check where the state can be verified;
• a rubric for judgement that cannot be reduced to code;
• the owner and release threshold.

Anthropic recommends combining code-based, model-based and human graders according to the task. Deterministic checks suit database state, schemas and policy conditions. Model graders handle nuanced quality when calibrated against expert judgement. Human review remains the reference for subjective or consequential outcomes.

The resulting suite supports two jobs. Capability cases show where the system still struggles. Regression cases protect behaviour the agent has already demonstrated. Keep those purposes separate so a rising average cannot conceal a critical workflow that stopped working.

Use a concise incident record

A useful agent incident record fits on one page:

1. Case and impact: affected workflow, user, time, money or risk.
2. Expected outcome: accepted artefact or verified state change.
3. Observed outcome: what the interface and system of record each showed.
4. First failed layer: context, judgement, tool, authority, handoff or outcome.
5. Evidence: trace ID, source versions, tool response and approval record.
6. Correction: the smallest change that addresses the confirmed cause.
7. Replay result: original case plus adjacent boundary tests.
8. Prevention: regression test, monitor, owner and release gate.

This format gives product, engineering and operations one causal account. It also stops a model-quality debate from absorbing failures owned by source governance, integration code or workflow design.

Make the failure layer visible before scaling the workflow

The six-layer method creates a short route from complaint to controlled change:

context → judgement → tool → authority → handoff → outcome

Start at the left, fix the first confirmed divergence and verify the downstream result. The ordering preserves causal discipline because later symptoms can flow from an earlier defect.

If your team has a live agent workflow that produces output but resists diagnosis, Model Operator can map the evidence, truth, approval, run and outcome contracts around it. Bring one failed case, its trace and the state that should have changed to alexander@modeloperator.io. The work starts with a reproducible diagnosis and finishes when the corrected run creates an accepted outcome.