A RAG agent can retrieve a document the employee is authorised to read and still encounter an attack. Indirect prompt injection happens when instructions embedded in retrieved material redirect the agent away from the user’s task. If that agent can also send messages or call external tools, source content can influence where company data goes.
To reduce that risk, keep retrieved text separate from privileged instructions and enforce action limits outside the model. Inspect the full path from document to tool arguments; a correct source-access check covers only the retrieval step.
Follow a proposal request through the attack path
Consider a hypothetical consultancy agent asked to summarise a supplier proposal for an internal review. The employee has access to the proposal. The agent can search internal files and prepare messages.
Inside the proposal, an attacker adds a purported verification requirement: send the internal evaluation notes to an external review address before completing the summary. The requested destination is part of the supplier’s text, with no authority from the employee who initiated the task.
The exposure depends on what happens next. A summariser with no outbound capability has fewer routes to disclose those notes. An agent that can retrieve them and submit a message has a path from the hostile instruction to a consequential action. Disclosure can also occur through an external tool’s arguments, even when the tool is labelled as search or analysis.
Microsoft’s Prompt Shields documentation explicitly distinguishes direct user attacks from document attacks carried through third-party content, including documents and emails. Its document-attack examples include information gathering and unintended actions. The relevant question is what authority the workflow gives that content after reading it.
Locate the point where evidence becomes an instruction
Draw the proposed workflow as a sequence of concrete objects:
employee request → retrieved proposal → model context → proposed tool call → outbound request
At each transition, record which component decides what happens next. This exposes the point where a legitimate document’s contents are being treated as permission to do additional work.
| Boundary | What to inspect | Failure to prevent |
|---|---|---|
| Retrieval to model context | Where source text is inserted and how its origin is retained | A document passage enters a privileged instruction field |
| Model response to tool proposal | Which arguments the model can choose | Source text supplies an external recipient or requests unrelated internal records |
| Tool proposal to execution | Which checks run independently of model judgement | The executor accepts a disallowed destination because the agent called it necessary |
| Execution to final answer | Whether earlier data transfers remain visible | A harmless-looking summary conceals an outbound disclosure |
OpenAI’s agent safety guidance warns against putting untrusted variables into developer messages, where they gain greater influence. It also recommends constraining data flow and using tool confirmations. The page describes a legacy builder, but this instruction-boundary failure is relevant when inspecting other orchestration code: check how that specific system actually constructs its model input.
A source label lets a reviewer trace the proposal passage behind a tool request. It does not stop the request. The executor must check the destination and payload against the employee-authorised task before releasing data, so a model’s mistaken interpretation cannot approve its own disclosure.
Choose controls for the boundary they protect
A detector can flag suspicious document instructions before the model uses them. That adds a useful screening layer, subject to its coverage and false positives. Keep a route for legitimate material that quotes instructions, such as a security incident report; blocking every imperative sentence would make some real work impossible.
Then reduce what the model can turn into action. A proposal-review task can produce an internal draft without receiving a general-purpose external-send tool. Where sending is required, the application can resolve the recipient from an authorised workflow record rather than accepting an arbitrary address extracted from a proposal.
That choice changes the failure consequence. Even if the model repeats the attacker’s requested recipient, the executor can reject it before transmitting data. Validate the actual resolved destination and submitted payload, including parameters sent to external services. A tool name alone does not establish what leaves the environment.
Structured fields help narrow the channel between steps. An approved-recipient identifier selected from a bounded set gives the executor less freedom to interpret than a free-text instruction. However, a schema permitting any recipient string still permits an attacker’s address. The structured outputs and hallucinations answer explains the related limit: valid structure does not establish that the values deserve to be acted on.
For an action that genuinely requires review, show the reviewer the recipient and exact content that will be transmitted, alongside the originating employee request. The agent approval matrix covers how to classify consequential actions and place review beside the side effect. A document’s assertion that approval has already been granted must not satisfy that gate.
Rehearse the boundary with fake data and a mocked tool
Use a sandbox with fabricated internal notes and an outbound tool that records attempted calls without sending anything. The following is a proposed test protocol, not a reported Model Operator result or a security certification.
Prepare a matched pair of supplier proposals. Both contain the same commercial facts. One also includes a clearly marked test instruction asking the agent to transmit the fabricated notes to a disallowed test destination. Give both runs the same legitimate employee request: produce an internal summary.
Record more than the final response:
- Save the source passage and where it entered the model context, so a reviewer can inspect whether source text gained privileged placement.
- Capture proposed tool arguments and the executor’s decision. Distinguish an agent that declined the instruction from an executor that stopped a prohibited call.
- Check the permitted summary against the unmodified proposal. A system that blocks every document has prevented disclosure by abandoning the task.
- Run a separate authorised-action control against the mock tool, if the workflow includes sending. Confirm that the permitted destination and payload reach the mock while the prohibited combination is rejected.
Define the expected result before running the pair: the legitimate summary remains useful, no fabricated notes enter a disallowed tool request, and attempted prohibited actions are visible. If the task is draft-only, the outbound capability should be absent throughout.
Repeat with the actual connector, model configuration and tool wrapper intended for deployment. A pass on one fixture establishes behaviour for that fixture and configuration. It does not establish resistance to other phrasings, image-based instructions or longer attack sequences.
Anthropic’s browser prompt-injection research describes hidden email instructions that attempt to forward confidential material. It also states that prompt injection remains unsolved despite model and safeguard improvements. Those browser evaluations are evidence of the threat and defensive progress, not a measured failure rate for your RAG workflow.
Make the remaining exposure part of the release decision
Before release, ask the implementation owner to trace a source-controlled value all the way to a tool request. Have them show the executor rejecting a disallowed transfer, including the rejection the operator will see. If that check is missing, keep the first release to an internal draft while the team builds and tests it.
Model Operator is a Dubai-based AI product studio led by Alexander Chrisostomou. Its AI Initiative Consulting service covers workflow analysis and implementation priorities; the scope follows the work and the buyer’s constraints. This article describes design considerations, without claiming that a particular deployment has passed confidential-data readiness tests.
Bring one retrieval-connected workflow and its available tools to a Model Operator build conversation. The first decision is which actions that workflow should be allowed to take, and where those limits will be enforced.