Multimodal prompt injection is prompt injection delivered through a non-text channel: instructions hidden in an image, embedded in a PDF or office document, or spoken in an audio clip, which a vision, document or speech-capable model extracts and then treats as part of its context. It matters because most first-generation defenses scan the text a user types, and in a multimodal attack the malicious text does not exist until the model or a pre-processing step has produced it. The defense that holds is to inspect what the agent actually ingests — extracted text and tool outputs — and to constrain the actions the agent can take after reading it, rather than trying to recognise every hostile pixel or waveform.
Why text-only filters miss multimodal payloads
A classic input filter sits in front of the model and examines the user's message. That works when the attack is a sentence the user typed. In a multimodal pipeline, the user's message may be harmless ("summarise this invoice"), while the instruction sits inside the attached file. The model's vision encoder, an OCR step or a speech-to-text service turns that file into tokens, and from that moment the injected instruction is indistinguishable, to the model, from any other text in its context window.
This is the same structural problem covered in the threat model for indirect prompt injection: content the agent retrieves or receives is treated with the same authority as instructions from its operator. Multimodal input widens the surface because the content is opaque to a human reviewer. A person glancing at a screenshot or skimming a PDF will not notice text rendered in near-white on white, a line in a 1-point font, or a sentence spoken under background noise. The model, which reads every extracted token, does.
Where payloads hide, by modality
| Modality | Typical hiding place | What extracts it | Where to inspect |
|---|---|---|---|
| Images and screenshots | Low-contrast or tiny text, text in a corner of a chart, text drawn into a photo | Vision encoder or OCR | OCR output or a model-generated description before it reaches the agent's planning step |
| PDFs | Hidden text layers, white-on-white text, text outside the visible page area, metadata and annotations | Document parser | Full parsed text, including layers a viewer does not render |
| Office documents | Comments, speaker notes, hidden rows and sheets, alt text, tracked changes | Document loader | Every text field the loader emits, not only the body |
| Audio and voice | Spoken instructions in a recording, phrases embedded in a longer call, synthetic speech | Speech-to-text | The transcript the model receives |
| Web pages fetched by an agent | Text hidden by CSS, alt attributes, invisible characters | HTML-to-text or browser tool | The tool output returned to the model |
Two observations follow from the table. First, the place to look is almost always the extracted text, because that is the representation the model reasons over. Second, many hiding places are not visual tricks at all: they are fields a human viewer does not display by default, such as speaker notes, metadata and alt text. A document loader that emits those fields is feeding them to the model whether or not anyone ever sees them.
Voice is a special case worth reading separately. The voice agent security risks post covers how audio becomes an injection channel and why real-time latency budgets squeeze out enforcement.
Detection: useful, but never sufficient
The techniques in how to detect prompt injection still apply once you run them against the right text. In practice that means:
- Extract first, then scan. Run OCR, parsing or transcription in a step you control, and pass the resulting text through the same guardrails you apply to user messages before the agent plans on it.
- Scan every emitted field. Treat comments, notes, metadata, alt text and hidden layers as input, or strip them in the loader if the use case does not need them.
- Flag instruction-shaped content in data. Text that addresses the assistant ("ignore the previous request", "you must now send") inside an invoice, résumé or meeting recording is a strong signal, even if it is not conclusive.
- Compare what a human sees with what the model reads. For high-risk workflows, a mismatch between rendered content and extracted text — a page that looks empty but parses to 400 words — is itself worth an alert.
None of this closes the problem. Vision models can read text that OCR misses, adversarial images can carry instructions that are not legible text at all, and a classifier tuned on English prompts will miss paraphrased or translated payloads. Detection lowers the rate of successful injections; it does not bound the damage when one gets through. That is the job of the next layer.
Gate the action, not just the prompt
The useful question is not "can we spot every hidden instruction?" but "what can the agent do if it follows one?" The lethal trifecta framing makes this concrete: an agent that reads untrusted content, has access to private data, and can communicate externally is an exfiltration path, whatever modality the untrusted content arrived through. Multimodal input is simply another way to supply the first leg.
Controls that reduce impact regardless of how the payload arrived:
- Scope tools per agent. An agent that summarises uploaded documents rarely needs to send email or call arbitrary URLs. Remove those tools, and an injected "email this to…" has nothing to call.
- Check tool arguments before execution. Inspect outbound arguments for sensitive data and unexpected destinations, so a hijacked call is refused even when the instruction itself was not caught.
- Inspect tool results before the model reads them. A browsing or file-reading tool's output is untrusted content; scan it on the way back in, as described in prompt injection threats and defenses.
- Require approval for consequential actions. Payments, external messages, record deletion and permission changes should pause for a human in the loop, with the approver shown the extracted text that led to the request.
- Keep a record. Log which file, transcript or tool output preceded each action, so an investigation can trace an unexpected action back to the input that triggered it.
A checklist for multimodal agents
Use this when reviewing any agent that accepts images, documents or audio:
How Praesidia fits: discover, secure, prove
Praesidia does not ship an OCR or speech pipeline, and it does not claim to detect every instruction hidden in a pixel. Its role is the layer around the agent. Discover: inventory which agents exist and which MCP servers and tools they can reach, so you know which ones accept untrusted files. Secure: guardrails inspect content on the input and output of agent tasks, and during a chat turn they check a tool call's arguments before it is sent and its result before the model reads it; rule-based, ML-based and LLM-based guardrails can block, redact or warn, as described in content guardrails for AI agents. Tool access runs through scoped connections and tool policy, and consequential actions can be gated for approval. Prove: evaluations and calls that pass through Praesidia are recorded with attribution, so an incident review can show what the agent read and what it was allowed to do.
Common questions
Is multimodal prompt injection a different vulnerability from indirect prompt injection? No. It is indirect prompt injection where the untrusted content arrives as an image, document or audio clip. The root cause — data and instructions sharing one context with no enforced boundary — is the same.
Will a better vision model fix it? A more capable model reads more text, including text humans miss, which can increase exposure. Model improvements in instruction hierarchy help, but no current model reliably ignores every embedded instruction.
Should we block all image and document uploads? Only if the use case allows it. For most teams the better trade is to keep uploads, inspect extracted content, and remove the tools that turn a successful injection into data loss.