Human-agent trust exploitation, listed as ASI09 in the OWASP Top 10 for Agentic Applications, is the class of failures where a person approves, executes or believes something harmful because an AI agent presented it convincingly. The agent might be compromised by prompt injection, poisoned by bad data, or just wrong. The human is the last control, and the attack works because that control has stopped checking. The defence is not "more human-in-the-loop". It is fewer approvals, placed on the actions that matter, with evidence the agent cannot shape.
What ASI09 describes
Most agentic risks are about what the agent does on its own: goal hijacking, tool misuse, privilege abuse. ASI09 is about what the agent gets a human to do. Typical patterns:
- Rubber-stamped approvals. The agent requests approval for a payment, a deletion or a permission change. The approver clicks "approve" because the request looks routine.
- Persuasive misinformation. A support or analyst agent states a wrong conclusion with confidence, and a human acts on it without verifying.
- Delegated social engineering. An injected instruction makes the agent ask the user for credentials, an MFA code, or to run a command "to complete the task".
- Authority laundering. An attacker's request arrives through the agent, so it carries the agent's perceived legitimacy instead of looking like an external message.
In each case the technical controls may work as designed. The approval gate fires. The human sees the request. The failure is in the human decision, and attackers can plan for that.
Why approval fatigue makes it worse
Human-in-the-loop approvals are the standard answer to excessive agency. Applied badly, they create the conditions for ASI09.
The mechanism is familiar from alert fatigue and MFA push bombing. When a person sees many similar prompts, nearly all harmless, they learn that approving is the right answer. Review time drops. Reading stops. The one malicious request in a hundred gets the same two-second click as the 99 others.
Three design mistakes cause most of this:
- Gating everything. If every file write or API call needs approval, people approve without reading.
- Summary-only requests. The approver sees the agent's description of the action ("Refund duplicate charge for customer 4411"), not the action itself. An injected or mistaken agent writes a reassuring summary.
- Self-approval paths. The person who started the task also approves its risky step, under time pressure, with a stake in finishing it.
How to design approvals that resist trust exploitation
| Design choice | Weak version | Strong version |
|---|---|---|
| What gets gated | Every tool call | Only actions that are hard to reverse or have a large blast radius |
| What the approver sees | The agent's summary | The exact action, its arguments, and the source data, next to the summary |
| Who approves | The requesting user | A separate owner for high-risk tiers; two people for the highest |
| Default on timeout | Approve | Deny |
| Context | None | Why this was flagged: which rule, which risk signal |
| Feedback | None | Rejections and overrides are logged and reviewed |
Tier the risk
Sort agent actions into tiers by reversibility and blast radius. Low-tier actions run without prompts and are logged. Medium-tier actions may need a lightweight confirmation. High-tier actions, such as payments above a threshold, permission grants, bulk deletions and external sends of sensitive data, need a separate approver who sees the evidence. Approval workflows and break-glass access covers how to structure these tiers and emergency overrides.
Show evidence the agent did not write
The approval screen should show the raw inputs: the tool name and arguments, the target record, the diff, and the source document or message that triggered the action. The agent's explanation can appear too, clearly labelled as the agent's claim. If an approver can only see what the agent wrote, they are trusting the agent, which is the exact weakness ASI09 describes.
Make declining easy and normal
If rejecting a request means a long form or an awkward conversation, people will approve. One-click deny, with an optional reason, should be as easy as approve.
Never let the agent ask for secrets
An agent should never ask a user for passwords, MFA codes or session tokens, and users should be told this. Treat any such request as a security event. That turns a social-engineering attempt into a detection.
Measure the approval process
You cannot see rubber-stamping by looking at one decision. You can see it in aggregate:
- Approval rate per tier. A high-risk tier approved almost every time either has too many benign requests in it or is not being reviewed.
- Time to decision. Consistent decisions in a few seconds on high-risk items suggest nobody is reading.
- Volume per approver. Too many requests per person per day predicts fatigue.
- Post-hoc reversals. Approved actions that were later undone, refunded or reported are your real error rate.
Review these numbers on a schedule. When a tier shows fatigue signals, move low-risk actions out of it rather than asking people to try harder.
ASI09 checklist
How ASI09 relates to other frameworks
ASI09 sits at the end of most attack chains: injection or poisoning compromises the agent, and trust exploitation turns that into a human action. MITRE ATLAS covers the adversary techniques that lead there. For how the two frameworks map, see MITRE ATLAS vs the OWASP Agentic AI Top 10, and for the full list, the OWASP Agentic AI Top 10 guide.
How Praesidia fits
Praesidia's role in ASI09 is to make approvals rarer and more meaningful. Discover: the agent inventory shows which agents exist and which tools they can reach, so you know where approvals belong. Secure: guardrails apply block, redact or warn dispositions automatically, and step-up rules can hold a specific class of tool call for human approval rather than sending every action to a person. MCP connections can deny tools or constrain argument values, which removes whole categories of request from the approval queue. Prove: guardrail events and tool calls are recorded in a signed, hash-chained audit trail, so a reviewer can check later what was requested, what was decided, and what ran.