Computer-Use Agents: Security Risks of AI That Clicks

Computer-use agents are risky because they operate a real desktop the way a person does: they read the screen, move the mouse and type, and so they inherit every logged-in session, saved credential and open application on that machine. The main threats are prompt injection from on-screen content, actions with too much authority, and irreversible steps taken without a human seeing them. The baseline defence is an isolated virtual machine with minimal privileges, an allow-list of applications and destinations, and mandatory human confirmation for anything that spends money, sends data, deletes data or signs in.

What makes computer use different from tool use

A typical tool-using agent calls defined functions: search_tickets(query), create_invoice(customer, amount). Each function has a schema, and you can authorise each one separately. That is the model covered in tool-use safety and sandboxing.

A computer-use agent has one very general tool: the screen. It takes a screenshot, decides where to click or what to type, and repeats. That changes three things:

No per-action schema. "Click at (812, 440)" carries no meaning a policy engine can check. The meaning depends on what is on screen at that moment.
Ambient authority. If the browser in the VM is logged into email, the agent can send email. If a terminal is open with cloud credentials, it can use them. Permissions come from the environment, not from a grant.
Untrusted input everywhere. Every pixel is input. A web page, a PDF, a calendar invite or a pop-up can contain text written to steer the agent.

Browser-only agents share part of this risk profile. Browser-use agent security covers them in detail. Full computer use widens the surface to native applications, file systems and terminals.

The main risks

Prompt injection from the screen

Anthropic's documentation for its computer use tool warns that the model may follow instructions found in content even when they conflict with the user's instructions. It recommends isolating the model from sensitive data and actions to reduce prompt injection risk (Anthropic computer use tool docs). The same documentation describes classifiers that scan screenshots for likely injections. That is a useful layer, but it does not replace isolation.

Practical injection vectors include hidden or small text on a web page, instructions in a document the agent is asked to summarise, a crafted error dialog, and an email in an inbox the agent is triaging.

Inherited sessions and credentials

The most damaging computer-use incidents do not need a clever exploit. The agent simply runs on a machine where someone is already logged in. A password manager that autofills, a browser with a live session for the admin console, or SSH keys in the home directory all become reachable through a few clicks.

Irreversible actions

Clicking "Buy", "Send", "Delete", "Approve" or "Merge" takes one step. Undoing it may not be possible. Agents also misread interfaces: a misplaced click on a confirmation dialog is a reliability failure with security consequences.

Escape from the sandbox

If the agent can open a terminal, it can run code. Code execution inside a poorly configured VM can lead to the host, the network, or cloud metadata endpoints. Agent sandbox escape covers the container and VM hardening side.

Data exfiltration through the UI

An agent that can type into a web form can paste data into it. Exfiltration does not need an API call. It can be a search query, a form submission or a URL with a query string.

Controls that work

Risk	Control	Notes
Inherited sessions	Dedicated VM or container per task, created clean and destroyed after use	No personal profile, no password manager, no saved cookies
Ambient credentials	Inject only the task-specific credentials, short-lived	Never mount the developer's cloud or SSH config
Screen-borne injection	Content classifiers on screenshots and text, plus the controls below	Assume some injections will get through
Network exfiltration	Egress allow-list by domain	Block cloud metadata endpoints and internal ranges
Irreversible steps	Human confirmation before purchase, send, delete, publish, login	The agent pauses, a human sees the screen state
Scope drift	Allow-list of applications and URLs per task	Deny the terminal unless the task needs it
Runaway sessions	Step limits, time limits and a stop control	Kill the VM, not only the agent loop
No accountability	Per-step screenshots and action logs	Store them where the agent cannot edit them

Two points matter most. First, isolation is the primary control. Classifiers and model training reduce injection success. They do not remove it, and a computer-use agent with access to a privileged session will eventually be steered. Second, confirmations must show the real screen. The human should see the page the agent is about to act on, not a text summary written by the agent, which an injection could also shape.

A deployment checklist

The agent runs in a dedicated VM or container, created per task or per session.
The environment has no personal accounts, password manager, browser profile or developer credentials.
Task credentials are short-lived and scoped to the target application.
Network egress is limited to an allow-list of domains, and metadata endpoints are blocked.
Applications the agent may open are allow-listed, and the terminal is off unless needed.
Purchases, outbound messages, deletions, publishing, approvals and logins need human confirmation.
Step and time limits are set, and an operator can stop the environment at once.
Screenshots and actions are logged per step to storage the agent cannot write to.
The agent's identity is registered in your inventory with an owner and a purpose.
Your kill switch procedure covers computer-use sessions.

Where computer use fits and where it does not

Computer use is the right tool when no API exists: legacy desktop software, internal portals without integrations, or one-off workflows across several UIs. It is the wrong tool when an API or an MCP server already exposes the same action, because a defined tool call can be authorised, constrained and logged far more precisely than a click. A sensible policy: use structured tools where they exist, and allow computer use only for named workflows with a named owner. The MCP security guide covers how to authorize and log those structured tool calls.

How Praesidia fits

Praesidia does not run the VM or drive the screen. It governs the agent around it. Discover: register computer-use agents in the agent inventory with an owner, so they are not invisible automation. Secure: when a computer-use agent also calls MCP tools or models through Praesidia, guardrails check inputs and tool-call arguments for injection patterns and sensitive data with block, redact or warn dispositions, MCP connections can deny tools or constrain argument values, and step-up rules can hold an MCP tool call for human approval. Prove: tool calls and guardrail events that pass through Praesidia are written to a signed, hash-chained audit trail.

For the wider picture, see the AI agent security guide and the OWASP Agentic AI Top 10 guide.