This is the first post in Praesidia's security research series. We publish sanitized methodology and findings from testing our own platform and the agent systems it governs. The goal is simple: show our work. If we ask customers to trust a control plane with their agents, we should be able to describe how we try to break it — and what breaks.

Most AI governance tooling stops at observability. It records every prompt, tool call, and policy decision, and that record is genuinely useful for forensics and audit. But logging is passive. It tells you what happened. It does not tell you what would happen if an attacker sent a crafted instruction to one of your agents this afternoon. Answering that question requires actually sending the crafted instruction — safely, against the right target, and in a way that never touches anyone else's data. That is what continuous adversarial testing does, and it is what shipped in the Praesidia red-team module.

From "we log everything" to "we tried to break this"

The distinction matters more for agents than for traditional software. A web app has a relatively fixed attack surface. An AI agent's surface shifts every time you change its system prompt, add a tool, widen a scope, or connect it to a new MCP server. A guardrail that blocked an instruction-override attempt last month may not catch a reworded one today, and a tool that was safe in isolation can become dangerous once it can be chained after another tool. Passive logging never revisits those assumptions. Adversarial testing does, on a schedule.

The red-team module turns the flat claim "we log everything" into the sharper one: "we tried to break this, and here is what we found." It reuses the same asynchronous run infrastructure as Praesidia's evaluation engine, so a red-team campaign is a first-class, queued job with a run record, findings, and history — not a one-off script.

The hard invariant: you can only attack what you own

Adversarial testing is dangerous if it can point anywhere. The single most important component of the module is the scope guard, and it exists to enforce one invariant: a probe may only ever run against the org's own registered, explicitly opted-in entity.

Before any campaign executes, the scope guard checks the target and refuses it unless every condition holds. It denies unknown target types, unregistered identifiers, targets that belong to another organization, and targets that have not been explicitly opted in. Testing is not on by default and it cannot be aimed at another tenant — cross-org targeting is rejected outright, and every denial is logged. This mirrors the isolation guarantees the rest of the platform already enforces: what is yours is testable by you, and nothing else is.

Opt-in is per target, not per account. Registering an agent does not enroll it in red-teaming; you choose which of your own agents and MCP servers to include, and you can see and change that choice.

Synthetic payloads, no external egress

A second design choice keeps testing safe even against a target you legitimately own: the probe executor performs no external egress and never calls a target's real endpoint. It pushes a synthetic payload through the same governance layers a real attacker's traffic would hit — the guardrail content scan and the intent-detection layer — and evaluates whether those defensive layers would have caught it. The test measures your defenses, not your production system's live behavior, and it does so without generating outbound traffic or side effects.

That is why continuous testing can run frequently without operational risk. A campaign does not hammer an endpoint or exfiltrate anything; it replays a known-bad input against your own rules and records which layer, if any, stops it.

What the attack packs cover

Attack content is organized into extensible packs. Without publishing the payloads themselves, the classes we test are the ones that actually hurt agent systems:

  • Prompt injection and jailbreaks. Classic instruction-override and persona-jailbreak phrasings that try to talk an agent out of its guardrails. These are expected to be caught at the guardrail layer.
  • Scope escalation. The "right API, wrong data" pattern — a legitimate-looking call that reaches for data outside the task's intended scope.
  • Over-permissioned and out-of-scope tool use. Probes that attempt tool calls the agent should not be able to make, surfacing tools whose permissions are broader than the task requires.
  • Bulk PII exfiltration. Mass-data-pull patterns, expected to be caught by the intent layer's bulk-exfil detection.

For MCP servers specifically, the scanner adds three model-context-protocol-native classes: tool poisoning (static analysis of a server's tool descriptions and schemas for misleading or injection-laden content), unauthorized tool chaining (replaying recorded chain traces to flag downstream tool calls that cross a policy boundary), and credential leakage (running secret and PII detectors over tool responses to catch keys or tokens echoed back in output).

Findings that go where your incidents already live

A test result is only useful if it lands somewhere a human will act on it. Each finding is recorded with a severity, a reproduction, and a remediation hint, and it shares the same severity scale as Praesidia incidents. Findings escalate into the incidents workflow — so an adversarial-testing result is not a separate report you have to remember to check; it shows up alongside the production security events your team already triages.

That closes the loop: schedule a campaign, let scope-guarded synthetic probes exercise your defenses, and get back prioritized, reproducible findings in the place you already work.

Common questions

Does adversarial testing touch other customers' agents? No. The scope guard rejects any target that is not your own registered, opted-in entity, and cross-organization targeting is denied outright. Testing is scoped to what you own and have explicitly enabled.

Do the probes send real attacks to my live endpoints? No. The probe executor performs no external egress and never calls a target's real endpoint. It pushes a synthetic payload through your own guardrail and intent layers and evaluates whether they would have caught it.

Is red-teaming on by default? No. It is opt-in per target. You choose which of your own agents and MCP servers to include, and you can change that at any time.

Where do findings go? Each finding carries a severity, reproduction, and remediation hint, and escalates into the incidents workflow — the same place your other security events are handled.

Where to go next

If you are threat-modeling agent systems, our writeup on prompt injection threats and defenses pairs well with this one, as does the MCP server security checklist. To report a vulnerability in Praesidia itself, see our Vulnerability Disclosure Policy, and for the full control set and framework mapping, visit Security & Compliance.