This is the first post in Praesidia's security research series. We publish sanitized methodology and findings from testing our own platform and the agent systems it governs. The goal is simple: show our work. If we ask customers to trust a control plane with their agents, we should be able to describe how we try to break it — and what breaks.

Most AI governance tooling stops at observability. It records every prompt, tool call, and policy decision, and that record is genuinely useful for forensics and audit. But logging is passive. It tells you what happened. It does not tell you what would happen if an attacker sent a crafted instruction to one of your agents this afternoon. Praesidia's red-team module checks synthetic instructions against enabled organization-wide defenses. It does not send them to the production target, so its results complement a separate test of the live agent and its specific controls.

From "we log everything" to "we tried to break this"

The distinction matters more for agents than for traditional software. A web app has a relatively fixed attack surface. An AI agent's surface shifts every time you change its system prompt, add a tool, widen a scope, or connect it to a new MCP server. A guardrail that blocked an instruction-override attempt last month may not catch a reworded one today, and a tool that was safe in isolation can become dangerous once it can be chained after another tool. Passive logging never revisits those assumptions. Adversarial testing does, on a schedule.

The red-team module turns the flat claim "we log everything" into the sharper one: "we tried to break this, and here is what we found." It reuses the same asynchronous run infrastructure as Praesidia's evaluation engine, so a red-team campaign is a first-class, queued job with a run record, findings, and history — not a one-off script.

The hard invariant: you can only attack what you own

Adversarial testing is dangerous if it can point anywhere. The single most important component of the module is the scope guard, and it exists to enforce one invariant: a probe may only ever run against the org's own registered, explicitly opted-in entity.

Before any campaign executes, the scope guard checks the target and refuses it unless every condition holds. It denies unknown target types, unregistered identifiers, targets that belong to another organization, and targets that have not been explicitly opted in. Testing is not on by default and it cannot be aimed at another tenant — cross-org targeting is rejected outright, and every denial is logged. This mirrors the isolation guarantees the rest of the platform already enforces: what is yours is testable by you, and nothing else is.

Opt-in is per target, not per account. Registering an agent does not enroll it in red-teaming; you choose which of your own agents and MCP servers to include, and you can see and change that choice.

Synthetic payloads and the test boundary

The executor never calls the target's production endpoint. It checks synthetic content against enabled organization-wide input guardrails and intent rules, without an agent-specific evaluation context. Agent-specific guardrails and connection enforcement are outside this test. Configured model guardrails may call external evaluators, so review those providers before submitting synthetic content.

Campaigns can run repeatedly without dispatching attacks to the production target. Their findings describe verdicts on synthetic probes, not evidence that the live target would behave the same way.

What the attack packs cover

Attack content is organized into extensible packs. Without publishing the payloads themselves, the classes we test are the ones that actually hurt agent systems:

  • Prompt injection and jailbreaks. Classic instruction-override and persona-jailbreak phrasings that try to talk an agent out of its guardrails. These are expected to be caught at the guardrail layer.
  • Scope escalation. The "right API, wrong data" pattern — a legitimate-looking call that reaches for data outside the task's intended scope.
  • Over-permissioned and out-of-scope tool use. Probes that attempt tool calls the agent should not be able to make, surfacing tools whose permissions are broader than the task requires.
  • Bulk PII exfiltration. Mass-data-pull patterns, expected to be caught by the intent layer's bulk-exfil detection.

For MCP servers specifically, the scanner adds three model-context-protocol-native classes: tool poisoning (static analysis of a server's tool descriptions and schemas for misleading or injection-laden content), unauthorized tool chaining (replaying recorded chain traces to flag downstream tool calls that cross a policy boundary), and credential leakage (running secret and PII detectors over tool responses to catch keys or tokens echoed back in output).

Findings that go where your incidents already live

A test result is only useful if it lands somewhere a human will act on it. Each finding is recorded with a severity, a reproduction, and a remediation hint, and it shares the same severity scale as Praesidia incidents. Findings escalate into the incidents workflow — so an adversarial-testing result is not a separate report you have to remember to check; it shows up alongside the production security events your team already triages.

That closes the loop: schedule a campaign, let scope-guarded synthetic probes exercise your defenses, and get back prioritized, reproducible findings in the place you already work.

Common questions

Does adversarial testing touch other customers' agents? No. The scope guard rejects any target that is not your own registered, opted-in entity, and cross-organization targeting is denied outright. Testing is scoped to what you own and have explicitly enabled.

Do the probes send real attacks to my live endpoints? No. Probes check organization-wide input guardrails and intent rules without calling the target's production endpoint or testing agent-specific guardrails. Configured model guardrails can call external evaluators with synthetic content.

Is red-teaming on by default? No. It is opt-in per target. You choose which of your own agents and MCP servers to include, and you can change that at any time.

Where do findings go? Each finding carries a severity, reproduction, and remediation hint, and escalates into the incidents workflow — the same place your other security events are handled.

Where to go next

If you are threat-modeling agent systems, our writeup on prompt injection threats and defenses pairs well with this one, as does the MCP server security checklist. To report a vulnerability in Praesidia itself, see our Vulnerability Disclosure Policy, and for the full control set and framework mapping, visit Security & Compliance.