Test every agent release against your own cases, your own defenses and your own bar before it ships.

Evaluations and runs

An evaluation scores an agent's answers against criteria you write, either as pass or fail or on a scale, with a passing threshold you set. Each run tests one active agent on its current prompt or on one specific prompt version and records the run's pass rate, mean score, response latency and cost. A completed run opens case by case, and runs can be compared side by side.

Datasets and regressions

A dataset is a named set of test cases — an input, the output you expect and your own tags — added by hand or imported from your agents' recent completed tasks, with personal data detected in an imported input redacted before the case is stored. Two runs over the same dataset are compared case by case: which cases regressed, which improved and which were unchanged, alongside each run's pass rate.

Red-team campaigns and attack packs

Red-team campaigns send attack probes at your own registered agents, connections and MCP servers, on demand or on a schedule, and only at targets your organization has opted in — any other target is refused. The attack library covers prompt injection and jailbreaks, scope escalation, out-of-scope tool use, bulk PII exfiltration, RAG poisoning, memory poisoning, malicious skills, cross-agent escalation, lateral movement, privilege escalation and policy bypass. MCP servers are also checked for poisoned tool descriptions, unauthorized tool chaining and credential leakage. Red-team campaigns are a plan feature; see pricing.

What a red-team run tells you

Probes test your defenses, not your production systems: each one is checked against the guardrails and rules your organization has configured today, without calling the target itself, and records whether they stopped it. Each run reports how many attacks were caught, how many got through and the worst severity found, and an attack that gets through at high or critical severity is escalated to a security incident linked from its finding.

Release gates

Each AI system can carry release-gate rules on an evaluation run's pass rate, mean score, median latency, 95th-percentile latency or total cost, each marked blocking or advisory. Your deployment pipeline asks for a verdict with an organization API key and gets pass, fail when a blocking rule is missed, or advisory fail when only advisory rules are. A failing verdict can be overridden only with a written reason, and every verdict and override stays on the AI system's record.

What it is not

A passing evaluation, a clean red-team run or a green release gate is evidence about the cases, attacks and rules you ran — not a guarantee that an agent behaves safely in every situation, and not a substitute for your own review before a release.

Read the docs: evaluation datasets. Read more: red teaming AI agents · scoring agent quality and regressions. Then: runtime security · risk management.