Test every agent release against your own cases, your own defenses and your own bar before it ships.

Evaluations and runs

An evaluation scores an agent's answers against criteria you write, either as pass or fail or on a scale, with a passing threshold you set. Each run tests one active agent on its current prompt or on one specific prompt version and records the run's pass rate, mean score, response latency and cost. A completed run opens case by case, and runs can be compared side by side.

Datasets and regressions

A dataset is a named set of test cases — an input, the output you expect and your own tags — added by hand or imported from your agents' recent completed tasks, with personal data detected in an imported input redacted before the case is stored. Two runs over the same dataset are compared case by case: which cases regressed, which improved and which were unchanged, alongside each run's pass rate.

Red-team campaigns and attack packs

Red-team campaigns use synthetic attack probes selected from the recorded context of your own registered agents, connections and MCP servers, on demand or on a schedule. Targets must be explicitly opted in by your organization; other targets are refused. The attack library covers prompt injection and jailbreaks, scope escalation, out-of-scope tool use, bulk PII exfiltration, RAG poisoning, memory poisoning, malicious skills, cross-agent escalation, lateral movement, privilege escalation and policy bypass, with MCP-specific probe classes for poisoned tool descriptions, unauthorized tool chaining and credential leakage. Red-team campaigns are a plan feature; see pricing.

What a red-team run tells you

Probes check enabled organization-wide input guardrails and intent rules without calling the target's production endpoint. They do not test agent-specific guardrails or the live target's behavior. Configured model guardrails may call external evaluators with the synthetic content. Runs report observed verdicts and severity; high or critical severity findings can escalate to linked security incidents.

Release gates

Each AI system can carry release-gate rules on an evaluation run's pass rate, mean score, median latency, 95th-percentile latency or total cost, each marked blocking or advisory. Your deployment pipeline asks for a verdict with an organization API key and gets pass, fail when a blocking rule is missed, or advisory fail when only advisory rules are. A failing verdict can be overridden only with a written reason, and every verdict and override stays on the AI system's record.

What it is not

A passing evaluation, a clean red-team run or a green release gate is evidence about the cases, attacks and rules you ran — not a guarantee that an agent behaves safely in every situation, and not a substitute for your own review before a release.

Read the docs: evaluation datasets. Read more: red teaming AI agents · scoring agent quality and regressions. Then: runtime security · risk management.