An agent did something it should not have. A record was modified, data went somewhere unexpected, a payment was issued, spend spiked. Now you have to determine what happened, how far it went, whether it is ongoing, and what to change.
For conventional software this is a familiar exercise: read the logs, reproduce with the same inputs, find the bug. For agents, both halves break. The logs typically record actions without the context that produced them, and reproduction is unreliable because the system is non-deterministic — the same inputs may produce different behavior. What you did not capture at the time is generally unrecoverable.
That makes agent incident readiness primarily a question of what you record before anything goes wrong.
Three Linked Record Types
The decision context. What the model saw before it acted: the system prompt version, the conversation or task history, retrieved documents with their sources, tool responses received, and inter-agent messages. This is what answers why. Without it, an investigation can describe behavior but not explain it, and the remediation ends up being a guess.
The action. The tool call: which tool, which arguments, which target, the result, latency, cost, timestamp. Most teams have some version of this, often incomplete — arguments truncated, targets not recorded, results omitted.
The authorization. Which agent, which asserted principal, which connection, which policy evaluated, what the decision was. This is what answers how it was permitted, and it is the piece that determines whether the remediation is a scope change, a policy fix, or a code fix.
The three must be linked by a common identifier. A task or trace id generated at the start and propagated through every hop — model calls, tool calls, sub-agent dispatches, queue messages, downstream service calls — is what makes reconstruction possible. Without it you have three sets of records that cannot be joined, and joining them by timestamp across concurrent tasks does not work at agent volumes.
What Is Usually Missing
In practice, investigations stall on the same gaps.
Retrieval provenance. Which documents were retrieved, from which source, at which version. Without it, "why did the agent believe that" is unanswerable, and you cannot determine whether a poisoned corpus entry is still present and still firing. See RAG pipeline security.
Prompt and configuration versions. The system prompt, model version, temperature, tool inventory, and guardrail configuration active at that moment. All of these change, and an investigation against current configuration when the incident happened under a previous one produces wrong conclusions. Version identifiers on every record are cheap; reconstructing them later is not. See versioning and rollback for AI agents.
Allowed decisions, not only blocked ones. Teams log denials because denials look like security events. But an incident is a sequence of allowed actions — that is precisely what makes it an incident. Recording allows is also what lets you demonstrate the control was live and evaluating rather than misconfigured into permissiveness.
The principal, distinct from the agent. A record naming only the agent cannot answer whose authority was used, which is the central question in any delegation or confused deputy incident.
Sub-agent chains. Which agent dispatched which, with what scope, at what depth. Multi-agent incidents are usually chain problems, and a flat log of actions per agent does not reveal the chain.
Tool arguments in full. Truncated arguments hide the target. "Called query_records" tells you nothing; "called query_records with tenant_id=*, limit=100000" tells you everything.
The Investigation Sequence
1. Contain first, investigate second. Halt the agent, revoke its credentials, disable its triggers. Contain before you understand — the understanding takes hours and the exposure is ongoing. Containment must take effect on in-flight work, not only on new invocations. See rogue AI agents and kill switches.
2. Preserve. Snapshot logs, audit records, retrieval indexes, memory stores, and any queue contents before retention or ordinary operation erases them. Vector store contents in particular get reindexed, and agent memory gets updated. Preserving them is time-sensitive in a way that log retention usually is not.
3. Establish the timeline from the audit trail. Pull every record carrying the task identifier, ordered by sequence rather than by wall-clock timestamp — clock skew across services will mislead you at millisecond resolution. A hash-chained audit log gives you sequence for free and tells you whether anything is missing.
4. Find the divergence point. Walk forward until the agent's behavior stops matching its task. Then look at what entered the context immediately before: a retrieved document, a tool response, a user message, a sub-agent message. This is usually where the cause is, and it is why decision context matters more than action logs.
5. Determine the authorization path. For each harmful action, identify the connection and policy that permitted it. There are three possible answers, and they lead to different fixes: the scope was too broad (narrow it), the policy had a gap (fix the rule), or enforcement was bypassed (fix the architecture — the most serious of the three).
6. Establish blast radius. Everything the agent touched during the window, not only the actions you noticed. Data read (a read is a potential exfiltration), records written, messages sent, credentials used, downstream systems called, other agents dispatched. Then check whether any of it persisted: memory entries, corpus writes, cached state, scheduled follow-ups.
7. Check for propagation. In multi-agent systems, did the poisoned content or the escalated authority reach other agents? Did anything get written to a shared store that other agents read? See cascading failures in multi-agent systems.
8. Remediate structurally, and test the remediation. Convert the finding into a scope reduction, a policy rule, a cap, or a gate — then write a test asserting the action is now blocked. A remediation without a regression test regresses on the next configuration change.
Integrity of the Evidence
An audit record that the agent's own compromised credentials could modify is not evidence. If an attacker with agent-level access can delete or edit log entries, the absence of evidence proves nothing.
Three properties matter:
Append-only with hash chaining. Each entry includes the hash of the previous one, so deletion or modification breaks the chain and is detectable. This also gives you a definitive answer to "are any records missing," which timestamp-ordered logs cannot provide.
Signed, ideally per tenant. Signature under a key the agent cannot reach, held in a KMS or HSM. See BYO KMS and per-tenant signing keys.
Independently anchored. Periodic anchoring of the chain head to an external transparency log or immutable store establishes that the record existed at a point in time, independent of your own systems' claims. This is what converts an internal log into something an auditor or a court can weigh. See tamper-evident audit logs.
Write path matters too: the enforcement point should write the record, synchronously, before the action's result is returned. Records written by the agent itself, or asynchronously by a component that can drop them under load, have gaps exactly when you need them.
Retention and Redaction Tension
Forensics wants everything kept forever. Privacy law and data minimization want the opposite, and prompts and tool arguments frequently contain personal data.
The resolution that works in practice is tiered: full detail — arguments, retrieved content, context — for a shorter window sized to your realistic investigation horizon (60 to 90 days covers most incidents), and structural metadata without content for the longer compliance window. Structural metadata is enough to prove what happened and who authorized it; content is what you need to explain why, and the need for it decays quickly.
Redaction has to be selective rather than blanket. Redacting the entire tool argument removes the target, which destroys the record's forensic value. Redact the personal data field and keep the structure. And keep redaction reversible-by-authorization for the short window if your legal position allows, because an investigator who cannot see the argument cannot investigate. See GDPR for AI systems and erasure rights.
Common questions
Can I reproduce an agent incident by replaying the same inputs?
Rarely, and you should not depend on it. Sampling makes generation non-deterministic, provider models change beneath you, retrieval results shift as the corpus changes, and tool responses differ. Replay is useful for testing a fix and for understanding a class of behavior; it is not a reliable way to establish what happened. The record is the evidence. Where determinism matters, capture the exact context rather than trying to recreate it.
How much does recording full context cost?
Storage is the smaller part; the larger cost is discipline about what to keep and for how long. Full prompt and retrieval capture on high-volume agents produces significant volume, which is why tiering matters — full detail for a bounded window, metadata beyond. Compression is very effective on this data because prompts are highly repetitive. The cost is almost always less than the cost of one unexplainable incident.
Who should be able to read these records?
Fewer people than can read ordinary logs, because the content includes prompts, retrieved documents, and tool arguments — often the most sensitive data in the system. Access to audit content should itself be audited, and read access should be separated from the ability to configure retention. An audit system whose administrators can silently reduce retention has a gap.
What if the incident involves a third-party agent or MCP server?
You will have your side of the interaction and not theirs, which is why recording the full request and response at your boundary matters more for external counterparties than for internal calls. Establish in advance, contractually, what logs a counterparty will provide and on what timeline; discovering during an incident that a vendor keeps nothing is a bad surprise. See cross-org agent federation and trust manifests.
How Praesidia approaches forensic readiness
Praesidia writes the record at the enforcement point, so every governed call produces a tamper-evident entry covering what was attempted, what was permitted, and what resulted — for allowed calls as well as blocked ones. Integrity is verifiable independently of the platform rather than on the platform's own assertion.
Task identifiers propagate across model calls, tool calls, and cross-agent dispatch, which is what makes a multi-hop timeline reconstructable, and an exported record can be verified offline rather than through a platform API. See audit trails that hold up, incident response for AI agent breaches, and the Praesidia documentation.