Golden Signals for Agent Services

The four golden signals — latency, traffic, errors, and saturation — remain the right starting framework for monitoring agent services, but each needs to be reinterpreted for a workload where a single request can take a widely variable path, and a fifth signal, task outcome quality, has to be added, because the original four cannot detect a request that completes cleanly while still failing the user's actual goal.

This is a companion to the broader instrumentation guidance in observability for AI agents and the SLI/SLO framing in SLOs for AI services. Golden signals are the minimum dashboard; SLOs are what you set as targets against them.

Why the original four need reinterpretation

The golden-signals framework was built for services with a roughly uniform unit of work — a request comes in, does a bounded amount of work, and returns. Agent tasks break that assumption directly: the same "task" can resolve in a single model call or expand into a dozen tool calls and sub-agent delegations depending on the input, the model's own behavior, and what it decides is necessary. Each of the four signals has to be read differently as a result.

Latency: percentiles across a wider, more meaningful distribution

Latency for an agent task spans a much wider range than for a conventional request, and the shape of that distribution carries more information than usual. A bimodal latency distribution — a cluster of fast, single-call tasks and a separate cluster of slow, multi-hop tasks — is normal for agents and shouldn't be flattened into a single average, which would describe neither cluster accurately.

Break latency down by task type or complexity class where you can, rather than reporting one blended percentile set for the whole fleet. And instrument stage-level latency — how long was spent in the model call versus tool calls versus orchestration overhead — because end-to-end latency alone tells you a task was slow without telling you where the time went. See observability for AI agents for the tracing pattern that makes stage-level latency visible.

Traffic: task volume, not just request count

Traffic for an agent service is usefully measured as task volume, but task volume alone under-describes load, because tasks vary so much in the work they generate. A traffic graph that shows steady task count while token throughput and downstream tool-call volume climb is telling you that tasks are getting more complex, not that volume is flat — and that's a capacity-relevant signal that a request-count graph alone would miss entirely.

Pair task volume with token throughput and fan-out (calls generated per task) to get a traffic picture that actually reflects load on the systems that matter — the model provider and downstream tools — rather than just the front door.

Errors: separate infrastructure failures from task-level non-success

This is where agent monitoring diverges most from convention. A conventional error rate counts requests that failed — typically a 5xx response or an exception. For an agent, there are two categorically different kinds of "not successful":

• Infrastructure errors: the model provider returned an error, a tool call timed out, the orchestration layer crashed. These are genuinely analogous to a conventional error rate.
• Task-level non-success: the agent ran to completion without any infrastructure failure, but the task didn't succeed — it hit a guardrail block, exhausted its retry budget on a sub-task, or determined it couldn't complete the request and reported that gracefully.

Conflating these into one error rate hides which kind of problem you actually have. A spike in infrastructure errors points you toward a provider or tooling issue. A spike in graceful task-level non-success, with infrastructure errors flat, points you toward a prompt, guardrail, or task-difficulty issue instead — a completely different investigation. Track them as separate signals, even though both eventually roll up into "tasks that didn't succeed" for a top-level success-rate SLI.

Saturation: provider and budget headroom, not CPU

Saturation traditionally measures how full a resource is — CPU, memory, disk, connection pool. For an agent fleet, the resource that actually saturates first is almost always external to your own infrastructure: provider rate-limit headroom, token budget headroom, or a downstream tool's own capacity ceiling. A dashboard that shows worker CPU utilization as the saturation signal for an agent fleet is very often measuring the wrong resource entirely, because a fleet can be maximally constrained by provider quota while every worker process sits well under its own resource limits.

Build the saturation view around the constraints that actually bind: current token throughput against provider quota, current spend against budget cap, queue depth as the leading indicator that a ceiling is being approached before task failures start. See budgets versus rate limits for the distinction between the cap you set and the ceiling the provider sets, and token usage forecasting and capacity planning for how to project this signal forward rather than just reading it at a point in time.

The fifth signal: task outcome quality

None of the four classic signals can detect the failure mode that matters most for agent services: a task that completes without any infrastructure error and without a graceful non-success report, but produces output that's simply wrong, low-quality, or not what the user actually needed. This is possible precisely because agent output correctness is a distribution rather than a boolean — a confidently wrong answer, delivered in the expected format, is indistinguishable from a correct one to every signal except one that actually evaluates the output.

Task outcome quality — measured through evaluation scoring against a golden set, sampled or continuous — is the signal that closes this gap. See agent evaluations: scoring quality and regressions and eval datasets and golden sets for agents for how that evaluation layer works. Treat it as a golden signal with the same standing as the original four, not as an optional extra that only gets checked after a user complains — because for agent services, it is frequently the only signal that would have caught the problem at all.

A comparison table

Building the dashboard on-call actually opens

A golden-signals dashboard is only useful if it's the artifact an on-call engineer opens first, and it should answer, at a glance, which category of problem is occurring: is this latency, traffic, errors, saturation, or quality? Structuring the dashboard around these five categories — rather than an undifferentiated grid of every metric you happen to collect — is what lets someone unfamiliar with the specific incident orient quickly. This is the same classification discipline a good on-call process depends on; the dashboard is where that classification starts. See the operations dashboard for your AI estate for situational awareness beyond the golden-signals view during an active incident, and advanced analytics for AI operations for per-agent breakdowns once the dashboard has narrowed down which category of problem you're looking at.

Rolling the five signals up per agent and per organization

A fleet-wide view of the five signals tells you whether the platform as a whole is healthy, but it can hide a single agent or organization in real trouble behind an otherwise healthy aggregate. Because agent behavior and workload shape vary so much between agents, the same golden-signals framework needs to be available at the per-agent and per-organization level, not only rolled up to the fleet.

This is the same reasoning that applies to metrics generally: a fleet-wide average latency that looks acceptable can be masking one agent whose latency has quietly doubled, if that agent's traffic is a small enough share of the total. Filtering the five signals down to a specific agent or organization should be a normal part of investigating a page, not a specialized deep-dive reserved for the hardest incidents.

What good looks like

A golden-signals setup for agent services tracks all five signals — the original four reinterpreted for variable-shaped work, plus task outcome quality as a first-class addition — with saturation built around provider and budget headroom rather than compute utilization, and errors split between infrastructure failures and graceful task-level non-success. The dashboard built from these signals should let an on-call engineer classify an incident's category within moments of opening it, before diving into any deeper investigation.