Abuse prevention for AI APIs is a wider problem than throttling. (For the throttling mechanics themselves, see how to rate-limit AI agents.) Rate limiting for AI APIs is not the same problem as rate limiting a conventional REST service. The combination of variable token usage, long-running tool calls, multi-agent fan-out, and opaque upstream model costs means that request counts alone are a poor signal. Effective throttling for AI workloads requires layered controls: per-connection rate limits, per-time-window request budgets, spend caps, and policy-level gating — all enforced before a token reaches a model or a tool reaches an external system. For a deeper look at the distinction between spend-based and request-based throttling, see budgets vs rate limits: controlling agent consumption.
Why AI APIs Are Different
A traditional API rate limit protects a server from being overwhelmed by counting requests per second or per minute. That is still relevant, but it misses most of the actual risk surface with AI:
Token volume is the real cost driver. Two requests to a language model may differ by three orders of magnitude in cost depending on context length and output. A naive request-count limit will stop a tight polling loop but let a single runaway agent with a 200k-token context window consume a month of budget in an afternoon.
Tool calls are asynchronous and unbounded. An agent that decides to call a tool — send an email, write to a database, call an external API — may trigger a chain of side effects that outlive the original request. The agent's next turn might call the same tool again, and again, in a loop. Rate limiting only the inbound API call does not contain this.
Multi-agent fan-out multiplies everything. When one orchestrating agent calls five sub-agents in parallel, and each calls a tool, the apparent "one request" becomes potentially dozens of downstream operations within seconds. Flat rate limits applied at the wrong layer collapse either too early or too late.
Abuse patterns differ from conventional APIs. Credential stuffing, scraping, and DDoS are well-understood for web services. For AI APIs, the threat is more often misconfiguration — a retry loop that does not back off, a workflow that re-triggers itself, or an agent given broader permissions than intended that methodically exhausts a capability. The distinction matters because the defenses differ. For a concrete attack-path analysis of runaway spend, see threat model: runaway agent spend.
The Control Points That Matter
To build abuse-resistant throttling for AI workloads, you need controls at multiple layers simultaneously.
Per-connection rate limits
Rather than a single global limit, apply rate limits at the connection level — the directed link between a specific calling agent and a specific target. This lets you set different tolerances for different relationships: a trusted internal orchestration agent can have a higher per-minute limit than a newly registered agent, and when a violation occurs, only that connection's tasks are refused rather than service degrading for all callers. In Praesidia this is how tasks between agents are limited, per minute and per hour on each connection. An MCP tool call is limited differently: by a per-minute limit on the tool, counted across all its callers, so one agent's runaway loop can use it up for every caller, and by an optional hourly limit for one agent, set in a per-tool permission on that agent's connection to the server.
Per-minute and per-hour windows serve different purposes. A per-minute window catches burst behavior — a loop that fires off 100 requests in 30 seconds. A per-hour window catches sustained high-throughput usage that might stay under the per-minute threshold by spreading out requests. Both are worth tracking.
Time-window restrictions
Some connections should only be active during business hours, or only overnight. A data enrichment agent that runs nightly does not need to send tasks to the agent it feeds at 2pm on a Tuesday. Restricting the active time window is a simple, hard limit that eliminates an entire class of unexpected behavior. In Praesidia each connection has one recurring window, set in whole UTC hours with optional days of the week; there is no one-off maintenance window, and the window applies to tasks between agents, not to MCP tool calls. For how to structure these policies across your agent fleet, see How to Rate-Limit AI Agents.
Spend-based caps
Because token costs are so variable, spend caps complement request-rate limits rather than replacing them. A monthly spend cap on a connection means that even if the rate limit is not tripped, the connection cannot accumulate indefinite cost over time. This is especially important for connections that do low-frequency but high-cost operations — a deep research agent that runs once a day can still consume significant budget if it is not constrained.
Spend caps introduce a time-of-check / time-of-use challenge common to any cap enforced against a running total, which the platform's enforcement model accounts for.
Task-type allow-lists
Not every agent should be able to initiate every type of operation over a given connection. An allow-list of permitted task types on a connection constrains what can happen even if authentication succeeds. In Praesidia the types are a message, a tool call and a delegated multi-step job, so a connection can carry messages to an agent without letting the caller hand it whole jobs. Task types do not separate reads from writes: an agent that should only query a database needs that limit on the tools it can call, set by tool permissions and tool rules, not by a task type.
Model and tool allow-lists
For connections between agents, an explicit allow-list of permitted models and tools gives you a second layer of least-privilege enforcement: a task that names a model or tool outside the list is refused, and once a model list is set, a task that names no model is refused too. An integration configured for a lightweight model should not silently switch to a more expensive one because a system prompt was modified. A connection to an MCP server has no such list: its per-tool permissions apply to the tools they name and allow the rest. A tool newly added to the server is held back elsewhere: it is not offered to agents, and a call to it is refused, until the server's owner or an organization owner approves the server's updated tool list. After that, in enforce mode, the agent's tool rules still deny a tool that no rule matches, unless a broad pattern such as a wildcard already allows it.
Trust Levels as a Dispatch Gate
Rate limiting controls how often something can happen. Trust levels control whether it should happen at all. The two work together.
For tasks between agents, trust is checked on both sides of the rate limits, and a task submitted through the Praesidia API has already passed the API's request rate limit. A connection can set a minimum trust level for the calling agent, checked before the task's rate limits: a task from an agent below it is refused in any governance mode, and an agent whose trust cannot be computed is treated as untrusted. An agent policy can set a minimum trust score to run, or require approval below a score. That gate runs after the rate limits; in enforce mode it refuses or holds the task, and in observe mode it only records what it would have done. The score combines identity verification, activity history, how often guardrails trigger on the agent's traffic, its profile and its security settings, so an agent whose traffic keeps tripping guardrails loses score and can fall below a gate before its rate limit runs out. It is not an anomaly detector: a burst of activity does not lower the score by itself. MCP tool calls do not read the trust score; in enforce mode, a call from an agent that is not active, such as a suspended one, is refused by a status check instead. For how trust scores are computed and applied, see Trust Scores and Attestations: Deciding Which Agents to Trust.
When trust cannot be computed, the connection's minimum trust level treats the agent as untrusted, and an agent policy's trust gate refuses the task when that policy is enforced, meaning the organization is in enforce mode, the policy is not set to audit only and it is not held at an earlier rollout stage; otherwise the agent-policy gate records the error and lets the task go ahead. Refusing on an evaluation error is the right default for security-sensitive contexts; whether that trade-off is correct depends on how critical availability is relative to the sensitivity of the downstream action.
Guardrails as a Content Layer
Rate limits and trust gates handle quantity and identity. Guardrails handle the content of what is being sent and received. In the context of abuse prevention, guardrails matter because some attacks are not volumetric — they are semantic. A prompt injection that executes one carefully crafted request is invisible to a rate limiter that watches for bursts.
Guardrails on a connection can inspect requests and responses for policy violations — data exfiltration attempts, credentials in prompts, or outputs matching misuse patterns. Pairing content inspection with rate limiting closes the gap between volumetric abuse and low-frequency, high-impact attacks. See content guardrails for AI agents for how the inspection layer works in practice.
Circuit Breakers and Failover
Well-designed connections should degrade gracefully under failure. A common pattern is a backup route to a different agent or MCP server that takes traffic when the primary is unhealthy; Praesidia does not offer one, because automatic failover is disabled. Its circuit breaker sits on each receiving agent: when an agent keeps failing, Praesidia briefly stops routing tasks to it instead of sending them elsewhere. The breaker counts those failures itself; the connection health statistics do not feed it.
Practical Design Principles
Enforce before the expensive operation, not after. A rate limit that fires after a model call completes is a post-hoc record of cost already incurred, not a control.
Make limits observable. Log every violation with connection identity, caller, and count vs limit. An unexplained rejection is an operational mystery; a logged policy violation is actionable.
Keep limits per-actor, not global. A global limit set to avoid false positives will be too loose; set to prevent abuse it will be too tight. Per-connection limits tune to the expected behavior of each specific relationship.
Plan for retry amplification. When a request is rate-limited, callers retry. If all callers share the same backoff pattern, the retry wave arrives exactly when the window resets. Communicate a retry-after interval and design callers to apply jitter.
How Praesidia Handles This
Praesidia models rate limits, spend caps, task-type allow-lists, model allow-lists, tool allow-lists, and active time windows as first-class attributes of each connection — the governed link between an agent and its target. These policies are checked when a task is dispatched over the connection, before the work starts, so no expensive operation starts until all applicable checks pass, and a denial is written to the audit trail as a decision record naming the reason — rate limit, spend cap, task type, model, tool or time window. Connection-scoped guardrails add the content inspection layer on top.
The connections console in Praesidia exposes health stats per connection — latency, error rate, request counts — so operators can see each link's behavior over time and adjust policy before problems compound. There is no failover to a backup connection; a circuit breaker briefly stops routing tasks to a failing agent instead.
For teams building on top of AI APIs rather than simply consuming them, this approach — per-edge policy rather than global rate limiting — scales better with the complexity of multi-agent architectures. You get finer-grained control and clearer attribution when something goes wrong. For more on the connections model, see governed connections between agents and resources.
Common questions
Is request-count rate limiting enough for AI APIs? Request counting is necessary but not sufficient. Because AI operations vary widely in cost, token-based spend caps and per-connection policy controls are needed alongside request-rate limits to prevent both volumetric abuse and expensive low-frequency misuse.
What is the difference between a rate limit and a spend cap? A rate limit controls how often something can happen in a time window — requests per minute, for example. A spend cap controls how much cumulative cost a connection can accrue over a billing period. Both are needed because a caller can stay within a request-rate limit while still generating high cost through expensive model calls or tool invocations.
How should I handle rate-limit errors from AI agents? Log the event with full context (caller identity, connection, count vs limit), return a retry-after interval, and design callers to apply jitter to their retries. Investigate repeated violations — they often indicate a misconfigured retry loop or an agent operating outside its intended scope, rather than a genuine traffic spike.
How do rate limits interact with human-in-the-loop approval flows? Rate limits are evaluated at dispatch time, before the task executes. If a high-risk action also requires human-in-the-loop approval, that gate fires after the rate-limit check passes. A rate-limited request is rejected before it reaches the approval queue, keeping the queue free of volumetrically abusive submissions.