Provider API keys are the most consequential secret in most AI deployments and frequently the least carefully handled. They are handed out during prototyping, pasted into configuration, and shared across services because the fastest path to a working demo is a key in an environment variable.
Two distinct harms follow a leak. The obvious one is spend: the key bills against your account, and an attacker with a valid key can run up substantial cost quickly — a denial of wallet attack with the credential already in hand. The less obvious one is data: depending on the provider and your configuration, a key may grant access to stored assistants, files, fine-tunes, batch jobs, and usage history, which can include prompt content.
How These Keys Leak
Client-side inclusion. The single most common and most severe leak: a provider key embedded in a mobile app, a browser bundle, or a desktop application. Every user has it. Obfuscation does not help — anyone can read the network traffic. There is no configuration in which a provider key belongs in client-distributed code; the client must call your backend, which calls the provider.
Committed configuration. .env files, config maps, notebooks, and test fixtures in version control. Public repositories get scanned within minutes by automated harvesters, and private repositories leak later through forks, mirrors, and contractor access.
Container images and build output. A key baked into an image layer persists even if a later layer removes it. Keys echoed in CI logs are visible to everyone with build access, which is usually broader than intended.
Prompt and context leakage. If a key appears in a system prompt, a tool definition, or an error message that lands in context, the model can emit it. See system prompt extraction.
Observability pipelines. Full request headers captured in traces and forwarded to a vendor SaaS. The key is then in a system with a different access model and a long retention period.
Sprawl through convenience. One key copied into six services during debugging. Nobody tracks the copies, so rotation breaks something unexpected and gets rolled back.
Why Per-Agent Provider Keys Are the Wrong Answer
The instinct after a leak is to issue a separate provider key per agent or per service so the blast radius is smaller. It sounds like least privilege and it is mostly counterproductive.
More keys, more places to leak. Twenty keys distributed across twenty runtimes is twenty times the surface, each with its own storage and rotation problem.
Rotation becomes intractable. Twenty rotations coordinated with twenty deployments, and most providers cap the number of active keys.
Attribution is coarse anyway. Provider-side usage reporting per key gives you spend per key, not per tenant, per task, or per user — which is what you actually need for chargeback. See spend attribution and showback per agent.
No enforcement. Provider keys carry usage limits at best; they do not carry your policy. Per-key limits cannot express "this agent may spend $50/day on behalf of tenant X, with PII redacted from prompts."
The Configuration That Works: Proxy the Provider
Hold provider keys in exactly one place — a gateway that mediates model calls — and give agents internal credentials instead.
The agent authenticates to the gateway with its own short-lived, scoped credential. The gateway evaluates policy, attaches the provider key, forwards the request, meters the result, and records it. The agent never possesses the provider key, and no provider key exists in any agent runtime, image, or context window.
What this buys, in order of value:
Exposure reduction to one location. One secret, in one secret store, read by one service. Leak paths through agent code, images, prompts, and configuration disappear because there is nothing there to leak.
Rotation as a single operation. Update the key at the gateway. No coordinated deployment, no unknown copies. See key rotation for agent credentials.
Real attribution. The gateway knows the authenticated agent, the principal, the tenant, and the task, and it meters the call against all of them. This is attribution the provider cannot give you at any key granularity.
Enforcement in the path. Spend ceilings checked and reserved before dispatch, prompt guardrails and PII redaction applied, model routing by policy, rate limits per agent. All of it applies because the call physically passes through. See budget policies and hard spend caps and how to redact PII from agent prompts.
Instant revocation per agent. Disable the agent's internal credential and it stops immediately, without touching the provider key or affecting any other agent.
BYOK for tenants. A tenant can supply their own provider key, held by the gateway and used only for their traffic — so their model spend bills to their account and their prompt data is governed by their provider agreement. This is a frequent enterprise requirement and it is only workable when keys are centralized. See BYOK: managing LLM configurations.
The cost is a hop in the path — single-digit milliseconds against model latency measured in hundreds — and a component that must be available. Both are acceptable; the availability concern is addressed the same way as any other critical path component.
Provider-Side Controls Worth Using
Even with a gateway, use what the provider offers:
- Separate workspaces or projects per environment. Development keys must not be able to reach production resources or bill against production budgets.
- Restricted or scoped keys. Several providers now offer keys limited to specific endpoints or read-only access. Use the narrowest scope that works.
- Usage limits and alerts at the provider. A backstop below your expected ceiling. If your own controls fail, this is what catches it — and it should be set, because it is the only limit that holds if your gateway is bypassed.
- Source address restrictions where offered, pinning key use to your gateway's egress addresses. This alone renders a leaked key unusable from elsewhere.
- Provider audit logs, ingested into your own pipeline so key usage from unexpected sources is visible.
Detection and Response
Detection signals: provider-side usage from an unexpected address or region, spend divergence between your gateway's metered total and the provider's reported total (the strongest signal that a key is being used outside your path), usage of models or endpoints your gateway never calls, and secret-scanner hits in repositories and images.
The spend-divergence check deserves emphasis. If you meter every call at the gateway, your total should match the provider's invoice within a small margin. A gap means calls are happening that did not go through you. Reconciling those two numbers on a schedule is cheap and it is the most reliable leak detector available.
Response: rotate immediately, review provider-side usage history for the compromise window, check what non-inference resources the key could reach (files, fine-tunes, stored assistants), remove the key from wherever it leaked including image layers and CI history, and record the incident. Rotating without finding the leak path means it recurs. See incident response for AI agent breaches.
Common questions
Is a client-side key ever acceptable if I set a low usage limit?
No. A published key is abused within hours, and the usage limit only bounds the bill — it does not prevent access to whatever else the key can reach, and it does not stop your service from being used as free inference for someone else's product. The correct architecture is always client to your backend to the provider, with the client authenticating to your backend by its own means.
What about ephemeral or short-lived provider tokens for client use?
Where a provider supports minting a short-lived, narrowly-scoped token server-side for direct client use — for realtime voice sessions, for example — that is a legitimate pattern, because the client holds a credential that expires in minutes and can only do one thing. The key remains server-side. Verify the scope carefully: a short-lived token with broad capability is still a broad credential.
Do I need a gateway if I have only one service calling the provider?
The exposure argument is weaker, but the attribution and enforcement arguments still apply — and one service becomes three faster than expected. If you are not ready for a gateway, at minimum keep the key in a secret manager fetched at runtime rather than in configuration, and meter calls yourself so you can reconcile against the invoice.
How do I handle multiple providers and models?
That is one of the stronger reasons to centralize. A gateway holding keys for several providers can route by cost, capability, or availability without agents knowing which provider served a request, and failover becomes a policy decision rather than a code change. It also means per-provider key hygiene is solved once. See model routing and cost-quality tradeoffs.
How Praesidia approaches provider credentials
Praesidia holds provider credentials at the gateway and never in an agent's environment or context. Agents authenticate with their own short-lived, scoped credentials, and the gateway attaches the provider key after evaluating policy — spend ceilings reserved before dispatch, guardrails applied to prompt content, model routing per configuration.
Every call is metered with per-agent and per-tenant attribution, so attribution is finer than any provider-side key granularity and the metered total can be reconciled against provider invoices. Tenants can supply their own provider keys for their own traffic, and rotating any key is a single operation at the gateway. See organization API keys and scopes and the Praesidia documentation.