Book demo Start free
Model gateway

A BYOK LLM gateway: one URL for your models, and your provider keys stay yours.

Praesidia’s model gateway sits between your applications or agents and the model providers they call. Your code keeps its OpenAI or Anthropic SDK and changes two values, the base URL and the key. It sends a Praesidia application key. The gateway checks the call against your organization’s rules, then swaps in your own provider key server-side.

Route your first model call Read the quickstart
Your SDK, pointed at the gateway Placeholder host 
# OpenAI SDK: chat completions, completions, embeddings, Responses
client = OpenAI(
    base_url="https://<your-gateway-host>/openai/v1",
    api_key=os.environ["PRAESIDIA_APPLICATION_KEY"],  # pra_..., not your provider key
)

# Anthropic SDK: the native Messages API, passed through, not translated
client = Anthropic(
    base_url="https://<your-gateway-host>/anthropic/v1",
    api_key=os.environ["PRAESIDIA_APPLICATION_KEY"],
)

The pra_ key is checked by Praesidia and is never sent to the provider. The gateway removes it and adds your provider key for that one call. Each provider has its own path prefix on the same host.

How it works

What the gateway checks before your provider sees the call

Each check below can refuse the call before it reaches the provider. A refusal returns a named error, so your code can tell a budget stop from a guardrail block.

Who is calling
The application key is validated against your organization. An unknown key is refused. A key without permission to call models is refused too.
Whether you have stopped everything
If your organization is frozen, every model call is refused before any provider call, until the freeze is lifted.
What the request says
Input guardrails read the prompt, and a blocked request is refused with a named guardrail error. Tool definitions are checked against the agent’s tool policy. Provider-hosted MCP connectors are refused, so tools run through the connections you registered.
Which model
The requested model must match the model configured for that provider route. Any other model is refused with a named error.
What it may cost
If a generation request sets no output limit, the gateway adds one. The spend is then held against your budget before dispatch. Over budget is refused, and so is a call past an agent’s hourly spend cap, each with its own named error.
Where it goes
Your egress policy can allow or deny the upstream host, port and scheme. Private, loopback and cloud-metadata addresses are refused. Both checks run before your provider key is fetched, so a refused destination never receives it.

On the way back. Output guardrails check the response, and usage is settled from the token counts the provider reports. If a provider reports none, or an implausibly low count, the gateway estimates from what was actually exchanged. For streamed chat and completions, the gateway asks the provider to include usage, so tokens you never see, such as reasoning tokens, are still counted.

Capabilities

What the model gateway does

Model traffic only. Tool calls to your MCP servers are governed on their own path by the MCP gateway.

  • Your provider keys, never in your code

    You add a provider key once, in your workspace’s LLM configuration. Praesidia stores it encrypted with AES-256-GCM, bound to your organization and that configuration, and shows only its last four characters. Your applications hold a Praesidia key instead. If one leaks, revoke it without touching the provider account. The gateway removes every client credential header before forwarding, and refuses requests that put a credential in the query string. Bring-your-own-key model configuration is in every plan.

    How BYOK key storage works

  • The providers you already use

    Configure an agent's model with your own keys for OpenAI, Anthropic, Azure OpenAI, Google Gemini, Mistral AI, Cohere, DeepSeek, Qwen, Moonshot, Ollama or an OpenAI-compatible endpoint. Model configuration is in every plan. Gateway routing is a separate path: custom OpenAI-compatible routes require the Advanced tier or Enterprise and deployment enablement. Check the supported route and authentication mode for your provider.

    Anthropic’s Messages API goes through unchanged, in Anthropic’s own request, response and streaming format, not translated into another API’s shape. A self-hosted Ollama needs no provider key.

    Your first model request

  • Spend held before the call, settled from real usage

    Before a call leaves, the gateway holds its likely cost against the budget that applies, from the route’s own pricing. If a generation request sets no output-token limit, the gateway adds one (4,096 tokens), so the provider cannot bill past the amount held. Afterwards the hold is settled from the provider’s usage report, and the excess is released. Budget policies and spending limits are on Team Individual, Team Advanced and Enterprise.

    AI FinOps guide

  • Guardrails on prompts and responses

    Rule guardrails check the prompt on the way in and the response on the way out. A match can block, warn, redact or replace. ML guardrails and LLM-judged guardrails apply to gateway traffic too. Rule-based guardrails are in every plan. ML guardrails start at Team Individual, LLM-powered ones at Team Advanced.

    Guardrail docs

  • Tool calls the model asks for

    When a model answers with a tool call, the gateway reads the call before passing it on. It checks function, custom, MCP and built-in actions against the agent’s tool permissions and a per-minute call cap. An action type the gateway cannot read is refused rather than passed through.

    If an emergency freeze blocks MCP tool calls for your organization, the gateway refuses tool use on generation routes, whatever the tool permissions allow. A request that offers the model a tool is refused before it reaches the provider. A response that carries a tool call is held and checked first, so none of it reaches your agent. Both return 423 mcp_contained. Calls without tools still go through, unless the freeze also blocks model access.

    Tool policies

  • One switch, and a defined failure mode

    An organization freeze stops model traffic as well as agent traffic. If the gateway cannot reach Praesidia’s control plane, governed calls are refused, never forwarded unchecked. The gateway also will not start in a mode that allows them.

    Runtime security
Security

Where it runs, and who holds the keys

  • Hosted in Frankfurt. The model gateway runs on Amazon Web Services in eu-central-1 (Frankfurt, Germany). Gateway connections terminate TLS in Frankfurt, and the data Praesidia stores is held in that region.
  • Your provider still sees the prompt. The gateway forwards each call to the provider account you configured. That provider and account decide where the prompt is processed. The organization’s residency region is not checked on the gateway’s calls. For EU processing, choose a provider endpoint and account that process in the EU.
  • Keys are fetched for the call, then forgotten. The gateway fetches your provider key when it needs it. It keeps the key in memory for at most an hour per organization and provider, and never writes it to a log. A rotated key is fetched fresh, so the old key is not used again from the cache.
  • One organization cannot read another’s policy. The gateway caches each organization’s policy under a hash of that organization’s own key. A test runs two organizations through the shared cache and checks that neither gets the other’s bundle.
  • Built small. The gateway is written in Rust and uses rustls for TLS, with no OpenSSL. It ships as a distroless, non-root container image with digest-pinned base images.
  • Hosted service. Today the gateway is part of the shared SaaS service, the only deployment mode in production. Self-hosted and customer-VPC deployments are a conversation, not a shipped option.

Trust Center · Service provider inventory · Security

Technical details

What the gateway records, and where it stops

What reaches the audit trail
A gateway call reaches the audit trail only when the gateway blocks it or requires a step-up. An allowed call is not written there, and neither are the gateway’s redactions and warnings. Usage still shows in the workspace by application and organization, after a processing delay.
Streaming
With no blocking output rule or tool policy, the response streams through byte for byte. With one, the gateway buffers the full response and checks it before releasing any bytes, so the first token arrives later.
Stopping one agent
Once an agent is quarantined or revoked, the gateway keeps accepting its application key until the next policy refresh, up to a minute. Requests already running finish, and a streamed response can last up to 10 minutes.
Output ceiling
A generation request with no output-token limit gets a default of 4,096 tokens. Set your own limit if you need longer answers.
Private endpoints
The hosted gateway refuses private and loopback addresses. A self-hosted Ollama or other model server must be reachable at a public address. An Azure Private Link endpoint is not reachable; use the resource’s public endpoint.
Custom endpoints
Routing to your own OpenAI-style endpoint requires Team Advanced or Enterprise, plus enablement for your deployment.
Measured overhead
From our own release measurement: p50 0.332 ms · p90 0.927 ms · p99 3.39 ms. Measured on pure passthrough with control-plane and model governance excluded, on one host, against an in-process upstream, over 3,000 iterations. Treat it as the floor the proxy adds, not a prediction for your network.

Read the docs: model gateway quickstart · guardrails · policies · API reference. Then: MCP gateway · runtime security · architecture.

FAQ

Common questions about the model gateway

The OpenAI SDK works against the /openai/v1 path for chat completions, completions, embeddings and the Responses API. The Anthropic SDK works against /anthropic/v1 for the native Messages API, which the gateway passes through without translating. Each other supported provider has its own path prefix on the same host; the quickstart shows the OpenAI and Anthropic base URLs.

Praesidia stores each provider key encrypted with AES-256-GCM. The gateway fetches the key only to make a call, keeps it in memory for at most an hour, and never logs it. Your applications never hold the provider key. They send a Praesidia application key, which the gateway removes before forwarding.

The gateway runs in AWS eu-central-1 (Frankfurt), and gateway connections terminate TLS there. After that, the call goes to the provider account you configured, and that provider decides where the prompt is processed. Choose a provider endpoint and account that process in the EU if you need EU processing.

Governed calls are refused rather than forwarded unchecked. The gateway also refuses to start in a mode that would let them through. If a provider is down, only the calls to that provider fail.

Yes. Without a blocking output rule or tool policy, the response streams through byte for byte. With one, the gateway checks the full response before releasing it, so the first token arrives later. For streamed chat and completions, the provider is asked to include usage, so the call is metered from the provider’s own token counts.

Go deeper

Send one model request on your own key

Add a provider key, point your SDK at the gateway, and read the usage it records.

Route your first model call Book a demo