Portkey's public category is a managed AI gateway: a single API in front of multiple model providers, with virtual keys, budgets, rate limits, caching, and observability shipped as built-in product features rather than assembled configuration. It answers "how do we route, control, and observe LLM traffic across providers without operating that infrastructure ourselves." Teams evaluating alternatives are usually deciding along one axis — how much of that infrastructure they want to run themselves versus hand to a managed vendor — and along a second, less obvious axis: whether their actual traffic is LLM-provider calls, agentic tool-calling traffic, or both.
What a managed AI gateway is built for
An AI gateway sits between applications and model providers, giving a platform team one place to manage provider credentials, enforce spend and rate limits, cache repeated requests, and observe traffic across every team calling into it. Portkey's public materials describe this as a managed, product-first version of that layer: RBAC, SSO, budgets, and compliance-oriented controls presented as configurable features rather than infrastructure you build and patch yourself. That is a meaningful difference from a self-hosted approach — the operational burden of running and upgrading the proxy moves to the vendor, in exchange for a dependency on that vendor's roadmap and data-handling posture.
This category exists because provider-key sprawl and inconsistent per-team cost tracking are real, common failure modes once more than a couple of teams start calling model providers independently. A gateway consolidates that into one control point, which is valuable in direct proportion to how many teams and providers you actually have — a single team calling a single provider usually doesn't need a gateway in front of it at all.
What a gateway does not, by itself, cover
An AI gateway's traffic is LLM-provider calls: prompts and completions moving between an application and a model provider. Agentic systems introduce a second, distinct traffic type — tool calls an agent makes to MCP servers and other tools, often carrying different payloads, different authorization needs, and different risk (a tool call can execute an action, not just return text). Whether a given gateway's routing, budget, and observability features extend meaningfully to that second traffic type is a question to ask directly rather than assume from "AI gateway" branding — the two flows are covered in more depth in MCP gateway comparison 2026 and what to look for in an MCP gateway. A gateway strong on LLM-provider routing and weak on tool-call authorization is answering half of an agentic estate's traffic-governance question, not the whole thing.
Evaluation lenses that actually separate gateway options
- Operating model. Fully managed cloud, self-hosted open source, or an extension of an API gateway you already run — each trades operational burden for control differently, covered generally in self-hosted vs. managed AI governance.
- Enforcement vs. tracking on spend. Confirm whether a budget limit actually stops further calls once breached, or only reports spend after the fact — the difference determines whether you have a control or a dashboard, detailed in budgets vs. rate limits.
- Traffic scope. LLM-provider calls only, or also MCP/tool-calling traffic. Agentic estates need both covered, whether by one system or two.
- Provider breadth and consistency. Does policy (budgets, rate limits, logging) apply uniformly across every provider routed through the gateway, or does coverage vary by provider?
- Key and credential handling. How are provider credentials stored and rotated, and does the gateway reduce the number of places raw provider keys exist in your estate — see securing LLM provider API keys for why key sprawl is a security problem independent of which gateway you pick.
Comparing approaches
| Approach | Operating model | Reach for it when |
|---|---|---|
| Managed AI gateway (Portkey and similar) | Fully managed cloud | You want governance (RBAC, SSO, budgets) as a product feature without operating infrastructure |
| Self-hosted open-source gateway | You run and patch it | Full data-path control and no per-seat cost matter more than operational simplicity, see LiteLLM enterprise alternatives |
| Extension of an existing API gateway | Platform you already operate | You already run a general-purpose API gateway and want to add AI-specific routing to it |
| Edge-native managed gateway | Managed, tied to an edge network | Your traffic already flows through that edge network |
When a different category is the right answer
If your question is "how do we route and control LLM-provider traffic across teams without running our own proxy," a managed AI gateway is the right category, and Portkey's public positioning fits that question directly — evaluate it and the self-hosted and platform-extension alternatives above against your actual team count, provider count, and on-call capacity, per the LLM gateway comparison 2026. If your question has expanded to "we also need to authorize and audit what our agents do when they call tools, not just track what they say to a model," that is a broader agent-governance requirement that a pure LLM gateway is not built to answer end to end — it needs an identity and authorization layer over tool-calling traffic specifically, which is a different, complementary system. Many agentic estates run an LLM gateway for provider traffic and a separate governance layer for tool-calling traffic, rather than expecting one system to do both.
Migration and rollout considerations
Switching or adding a gateway is rarely a clean cutover, and the migration cost is concentrated in specific places worth planning for up front. If application code currently calls a model provider's SDK directly rather than through a stable internal interface, every call site needs to change, which is a larger project than swapping a configuration value. Teams that already route through some abstraction — even a thin internal wrapper — find the migration cost concentrated in gateway configuration and provider credential re-issuance instead, which is considerably smaller.
Rollout order also matters more than it first appears. Moving every team and every provider onto a new gateway simultaneously maximizes the blast radius of a misconfiguration; a staged rollout — one team, one provider, then expanding — surfaces integration issues (unexpected latency, a provider-specific feature the gateway doesn't pass through cleanly, a logging gap) while the cost of a mistake is still small. The same staged-rollout logic that applies to guardrail changes, described in staged guardrail rollout and shadow mode, applies just as well to a gateway migration: run the new path in parallel, observe it against production traffic, and only cut traffic over once its behavior matches expectations under real load.
Finally, plan for the rollback path before you need it. A gateway that becomes a single point of failure for every team's model access needs the same operational rigor as any other production-critical dependency — health checks, a documented fallback (direct provider access, or a secondary gateway instance), and a clear owner for on-call response. High availability for an AI control plane covers the general reliability pattern that a gateway carrying meaningful production traffic needs to meet.
What good looks like
- You know exactly how many teams and providers route through your gateway today, and that number, not brand preference, is driving the managed-vs-self-hosted decision.
- Budget limits are enforced, not just reported — verified by testing that a call is actually blocked once a limit is crossed.
- You have a documented answer for tool-calling traffic, whether that's the same gateway, a dedicated MCP gateway, or an explicit decision that it's out of scope for now.
- Provider credentials live in one place, not scattered across every application that calls a model directly.
- A proof-of-concept ran real traffic through the candidate, not just a routing and latency benchmark, before committing to a migration.
Common questions
What is Portkey known for? Per its public positioning: a managed AI gateway providing a unified API across model providers, with RBAC, SSO, budgets, rate limits, caching, and observability delivered as built-in product features.
Is an AI gateway the same thing as an agent governance platform? No. A gateway's core job is managing LLM-provider traffic — routing, keys, cost, reliability. Agent governance additionally covers per-agent identity, tool-call authorization, and audit evidence across agentic actions, which is a broader and partially distinct surface.
When does a team not need a gateway at all? When a single team calls a single model provider directly, a gateway typically adds operational overhead without a corresponding benefit — direct SDK integration is simpler. The case for a gateway strengthens as team count, provider count, and traffic volume grow.
Does a managed gateway remove the need to think about data residency? No — it changes where that responsibility sits. Confirm any managed vendor's current documentation on data handling and residency directly, especially if your traffic includes regulated data; see data residency for AI agents for the questions to ask.