LLM Gateway Overview

The LLM Gateway is the service on the Agent Gateway’s data path. Every request an agent sends through Viper lands here: the gateway verifies the request’s signature, runs the plugins your organization and project have configured, forwards the request to the model provider, and records the exchange. The Control Plane is the management path: it mints registration tokens, publishes plugins and configurations, and serves the audit history the gateway records.

Viper is the gateway’s main client, and most users never call the gateway themselves. Its API matters to operators checking its health, and to integrators building their own signer. Stored blobs and state history are read through the Control Plane API.

The LLM Gateway is part of the gateway stack, which is optional and disabled by default. It is served at the root of its own host, such as https://gateway.example.com. This is the address Guardian shows in agent onboarding commands and Viper uses, followed by /v1, as llm_gateway_api_url.

What the LLM Gateway Owns

• The trusted agent registry: each agent’s registered signing keys (DIDs) and their registration events, checked against the agent status the Control Plane sets
• Agent registration, with a registration token from the Control Plane or an API key, and the registration credentials it issues and renews
• Signature verification for every agent request, with freshness checks on all of them and replay checks on provider and relay requests
• The provider routes to Anthropic, OpenAI, and the ChatGPT backend, and the relay route for an agent’s other HTTP traffic
• Running the plugins in the active global and project configurations at each hook stage
• Writing audit exchanges, and removing them once they pass the retention period
• The CID-addressed blob store for state snapshots, attestations, and credentials, and each agent’s state history

What the LLM Gateway Depends On

The gateway keeps everything in the gateway database, the PostgreSQL database it shares with the Control Plane. It creates the tables and applies their migrations when it starts. It does not call the Control Plane at runtime; the two agree through that database and a few shared settings, such as the registration token secret.

It calls Auth Service only to check an API key an agent registers with, and only when API-key enrollment is configured. With plugins on, it downloads signed plugin artifacts from object storage and reloads the active configurations from the database. It forwards allowed requests to the provider each route names.

The gateway fails closed on identity: a request whose signature or signing key cannot be verified is denied and never reaches the provider. Audit capture does not block traffic. Exchanges queue on the pod’s disk while the database is unavailable, and once the queue is full, new records are dropped rather than refusing requests.

Where to Go Next

• Use the Agent Gateway guides for end-user workflows.
• Use the LLM Gateway API reference for exact request and response shapes. A running gateway also serves its own spec at /openapi.json.
• Use Chart Values to enable the gateway stack.

Related

• LLM Gateway Architecture — how it works with Viper, the Control Plane, and the platform services
• LLM Gateway Configuration — the settings an operator must provide
• LLM Gateway Operations — health checks, upgrade order, and symptom checks
• Gateway Basics — one request through the gateway, step by step
• Control Plane Overview — the management path for the gateway