Guide · 2026-09-11 · 10 min read

How an AI Gateway Stops Token Theft: Usage Monitoring and Per-Request Verification

An AI gateway is the one chokepoint where every model call can be measured, authenticated and capped. Here is what real-time usage monitoring and per-request verification look like in production — and what the gateway itself costs.

TL;DR

A gateway is a chokepoint, not a dashboard. Every model call passes through it, so it is the only place where spend can be measured and blocked in the same breath.
Usage monitoring turns theft from an invoice surprise into a same-hour alert: per-key spend velocity, model mix, and geography.
Per-request verification means every single call presents a short-lived, scoped credential the gateway checks against identity, model allowlist, rate limit and remaining budget.
The fee is small. Gateways charge single-digit percentages or a flat platform fee — see the AI gateway pricing comparison.

!OpenAI logo !Anthropic logo !OpenRouter logo

1. Why the Gateway Is the Only Place This Works

Provider dashboards tell you what you spent yesterday, aggregated by key. That is the wrong resolution for both cost control and security. You cannot see which feature spent it, which user triggered it, or whether the traffic came from your own infrastructure at all.

A gateway sits between your application and every provider. Because it terminates the request before the provider does, it can do four things a dashboard cannot:

1. Attribute the call to a user, team, feature and environment.

2. Authorize it against a budget and a model allowlist.

3. Reject it — a 402 or 403 that costs nothing instead of a token bill.

4. Record it with tokens, latency and cost while the request is still in flight.

The moment raw provider keys exist in more than one service, that chokepoint is gone. Consolidating the keys into the gateway is the architectural half of the fix; the rest is what you do with the traffic that now flows through it.

2. Usage Monitoring That Catches Theft, Not Just Cost

Cost dashboards are built for finance. Theft detection needs different signals, sampled far more often.

Signals worth alerting on

The last one matters more than people expect: stolen credentials are usually tested before they are exploited. A cluster of malformed requests against a key that has never failed is often the earliest signal you will get.

Make alerts cheap to act on

An alert that means "someone might be stealing from you" but requires an hour of investigation will be ignored by the third week. Wire each alert to a specific, reversible action: pause the key, drop its budget to zero, or restrict it to one cheap model. If the alert was wrong, restoring takes ten seconds.

Budget caps are a control, not a report

Set hard caps at three levels — per key, per team, per workspace — and set them low enough that the first breach is a nuisance rather than a catastrophe. A cap that has never fired is not evidence of safety; it is usually evidence that it was set too high to matter.

3. Per-Request Verification

Session-based thinking is the root of most AI credential incidents: authenticate once, then trust every call that follows. Attackers do not attack the login; they steal the artifact that outlives it.

Per-request verification inverts this. Each individual call is treated as untrusted until it proves four things:

Who is calling — a machine identity, not a shared secret pasted into an environment file.
What it may call — an explicit model allowlist, so a stolen credential cannot reach the expensive tier.
How much it may spend — the remaining budget for that identity, checked before forwarding.
How fast — a rate limit sized to the legitimate workload, not to the provider's ceiling.

Virtual keys and machine authentication

The practical shape of this is a virtual key: a credential the gateway mints, scoped to one service, with a TTL measured in hours and a budget measured in dollars. The provider key never leaves the gateway. When a virtual key leaks, you revoke one credential and nothing else changes.

Better still, skip long-lived credentials entirely. Where your platform supports workload identity, let the service exchange its identity for a short-lived bearer token at start-up and refresh it. A token that expires in an hour is worth far less on a resale market than a key that never expires.

Ban the long-lived personal key

The single highest-leverage rule: no personal, long-lived API key in any production path. They outlive employment, get pasted into notebooks, and never show up in an inventory. Every production caller should hold a rotating, service-scoped credential issued through the gateway.

4. A Rollout That Takes a Week, Not a Quarter

1. Day 1 — Inventory. List every place a provider key exists. Repos, CI secrets, notebooks, serverless configs, that one contractor's laptop.

2. Day 2 — Route. Point one non-critical service at the gateway. Confirm logs, cost attribution and latency overhead (usually a few milliseconds).

3. Day 3 — Mint virtual keys. One per service, each with a TTL, a budget and a model allowlist.

4. Day 4 — Cap and alert. Hard caps at key, team and workspace level; alerts on spend velocity and model mix.

5. Day 5 — Cut over and revoke. Move remaining services, then rotate every raw provider key so old copies die.

6. Ongoing — Rotate. Automate rotation so no credential is older than your incident-response window.

5. What the Gateway Costs

Gateway pricing falls into three shapes:

Compare these against the loss they prevent. One uncapped leaked key running frontier output tokens flat-out can spend more in a weekend than a year of gateway fees. The full breakdown, vendor by vendor, is in the AI gateway pricing comparison.

6. What a Gateway Does Not Solve

Be honest about the boundary. A gateway defends the inference side. It does nothing about the other half of the threat model: an infostealer lifting the browser session cookie that keeps you signed in to a consumer AI account. That attack skips your password, your MFA and your gateway entirely — see browser session security for that front.

It also does not fix careless prompts, unbounded agent loops or a model choice that is simply too expensive for the job. It makes all three *visible*, which is usually the first step to fixing them.

Checklist

[ ] Raw provider keys exist in exactly one place: the gateway.
[ ] Every production caller uses a short-lived, scoped virtual key.
[ ] No personal, long-lived API key in any production path.
[ ] Hard spend caps at key, team and workspace level.
[ ] Alerts on spend velocity, model mix and failure bursts.
[ ] Each alert wired to a one-click revoke or budget-zero action.
[ ] Automated rotation shorter than your incident-response window.
[ ] Per-request model allowlists so a leak cannot reach the frontier tier.