Analysis · 2026-04-30 · 11 min read

The Pulse: AI Token Spending Out of Control — What's Next?

Inside 15 tech companies seeing token spend grow 5–40× year-over-year, the FinOps responses they're rolling out, vendors that can't keep up with demand, and plummeting morale at Meta.

The Pulse: AI Token Spending Out of Control — What's Next?

Two years into the agent era, the line item nobody planned for has become the loudest one on the P&L. Across the 15 tech companies we spoke to for this issue — ranging from a 40-person YC seed startup to two FAANG-scale platforms — monthly token spend has grown between 5× and 40× year-over-year, and almost nobody projected it correctly. CFOs who waved through "$50K/month for AI experiments" in Q2 2025 are staring at $2M+ runs in Q2 2026 and asking the same question in fifteen different boardrooms: *what is this actually buying us, and how do we make it stop growing faster than revenue?*

This week's Pulse covers what we heard: the shape of the spend explosion, the FinOps playbook that's emerging, the supply-side bottlenecks at the major vendors, and the rapidly worsening morale picture at Meta as Llama's commercial story keeps getting pushed out.

1. The Shape of the Explosion: It's Agents, Not Chat

When we asked engineering leaders to break down where the new tokens are going, the answer was nearly unanimous: it isn't end-user chat anymore — it's internal agents.

A representative breakdown from a mid-sized SaaS company (~$80M ARR) that gave us last month's invoice:

Two patterns repeat across every company we talked to:

Coding agents are now the single largest line item. Multiple companies told us their Cursor + Claude Code spend alone exceeds their entire 2025 OpenAI bill. Each developer now generates 3–10× more tokens per day than the heaviest end user of their actual product.
Background agents are the fastest-growing segment. "Always-on" loops — ticket triage, PR reviewers, doc refreshers, eval graders — were ~4% of spend a year ago and are now north of 20%. They run while everyone sleeps and they don't stop.

The combined effect: the cost-per-employee of AI tooling at the companies we surveyed has gone from ~$80/month to $900–$1,400/month in twelve months. At a 500-person eng org, that's $5–$7M/year in just internal agent spend.

2. What 15 Companies Are Actually Doing About It

We grouped the responses by company stage. The interventions cluster cleanly.

Seed / Series A (5 companies)

The smallest teams have the simplest playbook because they have no leverage to build platforms:

Cap-and-route: a hard monthly dollar cap per developer, with auto-routing to a cheaper tier (DeepSeek V4 Flash, Gemini 3 Flash, or GPT-5.4 Nano) once 70% of the cap is consumed. Three of five mentioned this as their #1 lever.
Aggressive prompt caching: every team had turned this on. The two who hadn't were leaving an estimated 40–60% of their bill on the table.
Kill background agents on weekends. Sounds trivial, saved one team $11K/month.

Series B–D (6 companies)

The middle of the market is where the real FinOps muscle is being built:

A dedicated "AI cost" engineer or small team. Five of six now have at least one named owner. Twelve months ago, zero did.
Per-team budgets with weekly Slack-bot rollups. When the cost is visible, it stops growing. One platform told us "the week we shipped the cost dashboard, spend dropped 18% with no other change."
Cascade routing in production. Every company had moved off "default to flagship for everything." The cheapest model that passes a confidence check answers; only ~10–25% of traffic ever reaches a Tier 1 model. Read Save Money on AI Models with Smart Settings + Routing for the pattern.
Batch API for everything async. Eval runs, nightly doc regeneration, ticket backfills — all moved to Batch API for the standard ~50% discount.

Late-stage / public (4 companies)

The biggest companies are doing the most expensive thing: building their own gateway.

All four have built an internal "model gateway" that sits in front of OpenAI / Anthropic / Google / a self-hosted DeepSeek or Llama deployment. It does routing, caching, budget enforcement, audit logging, and per-team chargeback in one place.
Two have started moving large background workloads to self-hosted open-weight models (DeepSeek V4 and Llama 3.3 70B on H200s). Per-token costs are 8–15× lower; engineering cost to run them is non-trivial. The payback period the FinOps teams quoted was 5–9 months.
All four are renegotiating commits quarterly, not annually. With provider prices dropping 30–50% twice a year, an annual commit signed in Q4 2025 now looks expensive.

3. The Vendor Side: Supply Can't Keep Up

The other story we kept hearing — and this one will sound familiar to anyone who's tried to scale a workload past a few thousand requests per minute — is that the major vendors cannot keep up with demand.

Specifically, in the last four weeks:

OpenAI has tightened tier promotion criteria for new accounts and is pushing high-volume customers toward annual reserved capacity. Multiple teams reported being asked to commit to 6–12 month minimums to keep their existing rate limits.
Anthropic has hit Claude Opus 4.7 capacity limits during US business hours; one customer showed us a week of 429s clustered between 10am–2pm Pacific. Sales reach-outs explicitly mention "guaranteed throughput" tiers as a paid upgrade.
Google is the only major vendor with no visible supply pain — Gemini 3 Flash continues to absorb workload at the published rate limits — and unsurprisingly is winning the most net-new high-volume RAG and classification deployments.
DeepSeek API capacity has been spotty since its V4 Pro/Flash launch (see our launch coverage), pushing serious users toward self-hosted weights.

The strategic takeaway is the one nobody at the AI labs wants to say out loud: for the next several quarters, compute is the binding constraint, not model quality. The lab that ships the next benchmark king but can't serve it doesn't win the deal.

4. The Meta Story: Morale Is Rough

Almost separately — but it kept coming up — multiple sources at and adjacent to Meta described plummeting morale on the Llama and GenAI teams:

The Llama 4 launch slipped again and is now informally targeted for late summer 2026.
The internal narrative has shifted from "we're going to win open source" to "we're going to keep open source from being captured by DeepSeek."
Several senior researchers have left in the last quarter for Anthropic, an OpenAI competitor's frontier lab, and one well-funded stealth startup.
The recent re-org concentrating GenAI under a new SVP is widely read internally as "one more shot."

None of this is a death knell — Meta still has the GPU base, the data, and the ad-business cash flow to fund another two years of catch-up — but the contrast with the Anthropic and Google momentum is stark, and the people building Llama feel it.

5. Where Spending Goes Next: Three Predictions

Based on the conversations and the published vendor moves, here is where we think token spend curves over the next 6–12 months:

Prediction 1: The "agent tax" plateaus by Q4 2026

Coding agent spend is currently growing roughly 15–20% month-over-month at the companies we surveyed. That can't continue — every FinOps team we spoke to has a hard ceiling baked into 2026 budgets. Expect aggressive routing, per-developer caps, and self-hosted DeepSeek/Llama for the long-tail of background work to flatten the curve by year-end. Companies that *don't* build this muscle will be the headlines in Q1 2027 earnings calls.

Prediction 2: Self-hosting crosses the chasm

Twelve months ago, "we self-host Llama" was a flex from two or three exotic infra teams. Today, four of the fifteen companies we surveyed self-host at least one workload, and another six told us they have a serious 2026 plan to do so. By the end of 2026 we expect self-hosted open-weight models to handle 30–40% of background and high-volume workloads at companies above ~$100M ARR. See Open Source Models Cost Analysis for the math.

Prediction 3: "Token budgets" become a standard product feature

Right now, FinOps for AI is mostly hand-rolled — Slack bots, internal dashboards, gateway services. By the end of the year we expect every major model provider and every cloud to ship first-class budget, alerting, and per-team chargeback features as table stakes, mirroring what AWS Cost Explorer is for raw infra. The teams that resist building their own gateway today will get rescued by this in 2027.

6. What to Do This Week

Concrete, low-effort moves that came up repeatedly as "wish we'd done it six months ago":

1. Turn on per-team usage alerts. Most teams discovered the explosion only when finance forwarded an invoice. Daily Slack rollups change behavior immediately.

2. Run the Agent Loop Cost Estimator on your three most-used internal agents. Most teams find one loop running on a flagship model that should be on Tier 3 — single biggest one-day win.

3. Audit `max_tokens` per role. Default 4096s on tool-call steps are silently doubling agent bills. See Cost-Efficient AI Agents for per-role recommendations.

4. Move every async workload to Batch. ~50% off, no quality difference, no latency impact for jobs that already run overnight.

5. Renegotiate any commit older than one quarter. Prices have moved enough that Q4 2025 contracts are actively expensive. Vendors will renegotiate to keep volume.

If you do all five before next Pulse, the typical reported savings band is 35–55% on monthly token spend with no loss in quality. That's not a clever trick — it's just what FinOps looks like when it grows up.