Save Money on AI Models: Dial In Your Settings, Then Route by Task

You're Probably Paying 2–5× More Than You Need To

After auditing dozens of production AI stacks this year, the pattern is depressingly consistent: teams pick a flagship model (usually GPT-5.5 or Claude Opus 4.7), wire it up with default settings, and route every request to it — classification, summarization, frontier reasoning, all of it. The bill is enormous, and 60–80% of it is waste.

The good news: you don't need a rewrite to fix this. Two levers — request settings and task-based model routing — typically cut spend by half or more in a single afternoon. Here's the playbook.

Part 1: The Settings That Quietly Burn Your Budget

Before you switch models, fix what you're sending. These are the per-request knobs that matter most, ranked by how much money they leak.

1. `max_tokens` — The Single Highest-Leverage Setting

Output tokens cost 5–10× more than input tokens on every major provider. GPT-5.5 charges $5/M input vs $30/M output. Claude Opus 4.7 charges $15/M input vs $75/M output. If you leave `max_tokens` unset (or at some lazy default like 4096), the model will happily ramble all the way to the cap on a fraction of requests — and you pay for every token.

Fix: Set `max_tokens` to the smallest number that fits your real use case. Classifying a ticket? 50 tokens. Generating a tweet? 100. Writing a structured JSON response? Compute the realistic ceiling and add 20%. This single change routinely saves 20–40%.

2. Use Structured Output Instead of "Please Return JSON"

Asking the model to "return JSON" in the prompt wastes tokens on schema reminders, validation retries, and the model's apologetic preamble ("Sure, here is the JSON you requested..."). Native structured output / JSON mode / tool-calling modes (available on OpenAI, Anthropic, Google, and DeepSeek) skip all of that. Shorter outputs, higher reliability, fewer retries.

3. Lower `temperature` for Deterministic Tasks

For extraction, classification, routing, and tool calls, set `temperature: 0` (or near zero). You get shorter, more consistent answers — which means fewer retries, smaller outputs, and prompt caching actually works because identical inputs produce identical outputs.

4. Turn On Prompt Caching — and Structure Your Prompts For It

Every major provider now offers cached input pricing at 5–10× lower than uncached. The catch: caching only works on the *prefix* of your prompt. If your system prompt + tool definitions + few-shot examples come first and your dynamic user input comes last, you cache the expensive part. If you interleave them, you cache nothing.

Audit your prompt structure today: long static content first, dynamic content last. Read Prompt Caching Explained for the per-provider implementation details.

5. Disable Reasoning Mode by Default

GPT-5.5, Claude Opus 4.7, and DeepSeek V4 Pro all support an extended reasoning mode. It's brilliant for hard problems and wildly expensive for easy ones — reasoning tokens are billed at output rates and can be 10–50× the visible answer length. Leave it off by default; turn it on per-request only when the task actually needs it.

6. Stream Only When the User Sees It

Streaming costs nothing extra, but it locks you out of batch pricing — which is ~50% off on every major provider. If a request isn't user-facing in real time (overnight reports, embedding jobs, eval runs), send it through the Batch API instead.

7. Cap Context, Don't "Just Stuff Everything In"

A 200K-token context call on GPT-5.5 costs about $1.00 in input alone. If your RAG pipeline retrieves 30 chunks "to be safe" when 5 would do, you're paying 6× for marginal quality gains. Re-rank, filter, and truncate aggressively — and measure quality before assuming bigger context is better. The Context Window Cost Trap covers this in depth.

Part 2: Stop Sending Every Task to the Flagship

The single biggest lever is task-based routing. Different models are radically different prices, and the cheap ones are *good enough* for most of what you're doing. Here's a practical tiering for April 2026:

Tier 1 — Frontier (use sparingly)

GPT-5.5 Pro · Claude Opus 4.7 · DeepSeek V4 Pro (reasoning mode on)

Use for: complex coding agents, multi-step planning, hard math, anything where a wrong answer costs real money.

Avoid for: anything else.

These should be 5–10% of your traffic, max.

Tier 2 — Workhorse (the bulk of production)

GPT-5.5 · Claude Sonnet 4.7 · Gemini 3 Pro · DeepSeek V4 Pro (no reasoning)

Use for: high-quality summarization, RAG answers, code review, content generation, customer-facing chat.

Strong quality, mid-tier pricing. This is 30–40% of your traffic.

Tier 3 — Cheap & Fast (the long tail)

GPT-5.4 Mini · Claude Haiku 4 · Gemini 3 Flash · DeepSeek V4 Flash

Use for: classification, intent detection, routing, structured extraction, RAG re-ranking, draft generation, eval grading, anything high-volume and well-defined.

This should be 50–65% of your traffic — and on most teams it's currently 0%.

Tier 4 — Nano (when you barely need a model)

GPT-5.4 Nano · Gemini 3 Flash Lite

Use for: simple classification, language detection, profanity filtering, sentiment, "is this question about X?" gating before you call a bigger model.

Part 3: A Routing Pattern That Just Works

The cheapest-and-still-good production pattern in 2026 is the cascade:

1. Classify the task with a Tier 4 model (~$0.05/M tokens). One call, sub-50ms, costs basically nothing.

2. Route by classification. Easy/known-pattern → Tier 3. Standard work → Tier 2. Hard/novel/high-stakes → Tier 1.

3. Verify on the cheap tier first. For ambiguous cases, run Tier 3 first; if confidence is low or the answer fails a validator, escalate to Tier 2 or Tier 1. You only pay for the flagship when you actually need it.

A real-world example: a support automation pipeline that previously ran every ticket through GPT-5.5 ($2,500/mo) was restructured as Nano-classify → Flash-draft → Sonnet-only-on-low-confidence. New bill: $280/mo. Same answer quality on a blind eval.

Part 4: A 30-Minute Audit You Can Do Today

Open your top three production prompts and check, in order:

1. Is `max_tokens` set tight? If not, fix it now. (Biggest single win.)

2. Are you using native structured output / tool calling? If not, switch.

3. Is `temperature` 0 for deterministic tasks?

4. Are static instructions at the *front* of the prompt (cacheable prefix)?

5. Is reasoning mode on by default? If yes, turn it off and gate it behind a complexity check.

6. Is this user-facing in real time? If no, move to batch.

7. Is this actually a Tier 1 task, or could a Tier 2 or Tier 3 model do it just as well? Run a 50-sample eval and find out.

Most teams discover that 3 of these 7 are wrong, and fixing them halves the bill in a week.

Run the Numbers Before You Migrate

The temptation after reading this is to rip out GPT-5.5 and put DeepSeek V4 Flash everywhere. Don't. The math is workload-specific, and "cheap model + 2× retries + worse user experience" can easily be more expensive than "right model, first try."

Plug your real token volumes into the Token Cost Calculator at each tier and compare side-by-side. The right answer is almost always a mix — flagship for the hard 5%, workhorse for the standard 30%, cheap-and-fast for the 65% of work that doesn't actually need a frontier model.

Pick one prompt today, fix the settings, route it correctly, and watch a line item shrink. Then do the next one.