5 Tactics That Cut Our LLM Costs by 50%

Why LLM Costs Spiral Out of Control

Most teams start with a single model for everything. That's fine at prototype scale, but once you hit production traffic, costs grow linearly — and sometimes super-linearly if prompts aren't optimized. Here are five concrete tactics we used to cut our monthly LLM bill from $12,000 to under $6,000.

1. Prompt Caching

Many providers now support prompt caching, where repeated system prompts or prefixes are stored and reused at a discount. OpenAI offers cached input tokens at 50% off, and Anthropic offers up to 90% off for cached prompts.

Impact: If your system prompt is 2,000 tokens and you make 500K requests/month, caching saves you roughly $1,250/month on GPT-5 alone.

How to implement: Structure your prompts so the static portion (system instructions, few-shot examples) comes first. Most providers cache automatically based on prefix matching.

2. Tiered Model Routing

Not every request needs your most powerful model. We built a simple classifier that routes requests to different tiers:

• Simple queries (FAQ, classification) → GPT-5 Nano ($0.10/M input)
• Standard tasks (summarization, extraction) → GPT-5 Mini ($0.40/M input)
• Complex reasoning (analysis, code generation) → GPT-5 ($2.50/M input)

Impact: 60% of our traffic turned out to be "simple" — routing those to Nano saved us $3,200/month.

3. Batching API Calls

Most providers offer 50% discounts on batch API calls that don't need real-time responses. If you're processing documents, generating reports, or running evaluations, batch endpoints are free money.

Impact: We moved our nightly content generation pipeline to batch, saving $800/month.

4. Output Length Control

The most expensive tokens are output tokens — they cost 2-4× more than input tokens. We added strict `max_tokens` limits and restructured prompts to request concise responses.

Before: "Analyze this data and provide a comprehensive report..."

After: "Analyze this data. Return: 1) Key finding (1 sentence) 2) Recommendation (1 sentence) 3) Risk level (low/medium/high)"

Impact: Average output dropped from 450 tokens to 120 tokens, saving $1,800/month.

5. Response Caching

For deterministic queries (same input → same output), we added a Redis cache layer. Product descriptions, FAQ answers, and classification tasks often repeat.

Impact: Cache hit rate of 35% on our classification endpoint alone saved $400/month.

The Compound Effect

None of these tactics alone is revolutionary. But combined, they reduced our bill by 52%. The key insight: treat tokens like any other cloud resource — measure, optimize, and right-size.

Use our Calculator to model how these optimizations would impact your specific workload.

Related Reading

• Prompt Caching Explained — Deep dive into how caching works across OpenAI, Anthropic, and Google.
• Batch API Pricing Guide — The 50% discount most teams ignore — and how to architect for it.
• API Pricing Strategies — Layer committed use and provisioned throughput on top of these tactics.
• The Context Window Cost Trap — Why stuffing your context window is one of the most expensive mistakes.