Token prices collapsed 98% since late 2022 — from $20 per million tokens to roughly $0.40. Enterprise AI budgets didn’t follow. They grew from an average of $1.2M in 2024 to $7M in 2026, a 320% increase, according to The Next Web’s June 2026 analysis. The culprit isn’t the price of compute. It’s the consumption pattern. And fixing it is now one of engineering’s most practical cost levers.
Why cheaper tokens produce bigger bills
The math seems contradictory until you understand how agentic workflows change the cost structure. A simple AI-assisted interaction in 2023 cost approximately $0.04. An orchestrated agentic system doing the same job in 2026 costs around $1.20 — a 30x jump driven by multi-step reasoning, context windows, tool calls, and parallel agents running without hard budget caps. Lower per-token prices got absorbed and then overwhelmed by higher volumes.
Uber’s case has become the clearest illustration. After onboarding 5,000 engineers on Claude Code in December 2025, adoption climbed from 32% to 84% of the engineering org. Individual engineers were spending $500–$2,000 per month on tokens. The entire 2026 AI coding budget was exhausted by April. Approximately 70% of committed code already originates from AI; roughly 10% of live backend updates shipped with no human oversight. A Gartner forecast projects worldwide AI spending at $2.59 trillion in 2026 — a 47% year-over-year increase — and warns that G1000 organizations face up to a 30% rise in underestimated infrastructure costs by 2027.
The forecasting failures are systemic. According to Mavvrik AI’s 2026 cost statistics, 80–85% of enterprises miss AI infrastructure cost forecasts by more than 25%. One in five miss by more than 50%. Only 6% achieve payback within one year of AI investment. Less than 1% report ROI improvements of 20% or greater. The cause is organizational: teams optimized for static, license-based cloud procurement are trying to manage dynamic, consumption-based AI costs with the wrong tools.
The FinOps Foundation’s answer: treat AI like cloud infrastructure
The FinOps Foundation’s AI workload framework defines specific KPIs for AI cost governance that traditional cloud FinOps never needed: Cost Per Inference, Cost Per Token, Training Cost Efficiency, and Resource Utilization Efficiency. The maturity model maps to three stages: Crawl (prototyping with basic tagging), Walk (automated cost tracking per team/project), and Run (showback/chargeback at the model and workload level with optimization loops in place).
The Foundation’s State of FinOps 2026 survey — covering 1,192 practitioners managing $83B+ in annual cloud spend — found that 98% of organizations now actively manage AI spend, up from 31% two years ago. AI cost management became the #1 skillset FinOps teams want to acquire. Practices with C-suite engagement show 2–4x more influence over AI technology decisions, and 78% now report directly to the CTO or CIO.
The engineering levers that actually move the needle
Four cost optimization techniques have demonstrated measurable, repeatable results across enterprise AI workloads:
- Model routing with tiered allocation: Rather than routing all traffic to a premium model, a 70/20/10 split (budget model / mid-tier / premium) reduces average per-query cost by 60–80% with minimal impact on output quality for routine tasks. Premium capacity is reserved for complex, high-stakes inference where it actually justifies the spend.
- Prompt caching: According to Finout’s 2026 OpenAI pricing analysis, Prompt Caching cuts input token costs by up to 75% — from $0.40/million tokens to $0.10/million for cached inputs on GPT-4.1 Mini. For high-repetition enterprise workflows (repeated context, system prompts, reference documents), the savings compound quickly at scale.
- Batch API and spot instance scheduling: Batch processing delivers a flat 50% discount across most major providers. Spot GPU instances on AWS, Azure, and GCP run $1.95–$2.50/GPU-hour versus $3.00–$6.98/hour on-demand, according to CloudZero’s 2026 GPU pricing comparison — a 60–70% reduction for workloads tolerant of interruption. GCP cut its compute pricing an additional 8% across all regions in Q1 2026.
- Tagging and showback before chargeback: Cost attribution at the team, project, and use-case level is the prerequisite for everything else. Without tagging (Project, Team, CostCenter, UsageType), optimization is guesswork. Showback dashboards — making teams visible to their own token spend — consistently produce behavioral change before any formal chargeback mechanism is needed.
GPU utilization adds another layer. Average GPU utilization in enterprise AI environments runs as low as 5%, according to VentureBeat’s infrastructure analysis. Thirty to fifty percent of AI-related cloud spend evaporates into idle or overprovisioned capacity. Right-sizing compute — including model distillation and quantization for production inference — often cuts infrastructure costs significantly without measurable accuracy loss.
The ROI reality check
The honest picture: most organizations aren’t achieving meaningful AI ROI yet. 60% report minimal or no material value from current AI investments. Forty percent of time savings attributed to AI are lost to employees fixing AI errors — a rework tax that rarely appears in productivity dashboards.
The exceptions exist and they’re specific. GitHub Copilot remains one of the few tools with documented, repeatable ROI: per Keyhole Software’s 2026 AI cost survey, a 34% reduction in development effort translates to roughly six hours per engineer per week, generating approximately $1M in savings annually per 100 developers — at a licensing cost well below that. The pattern in high-ROI deployments is consistent: narrow use case, clear metric, governance in place before scale.
Infrastructure discipline is the real competitive differentiator
The organizations winning on AI cost in 2026 aren’t necessarily using different models or different providers. They’re applying the same discipline to AI spend that mature engineering organizations applied to cloud spend a decade ago — tagging, budgeting, observability, tiered procurement. The FinOps Foundation’s maturity model exists precisely because these practices don’t emerge organically; they have to be deliberately built.
Token prices will continue to fall. Consumption will continue to rise faster. The teams that establish cost governance at the Walk stage — before their AI workloads reach Run-level complexity — are the ones positioned to sustain the investment rather than cancel it. Gartner’s prediction that 25% of planned 2026 AI budgets will slip to 2027 is a deadline, not a statistic. The infrastructure work that prevents that slip starts before the next procurement cycle, not after.