Here’s a number that should stop any IT budget conversation cold: enterprise GPU fleets are running at roughly 5% average utilization, while Gartner projects $401 billion in new AI infrastructure spending for 2026 alone. Meanwhile OpenAI just raised its own infrastructure commitment to $750 billion through 2030. Those two facts sitting next to each other are not a coincidence. They’re a warning. If you’re the person signing off on GPU capacity this year, the question is no longer whether you can get access to compute. It’s whether you can justify what you already bought.
The utilization gap nobody puts in the budget deck
A separate VentureBeat survey found 86% of enterprises running GPUs report utilization at half capacity or less, with nearly half sitting at 25% or below. That’s not a rounding error. That’s the majority of a capital-intensive procurement line doing almost nothing most of the time.
For an engineering leader, this is the gap between the number on the slide that justified the purchase and the number a dashboard would show if anyone built one. GPU capacity gets approved based on peak-demand projections and competitive anxiety, then billed every month regardless of whether a single job touched it last week.
Why over-provisioning became the default answer
VentureBeat’s reporting on this pattern points to a simple asymmetry: engineers routinely request five to ten times the resources they actually need, because under-provisioning pages someone at 2 a.m. and over-provisioning just adds a line to a cloud invoice nobody reads closely. One failure mode is visible and career-limiting. The other is invisible and diffuse.
Layer onto that the contracts signed during the GPU shortage years, many locked into three- to five-year depreciation schedules, and you get fixed costs that can’t flex down even after the panic buying stops. Budget planning built for scarcity doesn’t self-correct once scarcity eases. Someone has to go back and rewrite the plan.
Hybrid infrastructure is becoming the pragmatic default
Part of the fix is architectural. Google Cloud’s State of AI Infrastructure report, based on a survey of more than 1,400 senior IT leaders, found 52% of organizations now run a hybrid-cloud approach to AI: public cloud for elastic, bursty workloads, private or regional infrastructure for anything touching regulated or sensitive data.
This isn’t a compromise position. It’s a recognition that data sovereignty and compliance requirements draw hard lines around where certain workloads can run, while everything else benefits from elastic capacity you only pay for when you use it. Planning infrastructure as one monolithic decision — all public cloud or all on-prem — is exactly the kind of thinking that produces 5% utilization in the first place.
Specialized AI clouds: small footprint, fast-growing option
The neocloud providers (CoreWeave, Lambda, Crusoe and similar GPU-focused platforms) still show up in a small slice of enterprise stacks today. One VentureBeat infrastructure survey put CoreWeave and Lambda at 3.5% of current stacks each, with the rest of the field below that. But adoption intent tells a different story: the same data shows AI-specialized clouds as the top category enterprises plan to evaluate next, at 44%, with the strongest net-momentum score of any infrastructure option tracked.
A companion VentureBeat data point sharpens the trend: specialized AI cloud adoption climbed from 30.2% to 35.9% in a single quarter, while “GPU availability” as a provider-selection factor fell from 20.8% to 15.4% over the same period. Enterprises are worrying less about getting access to chips and more about not paying for chips they don’t use. Even chipmakers are reading the shift — Groq just raised $350 million to pivot from selling AI chips to operating a neocloud itself.
For a capacity planner, specialized AI clouds function as a release valve: shorter commitments, workload-specific pricing, and none of the five-year depreciation math that turns a bad forecast into a permanent budget line. They won’t replace hyperscalers or on-prem for every workload, but they give teams a third option between “commit for years” and “build it ourselves.”
What responsible AI infrastructure planning looks like now
- Measure utilization before approving new capacity — most organizations still can’t answer this basic question.
- Segment workloads by data sensitivity first, then decide public cloud, specialized AI cloud, or on-prem for each segment.
- Favor shorter commitments and specialized providers for bursty or experimental workloads instead of defaulting to multi-year hyperscaler contracts.
- Treat over-provisioning as a tracked cost, not an invisible one — put it on the same dashboard as outages.
The real risk isn’t running out of GPUs
The infrastructure conversation of the last two years was about scarcity: who could get chips, how fast, at what price. The conversation enterprises need to have now is about waste — 95% of provisioned capacity sitting idle isn’t a technical footnote, it’s a budget line that keeps growing every quarter it goes unmeasured.
Before signing the next multi-year GPU contract, ask whether anyone can show you last quarter’s utilization numbers. If the answer is no, that’s the project to fund first — not more silicon.