Half the monthly GPU cost on a production LLM deployment

On a production LLM inference workload, Feniria cut monthly GPU cost by 50% and p99 time to first token by 48%, with no hardware added. Throughput rose 9% and end-to-end latency fell 37% on the same GPUs.

This is a representative inference deployment from our 2026 field analysis. The provider is anonymized.

The problem: utilization is not productivity

AI infrastructure tends to scale faster than teams can use it well. Most GPU fleets run well below productive capacity.

Standard dashboards report high utilization while productive compute stays low. That gap leads to hardware purchases, energy draw and carbon load that could have been avoided.

What we found

The deployment’s standard dashboard reported high GPU utilization. Hardware-level telemetry from Feniria told a different story.

During normal traffic, SM Active (the fraction of streaming multiprocessors doing productive compute work) was in single digits. Roughly 90% of processing capacity was idle at those moments. Across the full measurement window, including peak load, an average of 60% of available core hours were idle or unproductive, peaking at 82%.

What we changed

Feniria read utilization, throughput, latency and hardware-level metrics together, and used that picture to optimize the LLM serving engine and scale the deployment’s concurrency.

The goal throughout was low-level optimization: more productive work out of the GPUs already running, not more hardware.

50% lower monthly cost for the same workload

Baseline: a single g6e.48xlarge on AWS. Feniria found unused capacity and optimized the deployment.

Metric	Baseline	Feniria optimized	Result
Total monthly cost	$21,996	$10,998	50% lower cost
Waste	89%	20%	Idle capacity, over-provisioning and misconfiguration, now minimal and managed
Productive	11%	80%	More output, better usage, right-sized

Measured improvements

These are the metrics that showed the gap between reported utilization and actual productive work, and then closed it.

Metric	Change	Impact
p99 time to first token	−48%	Faster first response
Inference throughput	+9%	More tokens per second
End-to-end latency	−37%	Shorter request completion
Core idle time (proxy via core-hours)	−19 pp	Less unproductive capacity
Total energy (active + idle), same workload	−27%	Less kWh for the same inference load

Core idle time is (1 − SM Active), averaged across the full measurement window, including idle and load periods.

In real-world terms: a 100-GPU fleet

For a representative 100-GPU H100 fleet running the same workload 24/7, this deployment’s measured +19% capacity gain and −27% energy reduction translate into:¹

• ~€85,000 per month of cloud compute value. The same as running 19 extra GPUs for a full month, on the fleet you already have.²
• ~32 MWh per month of energy saved. 27% less energy for the same workload.³
• ~9.4 t CO₂ per month avoided. About 112 t CO₂ a year at the 2024 EU average, what ~5,100 mature trees absorb in a year.⁴⁵

Footnotes

1. Illustrative example for a representative 100-GPU H100 fleet running 24/7. Cloud compute value uses this deployment’s measured +19% effective capacity gain. Energy and CO₂ use the measured −27% energy reduction for the same workload. ↩

2. GPU cost: €6.20/GPU-hr (~$6.88 AWS on-demand). $6.88 is AWS p5.48xlarge (8× H100 at $55.04/h) divided per GPU. Operating hours: 720/month (30 days × 24 h). ↩

3. Power draw: 1.67 kW per GPU (700 W TDP × 1.7 overhead × 1.4 PUE). Baseline fleet: 100 × 1.67 kW = 167 kW, or 167 kW × 720 h = 120.24 MWh/month. Energy saved at −27%: 0.27 × 120.24 ≈ 32.46 MWh/month, shown as ~32 MWh. ↩

4. Grid carbon intensity: 288 kg CO₂/MWh, 2024 EU aggregate consumption mix (Fraunhofer ISE). Ember’s 2024 EU generation intensity is ~213 g CO₂/kWh; 288 is the more conservative consumption-mix factor. Avoided CO₂: 32.46 MWh × 288 kg/MWh ≈ 9.35 t/month, shown as ~9.4 t (≈112 t/year). ↩

5. Tree equivalence: ~22 kg CO₂ per mature tree per year (US EPA / Arbor Day Foundation). Annual avoided CO₂ of 112,200 kg ÷ 22 kg/tree ≈ 5,100 trees absorbing that amount in one year. ↩