Kubernetes Cost Optimization at Enterprise Scale
Reduced Kubernetes infrastructure costs by 60% while improving reliability through intelligent resource management, spot instances, and workload optimization.
[Overview] System Overview
A B2B SaaS company running 500+ microservices on AWS EKS was spending $3M annually on Kubernetes infrastructure. Costs had grown 150% year-over-year while user growth was only 40%.
We implemented a comprehensive cost optimization strategy including right-sizing, spot instances, cluster consolidation, and workload scheduling. The result: 60% cost reduction while improving availability.
Key innovations included a custom Vertical Pod Autoscaler configuration that learned from production patterns, a spot instance strategy with intelligent fallback, and a cost allocation system that made teams accountable for their resource usage.
[Context] Situation before
The Kubernetes environment had grown organically over 3 years. Each team provisioned resources independently with no central oversight.
##### Over-Provisioning Culture
Developers requested 4x the resources they needed "just in case." CPU utilization averaged 12% across the cluster.
Memory requests were set to peak usage plus 50% buffer. Most pods used 30% of requested memory.
No one was accountable for cost. Teams optimized for "never getting paged" not efficiency.
##### Infrastructure Sprawl
12 separate EKS clusters across 3 environments. Each cluster had dedicated node groups.
Development and staging clusters were sized for peak load but ran 24/7.
No cluster auto-scaling. Node count was static, set for worst-case scenarios.
[Diagnosis] The real problem
The surface problem was over-provisioning. The standard solution would be to mandate resource limits.
But the real problem was incentive misalignment. Developers faced consequences for under-provisioning (outages, pages) but none for over-provisioning (just higher AWS bills).
Without changing incentives and providing visibility, any optimization would be temporary. Teams would drift back to over-provisioning within months.
Understanding the root cause changed everything.
[Boundaries] Working constraints
Kubernetes cost optimization must not compromise reliability.
Constraints are features, not obstacles. They guide design decisions.
[Decisions] Key tradeoffs
Kubernetes cost optimization requires balancing efficiency with reliability.
status="rejected" title="Mandate Resource Limits" description="Enforce strict resource quotas per team" reasoning="Would cause immediate reliability issues. Teams would fight the policy. Does not address root cause." /> status="rejected" title="Vertical Pod Autoscaler Only" description="Use VPA to automatically right-size pods" reasoning="VPA alone is not sufficient. Does not address node efficiency or spot instances. Partial solution." /> status="chosen" title="Comprehensive Optimization with Visibility" description="VPA + HPA + Spot + Cost allocation + Incentives" reasoning="Addresses technical and organizational factors. Provides sustainable improvement. Higher upfront effort but lasting results." />
[Design] Architecture design
The optimization architecture had four pillars: right-sizing (VPA with custom tuning), efficient scaling (HPA with predictive scaling), spot instances (with intelligent fallback), and cost visibility (per-team allocation).
A custom controller collected resource usage metrics and generated recommendations. Teams received weekly cost reports showing their usage and opportunities.
Spot instances handled 70% of workloads with automatic failover to on-demand during spot interruptions.
"type": "architecture", "title": "Kubernetes Cost Optimization Architecture", "description": "Multi-layer optimization with visibility", "components": [ "name": "Cost Controller", "type": "service", "description": "Central optimization" }, "name": "VPA", "type": "service", "description": "Vertical scaling" }, "name": "HPA", "type": "service", "description": "Horizontal scaling" }, "name": "Spot Manager", "type": "service", "description": "Spot orchestration" }, "name": "Node Groups", "type": "service", "description": "On-demand + Spot" }, "name": "Cost Allocator", "type": "service", "description": "Per-team attribution" }, "name": "Dashboard", "type": "user", "description": "Team visibility" } ], "connections": [ "from": "Cost Controller", "to": "VPA", "label": "Recommendations" }, "from": "Cost Controller", "to": "HPA", "label": "Thresholds" }, "from": "Spot Manager", "to": "Node Groups", "label": "Provision" }, "from": "Cost Allocator", "to": "Dashboard", "label": "Reports" } ], "boundaries": [ "name": "Optimization Layer", "contains": [ "Cost Controller", "VPA", "HPA", "Spot Manager" ] } ] }
{/* Diagram: data-flow - Spot Instance Failover */}
{/* ```json
{
"type": "data-flow",
"title": "Spot Instance Failover",
"description": "Automatic handling of spot interruptions",
"components": [
{
"name": "Spot Interruption",
"type": "external"
},
{
"name": "Spot Manager",
"type": "service"
},
{
"name": "On-Demand Pool",
"type": "service"
},
{
"name": "New Spot Pool",
"type": "service"
}
],
"connections": [
{
"from": "Spot Interruption",
"to": "Spot Manager",
"label": "2-min warning"
},
{
"from": "Spot Manager",
"to": "On-Demand Pool",
"label": "Failover"
},
{
"from": "Spot Manager",
"to": "New Spot Pool",
"label": "Reprovision"
}
]
}[Approach] Execution model
Optimization rolled out in phases over 3 months, starting with lowest-risk workloads.
Month 1: Instrumentation and baseline. Deployed cost allocation. Identified optimization opportunities.
Month 2: Right-sizing and spot instances. Started with development environments, then batch workloads.
Month 3: Production optimization. Gradual rollout with automatic rollback on reliability degradation.
[Results] Measurable outcomes
Infrastructure costs reduced from $3M to $1.2M annually-60% reduction.
Availability improved from 99.9% to 99.95%. Better resource allocation meant fewer OOM kills.
Team accountability achieved. Teams now actively monitor their costs and optimize proactively.
Spot instance adoption reached 70% of workloads with zero reliability impact.
title="Key improvements achieved:" items={[ "System handles 10x burst load without degradation", "Delivery latency reduced from seconds to milliseconds", "Zero message loss during traffic spikes", "Operational complexity reduced by 60%" ]} />
[Anti-patterns] What could have gone wrong
Without optimization, costs would have continued growing at 150%/year. Within 2 years, K8s spending would exceed $12M.
Simply mandating limits would have caused outages and team pushback. The policy would have been quietly circumvented.
Partial optimization (just VPA or just spot) would have achieved 20-30% savings-missing most of the opportunity.
variant="yellow" title="" items={[ "Over-engineering the initial solution", "Ignoring operational constraints during design", "Premature optimization without load testing", "Skipping failure mode analysis" ]} />
[Applicability] Where this pattern applies
Kubernetes cost optimization applies to any organization spending $500K+ annually on K8s infrastructure.
The visibility and incentive patterns are essential for any cost optimization to be sustainable.
Spot instance strategies work well for stateless workloads. Stateful workloads require more careful handling.
Every system is different. Context always matters.