gke-cost-optimization
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseGKE Cost Optimization
GKE成本优化
This reference covers strategies and workflows for reducing Google Kubernetes
Engine (GKE) costs while maintaining a secure and reliable posture.
MCP Tools:,get_k8s_resource,describe_k8s_resource,apply_k8s_manifest,patch_k8s_resourceget_cluster
本参考文档涵盖了在保持安全可靠状态下降低Google Kubernetes Engine(GKE)成本的策略和工作流程。
MCP工具:,get_k8s_resource,describe_k8s_resource,apply_k8s_manifest,patch_k8s_resourceget_cluster
Golden Path Cost Features
黄金路径成本优化特性
The golden path already includes cost-optimizing settings:
| Setting | Value | Impact |
|---|---|---|
| | Aggressive node |
| : : : scale-down reduces idle : | ||
| : : : compute : | ||
| | VPA recommendations |
| : : : prevent : | ||
| : : : over-provisioning : | ||
| Autopilot pricing | Pay per pod request | No charge for unused |
| : : : node capacity : | ||
| Node Auto Provisioning | enabled | Right-sized node pools |
| : : : created automatically : |
黄金路径已包含以下成本优化设置:
| 设置项 | 值 | 影响 |
|---|---|---|
| | 主动缩容减少闲置计算资源 |
| | VPA建议避免过度配置 |
| Autopilot定价 | 按Pod请求计费 | 闲置节点容量不收费 |
| Node Auto Provisioning | enabled | 自动创建规格适配的节点池 |
Workflows & Optimization Strategies
工作流程与优化策略
1. Prerequisite: Cost Allocation & Monitoring
1. 前提条件:成本分配与监控
To enable GKE cost allocation () for billing tracking
across namespaces and labels, inspect live cluster utilization (),
or run historical cost breakdown queries in BigQuery (), use the
skill. Once tracking is active and waste is diagnosed,
apply the optimization workflows below.
--enable-cost-allocationkubectl topbqgke-cost-analysis要启用GKE成本分配()以跨命名空间和标签跟踪账单、查看集群实时利用率(),或在BigQuery()中运行历史成本细分查询,请使用****技能。一旦跟踪功能激活并识别出资源浪费,即可应用以下优化工作流程。
--enable-cost-allocationkubectl topbqgke-cost-analysis2. Configure Resource Quotas
2. 配置资源配额
Resource quotas restrict total resource consumption across tenants in
multi-tenant clusters, preventing runaway costs.
bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
name: compute-quota
namespace: {namespace}
spec:
hard:
requests.cpu: "4"
requests.memory: 16Gi
limits.cpu: "8"
limits.memory: 32Gi
EOF资源配额可限制多租户集群中各租户的总资源消耗,避免成本失控。
bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
name: compute-quota
namespace: {namespace}
spec:
hard:
requests.cpu: "4"
requests.memory: 16Gi
limits.cpu: "8"
limits.memory: 32Gi
EOF3. Pod Rightsizing (VPA & MPA)
3. Pod规格调整(VPA与MPA)
Adjust pod resource requests to match actual utilization. Over-provisioned
requests are one of the largest sources of waste.
- Use VPA in Recommendation Mode:
bash
undefined调整Pod资源请求以匹配实际利用率。过度配置的请求是资源浪费的主要来源之一。
- 在推荐模式下使用VPA:
bash
undefined1. Deploy VPA in recommendation mode
1. 以推荐模式部署VPA
kubectl apply -f - <<EOF
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: {deployment_name}-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: {deployment_name}
updatePolicy:
updateMode: "Off"
EOF
kubectl apply -f - <<EOF
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: {deployment_name}-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: {deployment_name}
updatePolicy:
updateMode: "Off"
EOF
2. Wait 24+ hours for data collection
2. 等待24小时以上以收集数据
3. Read recommendations
3. 查看推荐配置
kubectl get vpa {deployment_name}-vpa -o jsonpath='{.status.recommendation}'
- **Optimization Rules:**
Condition | Action | Savings
----------------------------- | ---------------------------------- | -------
CPU request >5x P95 actual | Reduce to `P95 * 1.2` | High
Memory request >3x P95 actual | Reduce to `P95 * 1.2` | High
CPU request >2x P95 actual | Reduce to `P95 * 1.2` | Medium
No resource requests set | Add requests (enables bin-packing) | Medium
- **Use MPA**: Reconcile HPA and VPA recommendations when scaling both
horizontally and vertically to avoid conflicting scale events.
- **Review Cost Recommendations**: Check Google Cloud Console (`Cost
Management` > `GKE Cost Optimization`) for built-in rightsizing suggestions.kubectl get vpa {deployment_name}-vpa -o jsonpath='{.status.recommendation}'
- **优化规则**:
条件 | 操作 | 节省幅度
----------------------------- | ---------------------------------- | -------
CPU请求 > 实际P95值的5倍 | 调整为 `P95 * 1.2` | 高
内存请求 > 实际P95值的3倍 | 调整为 `P95 * 1.2` | 高
CPU请求 > 实际P95值的2倍 | 调整为 `P95 * 1.2` | 中
未设置资源请求 | 添加请求(启用装箱调度) | 中
- **使用MPA**:在同时进行水平和垂直扩缩容时,协调HPA和VPA的建议,避免冲突的扩缩容事件。
- **查看成本建议**:在Google Cloud控制台(`成本管理` > `GKE成本优化`)中查看内置的规格调整建议。4. Spot VMs via ComputeClasses & NodeSelector
4. 通过ComputeClasses与NodeSelector使用Spot VMs
Use Spot VMs for fault-tolerant workloads to achieve 60-90% cost reduction.
为容错工作负载使用Spot VMs可降低60-90%的成本。
4.1 ComputeClass Configuration
4.1 ComputeClass配置
yaml
apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
name: spot-with-fallback
spec:
activeMigration:
optimizeRulePriority: true
priorities:
- machineFamily: n4
spot: true
- machineFamily: n4
spot: falseyaml
apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
name: spot-with-fallback
spec:
activeMigration:
optimizeRulePriority: true
priorities:
- machineFamily: n4
spot: true
- machineFamily: n4
spot: false4.2 Direct Workload Spot Selection (nodeSelector
)
nodeSelector4.2 直接选择工作负载使用Spot(nodeSelector
)
nodeSelectorFor stateless or batch workloads in GKE Autopilot, target Spot capacity directly
using :
nodeSelector[!WARNING] Preemption Warning: Spot VMs are interruptible and can be preempted at any time with a 30-second notice. Workloads must be fault-tolerant and run with at least 2 replicas for high availability. Always explicitly warn users about this preemption risk when recommending Spot VMs.
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: stateless-spot-app
spec:
replicas: 2
template:
spec:
nodeSelector:
cloud.google.com/gke-provisioning: Spot
terminationGracePeriodSeconds: 25 # Must be < 30s for Spot preemption handling
containers:
- name: app
image: {image_name}
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 5"]Spot-Suitable Workloads:
| Workload | Spot-Suitable? |
|---|---|
| Batch / data processing | Yes |
| Dev / test environments | Yes |
| Stateless web/API (replicas >= 2) | Yes (with PDBs) |
| Jobs with checkpointing | Yes |
| Stateful workloads (databases) | No |
| Single-replica critical services | No |
对于GKE Autopilot中的无状态或批处理工作负载,可直接使用指定Spot容量:
nodeSelector[!WARNING] 抢占警告:Spot VMs是可中断的,随时可能收到30秒通知后被抢占。工作负载必须具备容错能力,且至少运行2个副本以保证高可用性。在推荐使用Spot VMs时,务必明确向用户告知此抢占风险。
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: stateless-spot-app
spec:
replicas: 2
template:
spec:
nodeSelector:
cloud.google.com/gke-provisioning: Spot
terminationGracePeriodSeconds: 25 # 必须小于30秒以处理Spot抢占
containers:
- name: app
image: {image_name}
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 5"]适合使用Spot的工作负载:
| 工作负载类型 | 是否适合使用Spot |
|---|---|
| 批处理/数据处理 | 是 |
| 开发/测试环境 | 是 |
| 无状态Web/API(副本数≥2) | 是(需配合PDB) |
| 带检查点的任务 | 是 |
| 有状态工作负载(数据库) | 否 |
| 单副本关键服务 | 否 |
5. Machine Type Selection
5. 机器类型选择
When choosing node shapes or configuring ComputeClasses:
| Family | Use Case | Relative Cost |
|---|---|---|
| e2 | General purpose, burstable | Lowest |
| t2a / t2d | Scale-out (Arm/AMD), price-performance | Low |
| : : optimized : : | ||
| n4a | Axion Arm-based, general-purpose | Low |
| : : price-performance : : | ||
| n4 / n4d | General purpose (Intel/AMD), flexible shapes | Low-Medium |
| c4a | Compute-optimized (Arm), high efficiency | Medium-High |
| c3 / c4 | Compute-optimized (Intel) | Medium-High |
| c3d / c4d | Compute-optimized (AMD), high throughput | Medium-High |
| ek-standard | Autopilot enhanced (golden path) | Medium |
| m3 / x4 | Memory-optimized, SAP HANA, large databases | High |
| g2 (L4 GPU) | AI inference | High |
| a3 (H100 GPU) | AI training | Highest |
| a4 / a4x | Ultra-scale AI (Blackwell GPUs) | Highest |
在选择节点规格或配置ComputeClasses时:
| 系列 | 使用场景 | 相对成本 |
|---|---|---|
| e2 | 通用型、可突发性能 | 最低 |
| t2a / t2d | 横向扩展(Arm/AMD)、性价比优化 | 低 |
| n4a | Axion Arm架构、通用型 | 低 |
| n4 / n4d | 通用型(Intel/AMD)、灵活规格 | 中低 |
| c4a | 计算优化型(Arm)、高效能 | 中高 |
| c3 / c4 | 计算优化型(Intel) | 中高 |
| c3d / c4d | 计算优化型(AMD)、高吞吐量 | 中高 |
| ek-standard | Autopilot增强型(黄金路径) | 中 |
| m3 / x4 | 内存优化型、SAP HANA、大型数据库 | 高 |
| g2(L4 GPU) | AI推理 | 高 |
| a3(H100 GPU) | AI训练 | 最高 |
| a4 / a4x | 超大规模AI(Blackwell GPU) | 最高 |
6. Committed Use Discounts (CUDs)
6. 承诺使用折扣(CUDs)
For steady-state workloads with predictable baseline usage, purchase 1-year or
3-year CUDs:
- 1-year: ~20-30% discount
- 3-year: ~50-55% discount
- Applied automatically to matching usage across the region.
- Purchase via Google Cloud Console > Billing > Committed use discounts.
对于具有可预测基线使用率的稳态工作负载,购买1年或3年的CUDs:
- 1年期:约20-30%折扣
- 3年期:约50-55%折扣
- 自动应用于区域内匹配的使用量
- 通过Google Cloud控制台 > 账单 > 承诺使用折扣进行购买
7. Cluster Management & Multi-Tenancy
7. 集群管理与多租户
- Stop/start dev clusters: Idle dev clusters cost money even with no workloads due to control plane fees.
- Right-size node pools (Standard): Use Cluster Autoscaler with appropriate min/max limits.
- Multi-tenant consolidation: Share a single cluster across multiple engineering teams instead of maintaining per-team clusters, using Namespaces and ResourceQuotas to isolate workloads.
- 启停开发集群:闲置的开发集群即使没有工作负载,也会因控制平面费用产生成本。
- 调整节点池规格(Standard模式):使用集群自动扩缩容并设置合适的最小/最大限制。
- 多租户整合:将多个工程团队的工作负载共享到单个集群中,而非维护每个团队独立的集群,使用命名空间和资源配额隔离工作负载。
Cost & Utilization Monitoring
成本与利用率监控
To inspect live node/pod utilization (), view cluster
cost budgets (), or query detailed billing reports
in BigQuery (), refer to the skill.
kubectl top nodes/podsgcloud billing budgets listbq querygke-cost-analysis要查看节点/Pod实时利用率()、查看集群成本预算(),或在BigQuery()中查询详细账单报告,请参考****技能。
kubectl top nodes/podsgcloud billing budgets listbq querygke-cost-analysisDev/Test Cost Savings
开发/测试环境成本节约
For non-production environments, the following golden path deviations provide
cost efficiency without impacting production safety:
| Setting | Production (Golden | Dev/Test |
: : Path) : :
| ----------------------- | ------------------ | ----------------------------- |
| Cluster mode | Autopilot | Autopilot (cheaper with fewer |
: : : pods) :
| Release channel | Regular | Rapid (get fixes faster) |
| Private nodes | Required | Optional (simpler access) |
| Monitoring components | Full suite | only |
| Secret Manager rotation | 120s | Disabled |
| Maintenance windows | Configured | Not needed |
SYSTEM_COMPONENTS对于非生产环境,以下黄金路径的调整可在不影响生产安全性的前提下提升成本效率:
| 设置项 | 生产环境(黄金路径) | 开发/测试环境 |
|---|---|---|
| 集群模式 | Autopilot | Autopilot(Pod数量更少时更经济) |
| 发布渠道 | Regular | Rapid(更快获取修复) |
| 私有节点 | 必填 | 可选(简化访问) |
| 监控组件 | 完整套件 | 仅启用 |
| Secret Manager轮换 | 120秒 | 禁用 |
| 维护窗口 | 已配置 | 无需配置 |
Best Practices Summary
最佳实践总结
- Enable Cost Allocation: Always enable GKE cost allocation
() to gain billing transparency across namespaces and labels.
--enable-cost-allocation - Enforce Resource Quotas: Restrict namespace CPU/memory limits in multi-tenant environments to prevent runaway costs or noisy neighbors.
- Rightsize Continuously: Run VPA in recommendation mode () and adjust requests to match
updateMode: Off.P95 * 1.2 - Leverage Spot VMs: Use Spot VMs with or
nodeSelectorfor stateless, fault-tolerant workloads to save 60-90%.ComputeClass - Optimize Autoscaling Profile: Use for aggressive node scale-down on idle compute.
OPTIMIZE_UTILIZATION - Consolidate & Clean Up: Stop idle development clusters and consolidate multi-team workloads into shared multi-tenant clusters.
- 启用成本分配:始终启用GKE成本分配(),以获得跨命名空间和标签的账单透明度。
--enable-cost-allocation - 强制实施资源配额:在多租户环境中限制命名空间的CPU/内存上限,避免成本失控或邻居干扰。
- 持续调整规格:以推荐模式运行VPA(),并将请求调整为
updateMode: Off。P95 * 1.2 - 利用Spot VMs:为无状态、容错工作负载使用或
nodeSelector配置Spot VMs,节省60-90%成本。ComputeClass - 优化自动扩缩容配置文件:使用实现闲置计算资源的主动缩容。
OPTIMIZE_UTILIZATION - 整合与清理:停止闲置的开发集群,将多团队工作负载整合到共享的多租户集群中。