gke-cost-optimization

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

GKE Cost Optimization

GKE成本优化

This reference covers strategies and workflows for reducing Google Kubernetes Engine (GKE) costs while maintaining a secure and reliable posture.
MCP Tools:
get_k8s_resource
,
describe_k8s_resource
,
apply_k8s_manifest
,
patch_k8s_resource
,
get_cluster
本参考文档涵盖了在保持安全可靠状态下降低Google Kubernetes Engine(GKE)成本的策略和工作流程。
MCP工具:
get_k8s_resource
,
describe_k8s_resource
,
apply_k8s_manifest
,
patch_k8s_resource
,
get_cluster

Golden Path Cost Features

黄金路径成本优化特性

The golden path already includes cost-optimizing settings:
SettingValueImpact
autoscalingProfile
OPTIMIZE_UTILIZATION
Aggressive node
: : : scale-down reduces idle :
: : : compute :
verticalPodAutoscaling
enabled
VPA recommendations
: : : prevent :
: : : over-provisioning :
Autopilot pricingPay per pod requestNo charge for unused
: : : node capacity :
Node Auto ProvisioningenabledRight-sized node pools
: : : created automatically :
黄金路径已包含以下成本优化设置:
设置项影响
autoscalingProfile
OPTIMIZE_UTILIZATION
主动缩容减少闲置计算资源
verticalPodAutoscaling
enabled
VPA建议避免过度配置
Autopilot定价按Pod请求计费闲置节点容量不收费
Node Auto Provisioningenabled自动创建规格适配的节点池

Workflows & Optimization Strategies

工作流程与优化策略

1. Prerequisite: Cost Allocation & Monitoring

1. 前提条件:成本分配与监控

To enable GKE cost allocation (
--enable-cost-allocation
) for billing tracking across namespaces and labels, inspect live cluster utilization (
kubectl top
), or run historical cost breakdown queries in BigQuery (
bq
), use the
gke-cost-analysis
skill. Once tracking is active and waste is diagnosed, apply the optimization workflows below.
要启用GKE成本分配(
--enable-cost-allocation
)以跨命名空间和标签跟踪账单、查看集群实时利用率(
kubectl top
),或在BigQuery(
bq
)中运行历史成本细分查询,请使用**
gke-cost-analysis
**技能。一旦跟踪功能激活并识别出资源浪费,即可应用以下优化工作流程。

2. Configure Resource Quotas

2. 配置资源配额

Resource quotas restrict total resource consumption across tenants in multi-tenant clusters, preventing runaway costs.
bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: {namespace}
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 16Gi
    limits.cpu: "8"
    limits.memory: 32Gi
EOF
资源配额可限制多租户集群中各租户的总资源消耗,避免成本失控。
bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: {namespace}
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 16Gi
    limits.cpu: "8"
    limits.memory: 32Gi
EOF

3. Pod Rightsizing (VPA & MPA)

3. Pod规格调整(VPA与MPA)

Adjust pod resource requests to match actual utilization. Over-provisioned requests are one of the largest sources of waste.
  • Use VPA in Recommendation Mode:
bash
undefined
调整Pod资源请求以匹配实际利用率。过度配置的请求是资源浪费的主要来源之一。
  • 在推荐模式下使用VPA
bash
undefined

1. Deploy VPA in recommendation mode

1. 以推荐模式部署VPA

kubectl apply -f - <<EOF apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: {deployment_name}-vpa spec: targetRef: apiVersion: apps/v1 kind: Deployment name: {deployment_name} updatePolicy: updateMode: "Off" EOF
kubectl apply -f - <<EOF apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: {deployment_name}-vpa spec: targetRef: apiVersion: apps/v1 kind: Deployment name: {deployment_name} updatePolicy: updateMode: "Off" EOF

2. Wait 24+ hours for data collection

2. 等待24小时以上以收集数据

3. Read recommendations

3. 查看推荐配置

kubectl get vpa {deployment_name}-vpa -o jsonpath='{.status.recommendation}'

-   **Optimization Rules:**

Condition                     | Action                             | Savings
----------------------------- | ---------------------------------- | -------
CPU request >5x P95 actual    | Reduce to `P95 * 1.2`              | High
Memory request >3x P95 actual | Reduce to `P95 * 1.2`              | High
CPU request >2x P95 actual    | Reduce to `P95 * 1.2`              | Medium
No resource requests set      | Add requests (enables bin-packing) | Medium

-   **Use MPA**: Reconcile HPA and VPA recommendations when scaling both
    horizontally and vertically to avoid conflicting scale events.
-   **Review Cost Recommendations**: Check Google Cloud Console (`Cost
    Management` > `GKE Cost Optimization`) for built-in rightsizing suggestions.
kubectl get vpa {deployment_name}-vpa -o jsonpath='{.status.recommendation}'

-   **优化规则**:

条件                     | 操作                             | 节省幅度
----------------------------- | ---------------------------------- | -------
CPU请求 > 实际P95值的5倍    | 调整为 `P95 * 1.2`              | 高
内存请求 > 实际P95值的3倍 | 调整为 `P95 * 1.2`              | 高
CPU请求 > 实际P95值的2倍    | 调整为 `P95 * 1.2`              | 中
未设置资源请求      | 添加请求(启用装箱调度) | 中

-   **使用MPA**:在同时进行水平和垂直扩缩容时,协调HPA和VPA的建议,避免冲突的扩缩容事件。
-   **查看成本建议**:在Google Cloud控制台(`成本管理` > `GKE成本优化`)中查看内置的规格调整建议。

4. Spot VMs via ComputeClasses & NodeSelector

4. 通过ComputeClasses与NodeSelector使用Spot VMs

Use Spot VMs for fault-tolerant workloads to achieve 60-90% cost reduction.
为容错工作负载使用Spot VMs可降低60-90%的成本。

4.1 ComputeClass Configuration

4.1 ComputeClass配置

yaml
apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: spot-with-fallback
spec:
  activeMigration:
    optimizeRulePriority: true
  priorities:
  - machineFamily: n4
    spot: true
  - machineFamily: n4
    spot: false
yaml
apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: spot-with-fallback
spec:
  activeMigration:
    optimizeRulePriority: true
  priorities:
  - machineFamily: n4
    spot: true
  - machineFamily: n4
    spot: false

4.2 Direct Workload Spot Selection (
nodeSelector
)

4.2 直接选择工作负载使用Spot(
nodeSelector

For stateless or batch workloads in GKE Autopilot, target Spot capacity directly using
nodeSelector
:
[!WARNING] Preemption Warning: Spot VMs are interruptible and can be preempted at any time with a 30-second notice. Workloads must be fault-tolerant and run with at least 2 replicas for high availability. Always explicitly warn users about this preemption risk when recommending Spot VMs.
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stateless-spot-app
spec:
  replicas: 2
  template:
    spec:
      nodeSelector:
        cloud.google.com/gke-provisioning: Spot
      terminationGracePeriodSeconds: 25  # Must be < 30s for Spot preemption handling
      containers:
      - name: app
        image: {image_name}
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 5"]
Spot-Suitable Workloads:
WorkloadSpot-Suitable?
Batch / data processingYes
Dev / test environmentsYes
Stateless web/API (replicas >= 2)Yes (with PDBs)
Jobs with checkpointingYes
Stateful workloads (databases)No
Single-replica critical servicesNo
对于GKE Autopilot中的无状态或批处理工作负载,可直接使用
nodeSelector
指定Spot容量:
[!WARNING] 抢占警告:Spot VMs是可中断的,随时可能收到30秒通知后被抢占。工作负载必须具备容错能力,且至少运行2个副本以保证高可用性。在推荐使用Spot VMs时,务必明确向用户告知此抢占风险。
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stateless-spot-app
spec:
  replicas: 2
  template:
    spec:
      nodeSelector:
        cloud.google.com/gke-provisioning: Spot
      terminationGracePeriodSeconds: 25  # 必须小于30秒以处理Spot抢占
      containers:
      - name: app
        image: {image_name}
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 5"]
适合使用Spot的工作负载:
工作负载类型是否适合使用Spot
批处理/数据处理
开发/测试环境
无状态Web/API(副本数≥2)是(需配合PDB)
带检查点的任务
有状态工作负载(数据库)
单副本关键服务

5. Machine Type Selection

5. 机器类型选择

When choosing node shapes or configuring ComputeClasses:
FamilyUse CaseRelative Cost
e2General purpose, burstableLowest
t2a / t2dScale-out (Arm/AMD), price-performanceLow
: : optimized : :
n4aAxion Arm-based, general-purposeLow
: : price-performance : :
n4 / n4dGeneral purpose (Intel/AMD), flexible shapesLow-Medium
c4aCompute-optimized (Arm), high efficiencyMedium-High
c3 / c4Compute-optimized (Intel)Medium-High
c3d / c4dCompute-optimized (AMD), high throughputMedium-High
ek-standardAutopilot enhanced (golden path)Medium
m3 / x4Memory-optimized, SAP HANA, large databasesHigh
g2 (L4 GPU)AI inferenceHigh
a3 (H100 GPU)AI trainingHighest
a4 / a4xUltra-scale AI (Blackwell GPUs)Highest
在选择节点规格或配置ComputeClasses时:
系列使用场景相对成本
e2通用型、可突发性能最低
t2a / t2d横向扩展(Arm/AMD)、性价比优化
n4aAxion Arm架构、通用型
n4 / n4d通用型(Intel/AMD)、灵活规格中低
c4a计算优化型(Arm)、高效能中高
c3 / c4计算优化型(Intel)中高
c3d / c4d计算优化型(AMD)、高吞吐量中高
ek-standardAutopilot增强型(黄金路径)
m3 / x4内存优化型、SAP HANA、大型数据库
g2(L4 GPU)AI推理
a3(H100 GPU)AI训练最高
a4 / a4x超大规模AI(Blackwell GPU)最高

6. Committed Use Discounts (CUDs)

6. 承诺使用折扣(CUDs)

For steady-state workloads with predictable baseline usage, purchase 1-year or 3-year CUDs:
  • 1-year: ~20-30% discount
  • 3-year: ~50-55% discount
  • Applied automatically to matching usage across the region.
  • Purchase via Google Cloud Console > Billing > Committed use discounts.
对于具有可预测基线使用率的稳态工作负载,购买1年或3年的CUDs:
  • 1年期:约20-30%折扣
  • 3年期:约50-55%折扣
  • 自动应用于区域内匹配的使用量
  • 通过Google Cloud控制台 > 账单 > 承诺使用折扣进行购买

7. Cluster Management & Multi-Tenancy

7. 集群管理与多租户

  • Stop/start dev clusters: Idle dev clusters cost money even with no workloads due to control plane fees.
  • Right-size node pools (Standard): Use Cluster Autoscaler with appropriate min/max limits.
  • Multi-tenant consolidation: Share a single cluster across multiple engineering teams instead of maintaining per-team clusters, using Namespaces and ResourceQuotas to isolate workloads.
  • 启停开发集群:闲置的开发集群即使没有工作负载,也会因控制平面费用产生成本。
  • 调整节点池规格(Standard模式):使用集群自动扩缩容并设置合适的最小/最大限制。
  • 多租户整合:将多个工程团队的工作负载共享到单个集群中,而非维护每个团队独立的集群,使用命名空间和资源配额隔离工作负载。

Cost & Utilization Monitoring

成本与利用率监控

To inspect live node/pod utilization (
kubectl top nodes/pods
), view cluster cost budgets (
gcloud billing budgets list
), or query detailed billing reports in BigQuery (
bq query
), refer to the
gke-cost-analysis
skill.
要查看节点/Pod实时利用率(
kubectl top nodes/pods
)、查看集群成本预算(
gcloud billing budgets list
),或在BigQuery(
bq query
)中查询详细账单报告,请参考**
gke-cost-analysis
**技能。

Dev/Test Cost Savings

开发/测试环境成本节约

For non-production environments, the following golden path deviations provide cost efficiency without impacting production safety:
| Setting | Production (Golden | Dev/Test | : : Path) : : | ----------------------- | ------------------ | ----------------------------- | | Cluster mode | Autopilot | Autopilot (cheaper with fewer | : : : pods) : | Release channel | Regular | Rapid (get fixes faster) | | Private nodes | Required | Optional (simpler access) | | Monitoring components | Full suite |
SYSTEM_COMPONENTS
only | | Secret Manager rotation | 120s | Disabled | | Maintenance windows | Configured | Not needed |
对于非生产环境,以下黄金路径的调整可在不影响生产安全性的前提下提升成本效率:
设置项生产环境(黄金路径)开发/测试环境
集群模式AutopilotAutopilot(Pod数量更少时更经济)
发布渠道RegularRapid(更快获取修复)
私有节点必填可选(简化访问)
监控组件完整套件仅启用
SYSTEM_COMPONENTS
Secret Manager轮换120秒禁用
维护窗口已配置无需配置

Best Practices Summary

最佳实践总结

  1. Enable Cost Allocation: Always enable GKE cost allocation (
    --enable-cost-allocation
    ) to gain billing transparency across namespaces and labels.
  2. Enforce Resource Quotas: Restrict namespace CPU/memory limits in multi-tenant environments to prevent runaway costs or noisy neighbors.
  3. Rightsize Continuously: Run VPA in recommendation mode (
    updateMode: Off
    ) and adjust requests to match
    P95 * 1.2
    .
  4. Leverage Spot VMs: Use Spot VMs with
    nodeSelector
    or
    ComputeClass
    for stateless, fault-tolerant workloads to save 60-90%.
  5. Optimize Autoscaling Profile: Use
    OPTIMIZE_UTILIZATION
    for aggressive node scale-down on idle compute.
  6. Consolidate & Clean Up: Stop idle development clusters and consolidate multi-team workloads into shared multi-tenant clusters.
  1. 启用成本分配:始终启用GKE成本分配(
    --enable-cost-allocation
    ),以获得跨命名空间和标签的账单透明度。
  2. 强制实施资源配额:在多租户环境中限制命名空间的CPU/内存上限,避免成本失控或邻居干扰。
  3. 持续调整规格:以推荐模式运行VPA(
    updateMode: Off
    ),并将请求调整为
    P95 * 1.2
  4. 利用Spot VMs:为无状态、容错工作负载使用
    nodeSelector
    ComputeClass
    配置Spot VMs,节省60-90%成本。
  5. 优化自动扩缩容配置文件:使用
    OPTIMIZE_UTILIZATION
    实现闲置计算资源的主动缩容。
  6. 整合与清理:停止闲置的开发集群,将多团队工作负载整合到共享的多租户集群中。