Back to Details

building-ci-pipelines

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Building CI Pipelines

构建CI流水线

Purpose

用途

CI/CD pipelines automate testing, building, and deploying software. This skill provides patterns for constructing robust, secure, and efficient pipelines across GitHub Actions, GitLab CI, Argo Workflows, and Jenkins. Focus areas: supply chain security (SLSA), monorepo optimization, caching, and parallelization.
CI/CD流水线可自动化软件的测试、构建与部署流程。本技能提供了可在GitHub Actions、GitLab CI、Argo Workflows和Jenkins上构建健壮、安全且高效流水线的模式。重点关注领域：供应链安全（SLSA）、单体仓库优化、缓存和并行化。

When to Use This Skill

何时使用此技能

Invoke when:
  • Setting up continuous integration for new projects
  • Implementing automated testing workflows
  • Building container images with security provenance
  • Optimizing slow CI pipelines (especially monorepos)
  • Implementing SLSA supply chain security
  • Configuring multi-platform builds
  • Setting up GitOps automation
  • Migrating from legacy CI systems
在以下场景中调用：
  • 为新项目搭建持续集成流程
  • 实现自动化测试工作流
  • 构建带有安全溯源的容器镜像
  • 优化运行缓慢的CI流水线（尤其是单体仓库场景）
  • 实施SLSA供应链安全标准
  • 配置多平台构建
  • 搭建GitOps自动化流程
  • 从传统CI系统迁移

Platform Selection

平台选择

GitHub-hosted → GitHub Actions (SLSA native, 10K+ actions, OIDC) GitLab-hosted → GitLab CI (parent-child pipelines, built-in security) Kubernetes → Argo Workflows (DAG-based, event-driven) Legacy → Jenkins (migrate when possible)
GitHub托管 → GitHub Actions（原生支持SLSA，拥有10000+可用Actions，支持OIDC） GitLab托管 → GitLab CI（支持父子流水线，内置安全功能） Kubernetes环境 → Argo Workflows（基于DAG，事件驱动） 传统环境 → Jenkins（尽可能迁移至其他平台）

Platform Comparison

平台对比

FeatureGitHub ActionsGitLab CIArgoJenkins
Ease of Use⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐
SLSANativeManualGoodManual
MonorepoGoodExcellentManualPlugins
特性GitHub ActionsGitLab CIArgoJenkins
易用性⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐
SLSA支持原生手动配置良好手动配置
单体仓库支持良好优秀手动配置通过插件支持

Quick Start Patterns

快速入门模式

Pattern 1: Basic CI (Lint → Test → Build)

模式1：基础CI流程（代码检查 → 测试 → 构建）

yaml
undefined
yaml
undefined

GitHub Actions

GitHub Actions

name: CI on: [push, pull_request]
jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm run lint
test: needs: lint runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm test
build: needs: test runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm run build
undefined
name: CI on: [push, pull_request]
jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm run lint
test: needs: lint runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm test
build: needs: test runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm run build
undefined

Pattern 2: Matrix Strategy (Multi-Platform)

模式2：矩阵策略（多平台构建）

yaml
test:
  runs-on: ${{ matrix.os }}
  strategy:
    matrix:
      os: [ubuntu-latest, windows-latest, macos-latest]
      node-version: [18, 20, 22]
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm test
9 jobs (3 OS × 3 versions) in parallel: 5 min vs 45 min sequential.
yaml
test:
  runs-on: ${{ matrix.os }}
  strategy:
    matrix:
      os: [ubuntu-latest, windows-latest, macos-latest]
      node-version: [18, 20, 22]
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm test
并行运行9个任务（3种操作系统 × 3个Node.js版本）：耗时从45分钟缩短至5分钟。

Pattern 3: Monorepo Affected (Turborepo)

模式3：单体仓库增量构建（Turborepo）

yaml
build:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0  # Required for affected detection

    - uses: actions/setup-node@v4
      with:
        node-version: 20

    - name: Build affected
      run: npx turbo run build --filter='...[origin/main]'
      env:
        TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
        TURBO_TEAM: ${{ vars.TURBO_TEAM }}
60-80% CI time reduction for monorepos.
yaml
build:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0  # 增量检测必需配置

    - uses: actions/setup-node@v4
      with:
        node-version: 20

    - name: Build affected
      run: npx turbo run build --filter='...[origin/main]'
      env:
        TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
        TURBO_TEAM: ${{ vars.TURBO_TEAM }}
单体仓库CI耗时可减少60-80%。

Pattern 4: SLSA Level 3 Provenance

模式4：SLSA Level 3 溯源构建

yaml
name: SLSA Build
on:
  push:
    tags: ['v*']

permissions:
  id-token: write
  contents: read
  packages: write

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4
      - name: Build container
        id: build
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}

  provenance:
    needs: build
    permissions:
      id-token: write
      actions: read
      packages: write
    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v1.10.0
    with:
      image: ghcr.io/${{ github.repository }}
      digest: ${{ needs.build.outputs.digest }}
      registry-username: ${{ github.actor }}
    secrets:
      registry-password: ${{ secrets.GITHUB_TOKEN }}
Verification:
bash
cosign verify-attestation --type slsaprovenance \
  --certificate-identity-regexp "^https://github.com/slsa-framework" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/myorg/myapp@sha256:abcd...
yaml
name: SLSA Build
on:
  push:
    tags: ['v*']

permissions:
  id-token: write
  contents: read
  packages: write

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4
      - name: Build container
        id: build
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}

  provenance:
    needs: build
    permissions:
      id-token: write
      actions: read
      packages: write
    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v1.10.0
    with:
      image: ghcr.io/${{ github.repository }}
      digest: ${{ needs.build.outputs.digest }}
      registry-username: ${{ github.actor }}
    secrets:
      registry-password: ${{ secrets.GITHUB_TOKEN }}
验证命令：
bash
cosign verify-attestation --type slsaprovenance \
  --certificate-identity-regexp "^https://github.com/slsa-framework" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/myorg/myapp@sha256:abcd...

Pattern 5: OIDC Federation (No Credentials)

模式5：OIDC联邦认证（无需存储凭证）

yaml
deploy:
  runs-on: ubuntu-latest
  permissions:
    id-token: write
    contents: read
  steps:
    - uses: actions/checkout@v4

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
        aws-region: us-east-1

    - name: Deploy
      run: aws s3 sync ./dist s3://my-bucket
Benefits: No stored credentials, 1-hour lifetime, full audit trail.
yaml
deploy:
  runs-on: ubuntu-latest
  permissions:
    id-token: write
    contents: read
  steps:
    - uses: actions/checkout@v4

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
        aws-region: us-east-1

    - name: Deploy
      run: aws s3 sync ./dist s3://my-bucket
优势：无需存储长期凭证，凭证有效期1小时，具备完整审计追踪。

Pattern 6: Security Scanning

模式6：安全扫描

yaml
security:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0

    - name: Gitleaks (secret detection)
      uses: gitleaks/gitleaks-action@v2

    - name: Snyk (vulnerability scan)
      uses: snyk/actions/node@master
      env:
        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

    - name: SBOM generation
      uses: anchore/sbom-action@v0
      with:
        format: spdx-json
        output-file: sbom.spdx.json
yaml
security:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0

    - name: Gitleaks (密钥检测)
      uses: gitleaks/gitleaks-action@v2

    - name: Snyk (漏洞扫描)
      uses: snyk/actions/node@master
      env:
        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

    - name: SBOM生成
      uses: anchore/sbom-action@v0
      with:
        format: spdx-json
        output-file: sbom.spdx.json

Caching

缓存策略

Automatic Dependency Caching

自动依赖缓存

yaml
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: 'npm'  # Auto-caches ~/.npm
- run: npm ci
Supported: npm, yarn, pnpm, pip, poetry, cargo, go
yaml
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: 'npm'  # 自动缓存~/.npm目录
- run: npm ci
支持的包管理器：npm、yarn、pnpm、pip、poetry、cargo、go

Manual Cache Control

手动缓存控制

yaml
- uses: actions/cache@v4
  with:
    path: |
      ~/.cargo/bin
      ~/.cargo/registry
      target/
    key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
    restore-keys: |
      ${{ runner.os }}-cargo-
yaml
- uses: actions/cache@v4
  with:
    path: |
      ~/.cargo/bin
      ~/.cargo/registry
      target/
    key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
    restore-keys: |
      ${{ runner.os }}-cargo-

Multi-Layer Caching (Nx)

多层缓存（Nx）

yaml
- name: Nx Cloud (build outputs)
  run: npx nx affected -t build
  env:
    NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}

- name: Vite Cache
  uses: actions/cache@v4
  with:
    path: '**/node_modules/.vite'
    key: vite-${{ hashFiles('package-lock.json') }}

- name: TypeScript Cache
  uses: actions/cache@v4
  with:
    path: '**/tsconfig.tsbuildinfo'
    key: tsc-${{ hashFiles('tsconfig.json') }}
Result: 70-90% build time reduction.
yaml
- name: Nx Cloud (构建输出缓存)
  run: npx nx affected -t build
  env:
    NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}

- name: Vite缓存
  uses: actions/cache@v4
  with:
    path: '**/node_modules/.vite'
    key: vite-${{ hashFiles('package-lock.json') }}

- name: TypeScript缓存
  uses: actions/cache@v4
  with:
    path: '**/tsconfig.tsbuildinfo'
    key: tsc-${{ hashFiles('tsconfig.json') }}
效果：构建时间减少70-90%。

Parallelization

并行化策略

Job-Level Parallelization

任务级并行化

yaml
jobs:
  unit-tests:
    steps:
      - run: npm run test:unit

  integration-tests:
    steps:
      - run: npm run test:integration

  e2e-tests:
    steps:
      - run: npm run test:e2e
All three run simultaneously.
yaml
jobs:
  unit-tests:
    steps:
      - run: npm run test:unit

  integration-tests:
    steps:
      - run: npm run test:integration

  e2e-tests:
    steps:
      - run: npm run test:e2e
三个任务同时运行。

Test Sharding

测试分片

yaml
test:
  strategy:
    matrix:
      shard: [1, 2, 3, 4]
  steps:
    - run: npm test -- --shard=${{ matrix.shard }}/4
20min test suite → 5min (4x speedup).
yaml
test:
  strategy:
    matrix:
      shard: [1, 2, 3, 4]
  steps:
    - run: npm test -- --shard=${{ matrix.shard }}/4
20分钟的测试套件 → 5分钟完成（4倍提速）。

Language Examples

多语言示例

Python

Python

yaml
test:
  strategy:
    matrix:
      python-version: ['3.10', '3.11', '3.12']
  steps:
    - uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
    - run: pipx install poetry
    - run: poetry install
    - run: poetry run ruff check .
    - run: poetry run mypy .
    - run: poetry run pytest --cov
yaml
test:
  strategy:
    matrix:
      python-version: ['3.10', '3.11', '3.12']
  steps:
    - uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
    - run: pipx install poetry
    - run: poetry install
    - run: poetry run ruff check .
    - run: poetry run mypy .
    - run: poetry run pytest --cov

Rust

Rust

yaml
test:
  strategy:
    matrix:
      os: [ubuntu-latest, windows-latest, macos-latest]
      rust: [stable, nightly]
  steps:
    - uses: dtolnay/rust-toolchain@master
      with:
        toolchain: ${{ matrix.rust }}
        components: rustfmt, clippy
    - uses: Swatinem/rust-cache@v2
    - run: cargo fmt -- --check
    - run: cargo clippy -- -D warnings
    - run: cargo test
yaml
test:
  strategy:
    matrix:
      os: [ubuntu-latest, windows-latest, macos-latest]
      rust: [stable, nightly]
  steps:
    - uses: dtolnay/rust-toolchain@master
      with:
        toolchain: ${{ matrix.rust }}
        components: rustfmt, clippy
    - uses: Swatinem/rust-cache@v2
    - run: cargo fmt -- --check
    - run: cargo clippy -- -D warnings
    - run: cargo test

Go

Go

yaml
test:
  steps:
    - uses: actions/setup-go@v5
      with:
        go-version: '1.23'
        cache: true
    - run: go mod verify
    - uses: golangci/golangci-lint-action@v4
    - run: go test -v -race -coverprofile=coverage.txt ./...
yaml
test:
  steps:
    - uses: actions/setup-go@v5
      with:
        go-version: '1.23'
        cache: true
    - run: go mod verify
    - uses: golangci/golangci-lint-action@v4
    - run: go test -v -race -coverprofile=coverage.txt ./...

TypeScript

TypeScript

yaml
test:
  strategy:
    matrix:
      node-version: [18, 20, 22]
  steps:
    - uses: pnpm/action-setup@v3
      with:
        version: 8
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'pnpm'
    - run: pnpm install --frozen-lockfile
    - run: pnpm run lint
    - run: pnpm run type-check
    - run: pnpm test
yaml
test:
  strategy:
    matrix:
      node-version: [18, 20, 22]
  steps:
    - uses: pnpm/action-setup@v3
      with:
        version: 8
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'pnpm'
    - run: pnpm install --frozen-lockfile
    - run: pnpm run lint
    - run: pnpm run type-check
    - run: pnpm test

Best Practices

最佳实践

Security

安全层面

DO:
  • Use OIDC instead of long-lived credentials
  • Pin actions to commit SHA: 
    actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
  • Restrict permissions: 
    permissions: { contents: read }
  • Scan secrets (Gitleaks) on every commit
  • Generate SLSA provenance for releases
DON'T:
  • Expose secrets in logs
  • Use 
    pull_request_target
    without validation
  • Trust unverified third-party actions
建议：
  • 使用OIDC替代长期凭证
  • 将Actions固定到提交SHA：
    actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
  • 限制权限：
    permissions: { contents: read }
  • 每次提交都扫描密钥（Gitleaks）
  • 为发布版本生成SLSA溯源
禁止：
  • 在日志中暴露密钥
  • 未验证就使用
    pull_request_target
  • 信任未经验证的第三方Actions

Performance

性能层面

DO:
  • Use affected detection for monorepos
  • Cache dependencies and build outputs
  • Parallelize independent jobs
  • Fail fast: 
    strategy.fail-fast: true
  • Use remote caching (Turborepo/Nx Cloud)
DON'T:
  • Rebuild everything on every commit
  • Run long tests in PR checks
  • Use generic cache keys
建议：
  • 对单体仓库使用增量检测
  • 缓存依赖和构建输出
  • 并行运行独立任务
  • 快速失败：
    strategy.fail-fast: true
  • 使用远程缓存（Turborepo/Nx Cloud）
禁止：
  • 每次提交都重新构建所有内容
  • 在PR检查中运行耗时较长的测试
  • 使用通用缓存键

Debugging

调试技巧

yaml
undefined
yaml
undefined

Enable debug logging

启用调试日志

env: ACTIONS_STEP_DEBUG: true ACTIONS_RUNNER_DEBUG: true
env: ACTIONS_STEP_DEBUG: true ACTIONS_RUNNER_DEBUG: true

SSH into runner

SSH连接到运行器

  • uses: mxschmitt/action-tmate@v3
undefined
  • uses: mxschmitt/action-tmate@v3
undefined

Advanced Patterns

高级模式

For detailed guides, see references:
  • github-actions-patterns.md - Reusable workflows, composite actions, matrix strategies, OIDC setup
  • gitlab-ci-patterns.md - Parent-child pipelines, dynamic generation, runner configuration
  • argo-workflows-guide.md - DAG templates, artifact passing, event-driven triggers
  • slsa-security-framework.md - SLSA Levels 1-4, provenance generation, cosign verification
  • monorepo-ci-strategies.md - Turborepo/Nx/Bazel affected detection algorithms
  • caching-strategies.md - Multi-layer caching, Docker optimization, cache invalidation
  • parallelization-patterns.md - Test sharding, job dependencies, DAG design
  • secrets-management.md - OIDC for AWS/GCP/Azure, Vault integration, rotation
详细指南请参考以下文档：
  • github-actions-patterns.md - 可复用工作流、复合Actions、矩阵策略、OIDC配置
  • gitlab-ci-patterns.md - 父子流水线、动态生成、运行器配置
  • argo-workflows-guide.md - DAG模板、工件传递、事件驱动触发器
  • slsa-security-framework.md - SLSA 1-4级标准、溯源生成、cosign验证
  • monorepo-ci-strategies.md - Turborepo/Nx/Bazel增量检测算法
  • caching-strategies.md - 多层缓存、Docker优化、缓存失效策略
  • parallelization-patterns.md - 测试分片、任务依赖、DAG设计
  • secrets-management.md - AWS/GCP/Azure OIDC配置、Vault集成、密钥轮换

Examples

示例项目

Complete runnable workflows:
  • examples/github-actions-basic/ - Starter template (lint/test/build)
  • examples/github-actions-monorepo/ - Turborepo with remote caching
  • examples/github-actions-slsa/ - SLSA Level 3 provenance
  • examples/gitlab-ci-monorepo/ - Parent-child dynamic pipeline
  • examples/argo-workflows-dag/ - Diamond DAG parallelization
  • examples/multi-language-matrix/ - Cross-platform testing
可直接运行的完整工作流：
  • examples/github-actions-basic/ - 入门模板（代码检查/测试/构建）
  • examples/github-actions-monorepo/ - 配置远程缓存的Turborepo项目
  • examples/github-actions-slsa/ - SLSA Level 3溯源构建示例
  • examples/gitlab-ci-monorepo/ - 父子动态流水线示例
  • examples/argo-workflows-dag/ - 菱形DAG并行化示例
  • examples/multi-language-matrix/ - 跨平台多语言测试示例

Utility Scripts

工具脚本

Token-free execution:
  • scripts/validate_workflow.py - Validate YAML syntax and best practices
  • scripts/generate_github_workflow.py - Generate workflow from template
  • scripts/analyze_ci_performance.py - CI metrics analysis
  • scripts/setup_oidc_aws.py - Automate AWS OIDC setup
无需令牌即可执行的脚本：
  • scripts/validate_workflow.py - 验证YAML语法和最佳实践
  • scripts/generate_github_workflow.py - 从模板生成工作流
  • scripts/analyze_ci_performance.py - CI性能指标分析
  • scripts/setup_oidc_aws.py - 自动化AWS OIDC配置

Related Skills

相关技能

testing-strategies - Test execution strategies (unit, integration, E2E) deploying-applications - Deployment automation and GitOps auth-security - Secrets management and authentication observability - Pipeline monitoring and alerting
testing-strategies - 测试执行策略（单元测试、集成测试、E2E测试） deploying-applications - 部署自动化与GitOps auth-security - 密钥管理与认证 observability - 流水线监控与告警