Back to Details

gitlab-cicd-pipeline

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

GitLab CI/CD Pipeline

GitLab CI/CD 流水线

Overview

概述

Create comprehensive GitLab CI/CD pipelines that automate building, testing, and deployment using GitLab Runner infrastructure and container execution.
利用GitLab Runner基础设施和容器执行环境，创建可自动化构建、测试与部署的全面GitLab CI/CD流水线。

When to Use

适用场景

  • GitLab repository CI/CD setup
  • Multi-stage build pipelines
  • Docker registry integration
  • Kubernetes deployment
  • Review app deployment
  • Cache optimization
  • Dependency management
  • GitLab仓库CI/CD搭建
  • 多阶段构建流水线
  • Docker镜像仓库集成
  • Kubernetes部署
  • 预览应用部署
  • 缓存优化
  • 依赖管理

Implementation Examples

实现示例

1. Complete Pipeline Configuration

1. 完整流水线配置

yaml
undefined
yaml
undefined

.gitlab-ci.yml

.gitlab-ci.yml

image: node:18-alpine
variables: DOCKER_DRIVER: overlay2 FF_USE_FASTZIP: "true"
stages:
  • lint
  • test
  • build
  • security
  • deploy-review
  • deploy-prod
cache: key: ${CI_COMMIT_REF_SLUG} paths: - node_modules/ - .npm/
lint: stage: lint script: - npm install - npm run lint - npm run format:check artifacts: reports: codequality: code-quality-report.json expire_in: 1 week
unit-tests: stage: test script: - npm install - npm run test:coverage artifacts: reports: coverage_report: coverage_format: cobertura path: coverage/cobertura-coverage.xml junit: test-results.xml paths: - coverage/ expire_in: 1 week coverage: '/Coverage: \d+.\d+%/'
integration-tests: stage: test services: - postgres:13 - redis:7 variables: POSTGRES_DB: test_db POSTGRES_USER: test_user POSTGRES_PASSWORD: test_password script: - npm run test:integration only: - merge_requests - main
build: stage: build image: docker:latest services: - docker:dind variables: DOCKER_HOST: tcp://docker:2375 REGISTRY: registry.gitlab.com script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest - docker push $CI_REGISTRY_IMAGE:latest only: - main - tags
security-scan: stage: security image: alpine:latest script: - apk add --no-cache git - git clone https://github.com/aquasecurity/trivy.git - ./trivy image $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA allow_failure: true
deploy-review: stage: deploy-review environment: name: review/$CI_COMMIT_REF_SLUG url: https://$CI_COMMIT_REF_SLUG.review.example.com auto_stop_in: 1 week script: - helm upgrade --install review-$CI_COMMIT_REF_SLUG ./chart --set image.tag=$CI_COMMIT_SHA --set environment=review only: - merge_requests
deploy-prod: stage: deploy-prod environment: name: production url: https://example.com script: - helm upgrade --install prod ./chart --set image.tag=$CI_COMMIT_SHA --set environment=production only: - main when: manual
undefined
image: node:18-alpine
variables: DOCKER_DRIVER: overlay2 FF_USE_FASTZIP: "true"
stages:
  • lint
  • test
  • build
  • security
  • deploy-review
  • deploy-prod
cache: key: ${CI_COMMIT_REF_SLUG} paths: - node_modules/ - .npm/
lint: stage: lint script: - npm install - npm run lint - npm run format:check artifacts: reports: codequality: code-quality-report.json expire_in: 1 week
unit-tests: stage: test script: - npm install - npm run test:coverage artifacts: reports: coverage_report: coverage_format: cobertura path: coverage/cobertura-coverage.xml junit: test-results.xml paths: - coverage/ expire_in: 1 week coverage: '/Coverage: \d+.\d+%/'
integration-tests: stage: test services: - postgres:13 - redis:7 variables: POSTGRES_DB: test_db POSTGRES_USER: test_user POSTGRES_PASSWORD: test_password script: - npm run test:integration only: - merge_requests - main
build: stage: build image: docker:latest services: - docker:dind variables: DOCKER_HOST: tcp://docker:2375 REGISTRY: registry.gitlab.com script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest - docker push $CI_REGISTRY_IMAGE:latest only: - main - tags
security-scan: stage: security image: alpine:latest script: - apk add --no-cache git - git clone https://github.com/aquasecurity/trivy.git - ./trivy image $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA allow_failure: true
deploy-review: stage: deploy-review environment: name: review/$CI_COMMIT_REF_SLUG url: https://$CI_COMMIT_REF_SLUG.review.example.com auto_stop_in: 1 week script: - helm upgrade --install review-$CI_COMMIT_REF_SLUG ./chart --set image.tag=$CI_COMMIT_SHA --set environment=review only: - merge_requests
deploy-prod: stage: deploy-prod environment: name: production url: https://example.com script: - helm upgrade --install prod ./chart --set image.tag=$CI_COMMIT_SHA --set environment=production only: - main when: manual
undefined

2. GitLab Runner Configuration

2. GitLab Runner 配置

bash
#!/bin/bash
bash
#!/bin/bash

install-runner.sh

install-runner.sh

Register GitLab Runner

注册GitLab Runner

gitlab-runner register
--url https://gitlab.com/
--registration-token $RUNNER_TOKEN
--executor docker
--docker-image alpine:latest
--docker-privileged
--docker-volumes /certs/client
--description "Docker Runner"
--tag-list "docker,linux"
--run-untagged=false
--locked=false
--access-level not_protected
gitlab-runner register
--url https://gitlab.com/
--registration-token $RUNNER_TOKEN
--executor docker
--docker-image alpine:latest
--docker-privileged
--docker-volumes /certs/client
--description "Docker Runner"
--tag-list "docker,linux"
--run-untagged=false
--locked=false
--access-level not_protected

Start runner

启动Runner

gitlab-runner start
undefined
gitlab-runner start
undefined

3. Docker Layer Caching Optimization

3. Docker 层缓存优化

yaml
undefined
yaml
undefined

.gitlab-ci.yml

.gitlab-ci.yml

stages:
  • build
build-image: stage: build image: docker:latest services: - docker:dind variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
# Pull previous image for cache
- docker pull $CI_REGISTRY_IMAGE:latest || true

# Build with cache
- docker build
    --cache-from $CI_REGISTRY_IMAGE:latest
    --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    --tag $CI_REGISTRY_IMAGE:latest
    .

# Push images
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
cache: key: ${CI_COMMIT_REF_SLUG}-docker paths: - .docker/
undefined
stages:
  • build
build-image: stage: build image: docker:latest services: - docker:dind variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
# 拉取历史镜像用于缓存
- docker pull $CI_REGISTRY_IMAGE:latest || true

# 利用缓存构建
- docker build
    --cache-from $CI_REGISTRY_IMAGE:latest
    --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    --tag $CI_REGISTRY_IMAGE:latest
    .

# 推送镜像
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
cache: key: ${CI_COMMIT_REF_SLUG}-docker paths: - .docker/
undefined

4. Multi-Project Pipeline

4. 多项目流水线

yaml
undefined
yaml
undefined

.gitlab-ci.yml

.gitlab-ci.yml

stages:
  • build
  • test
  • deploy
build:backend: stage: build script: - cd backend && npm run build artifacts: paths: - backend/dist/
build:frontend: stage: build script: - cd frontend && npm run build artifacts: paths: - frontend/dist/
test:backend: stage: test needs: ["build:backend"] script: - cd backend && npm test artifacts: reports: junit: backend/test-results.xml
test:frontend: stage: test needs: ["build:frontend"] script: - cd frontend && npm test artifacts: reports: junit: frontend/test-results.xml
deploy: stage: deploy needs: ["test:backend", "test:frontend"] script: - echo "Deploying backend and frontend..." when: manual
undefined
stages:
  • build
  • test
  • deploy
build:backend: stage: build script: - cd backend && npm run build artifacts: paths: - backend/dist/
build:frontend: stage: build script: - cd frontend && npm run build artifacts: paths: - frontend/dist/
test:backend: stage: test needs: ["build:backend"] script: - cd backend && npm test artifacts: reports: junit: backend/test-results.xml
test:frontend: stage: test needs: ["build:frontend"] script: - cd frontend && npm test artifacts: reports: junit: frontend/test-results.xml
deploy: stage: deploy needs: ["test:backend", "test:frontend"] script: - echo "Deploying backend and frontend..." when: manual
undefined

5. Kubernetes Deployment

5. Kubernetes 部署

yaml
undefined
yaml
undefined

.gitlab-ci.yml

.gitlab-ci.yml

deploy-k8s: stage: deploy image: alpine/k8s:latest script: - mkdir -p $HOME/.kube - echo $KUBE_CONFIG_ENCODED | base64 -d > $HOME/.kube/config - chmod 600 $HOME/.kube/config
# Update image in deployment
- kubectl set image deployment/app-deployment
    app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    -n production

# Wait for rollout
- kubectl rollout status deployment/app-deployment -n production
environment: name: production kubernetes: namespace: production only: - main when: manual
undefined
deploy-k8s: stage: deploy image: alpine/k8s:latest script: - mkdir -p $HOME/.kube - echo $KUBE_CONFIG_ENCODED | base64 -d > $HOME/.kube/config - chmod 600 $HOME/.kube/config
# 更新部署中的镜像
- kubectl set image deployment/app-deployment
    app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    -n production

# 等待滚动更新完成
- kubectl rollout status deployment/app-deployment -n production
environment: name: production kubernetes: namespace: production only: - main when: manual
undefined

6. Performance Testing Stage

6. 性能测试阶段

yaml
undefined
yaml
undefined

.gitlab-ci.yml

.gitlab-ci.yml

performance: stage: test image: grafana/k6:latest script: - k6 run tests/performance.js artifacts: reports: performance: performance-results.json expire_in: 1 week allow_failure: true only: - main - merge_requests
undefined
performance: stage: test image: grafana/k6:latest script: - k6 run tests/performance.js artifacts: reports: performance: performance-results.json expire_in: 1 week allow_failure: true only: - main - merge_requests
undefined

7. Release Pipeline with Semantic Versioning

7. 基于语义化版本的发布流水线

yaml
undefined
yaml
undefined

.gitlab-ci.yml

.gitlab-ci.yml

release: stage: deploy-prod image: node:18-alpine script: - npm install -g semantic-release @semantic-release/gitlab
# Configure git
- git config user.email "ci@example.com"
- git config user.name "CI Bot"

# Run semantic-release
- semantic-release
only: - main when: manual
undefined
release: stage: deploy-prod image: node:18-alpine script: - npm install -g semantic-release @semantic-release/gitlab
# 配置Git
- git config user.email "ci@example.com"
- git config user.name "CI Bot"

# 运行语义化发布
- semantic-release
only: - main when: manual
undefined

Best Practices

最佳实践

✅ DO

✅ 建议做法

  • Use stages to organize pipeline flow
  • Implement caching for dependencies
  • Use artifacts for test reports
  • Set appropriate cache keys
  • Implement conditional execution with 
    only
    and 
    except
  • Use 
    needs:
    for job dependencies
  • Clean up artifacts with 
    expire_in
  • Use Docker for consistent environments
  • Implement security scanning stages
  • Set resource limits for jobs
  • Use merge request pipelines
  • 使用阶段组织流水线流程
  • 为依赖项实现缓存
  • 使用制品存储测试报告
  • 设置合适的缓存键
  • 用
    only
    和
    except
    实现条件执行
  • 用
    needs:
    定义作业依赖
  • 用
    expire_in
    清理制品
  • 使用Docker保证环境一致性
  • 实现安全扫描阶段
  • 为作业设置资源限制
  • 使用合并请求流水线

❌ DON'T

❌ 避免事项

  • Run tests serially when parallelizable
  • Cache everything unnecessarily
  • Leave large artifacts indefinitely
  • Store secrets in configuration files
  • Run privileged Docker without necessity
  • Skip security scanning
  • Ignore pipeline failures
  • Use 
    only: [main]
    without proper controls
  • 可并行的测试串行执行
  • 不必要地缓存所有内容
  • 永久保留大型制品
  • 在配置文件中存储密钥
  • 无必要地使用特权Docker
  • 跳过安全扫描
  • 忽略流水线失败
  • 无适当控制地使用
    only: [main]

Gitlab Runner Executor Types

GitLab Runner 执行器类型

bash
undefined
bash
undefined

Docker executor (recommended)

Docker执行器（推荐）

gitlab-runner register --executor docker
gitlab-runner register --executor docker

Kubernetes executor

Kubernetes执行器

gitlab-runner register --executor kubernetes
gitlab-runner register --executor kubernetes

Shell executor (local)

Shell执行器（本地）

gitlab-runner register --executor shell
gitlab-runner register --executor shell

Machine executor (for auto-scaling)

Machine执行器（用于自动扩缩容）

gitlab-runner register --executor machine
undefined
gitlab-runner register --executor machine
undefined

Resources

参考资源