Back to Details

pipeline-variable-management

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

流水线变量管理完整指南

Complete Guide to Pipeline Variable Management

Skill 概述

Skill Overview

Skill 名称: Pipeline Variable Management
适用场景: 流水线变量的全生命周期管理与字段扩展
重要性: ⭐⭐⭐⭐ (高优先级)
文档版本: 2.0
最后更新: 2025-01
流水线变量是 BK-CI 流水线中的核心功能，支持在流水线编排、构建执行过程中动态传递和管理数据。本 Skill 提供变量管理的两大核心主题：
  1. 变量生命周期 - 变量从创建到销毁的完整流程
  2. 变量字段扩展 - 如何扩展变量的数据结构
Skill Name: Pipeline Variable Management
Applicable Scenarios: Full lifecycle management and field extension of pipeline variables
Importance: ⭐⭐⭐⭐ (High Priority)
Document Version: 2.0
Last Updated: 2025-01
Pipeline variables are a core feature in BK-CI pipelines, supporting dynamic transfer and management of data during pipeline orchestration and build execution. This Skill provides two core themes for variable management:
  1. Variable Lifecycle - Complete process of variables from creation to destruction
  2. Variable Field Extension - How to extend the data structure of variables

一、变量管理架构概览

I. Overview of Variable Management Architecture

1.1 变量的定义与作用

1.1 Definition and Role of Variables

流水线变量是流水线中用于存储和传递数据的核心机制，支持：
  • ✅ 配置参数化 - 将流水线配置中的固定值替换为变量
  • ✅ 数据传递 - 在不同 Stage/Job/Task 之间传递数据
  • ✅ 动态计算 - 运行时动态生成和更新变量值
  • ✅ 用户输入 - 手动触发时由用户输入参数值
  • ✅ 系统内置 - 提供系统级别的预定义变量
Pipeline variables are the core mechanism for storing and transferring data in pipelines, supporting:
  • ✅ Configuration Parameterization - Replace fixed values in pipeline configurations with variables
  • ✅ Data Transfer - Transfer data between different Stage/Job/Task
  • ✅ Dynamic Calculation - Dynamically generate and update variable values at runtime
  • ✅ User Input - Users input parameter values when triggering manually
  • ✅ System Built-in - Provide system-level predefined variables

1.2 变量的两种核心模型

1.2 Two Core Models of Variables

BK-CI 中流水线变量存在两种数据模型的双向转换：
There is bidirectional conversion between two data models for pipeline variables in BK-CI:

1. BuildFormProperty（后端内部模型）

1. BuildFormProperty (Backend Internal Model)

  • 位置: 
    common-pipeline/src/main/kotlin/.../BuildFormProperty.kt
  • 用途: 微服务内部使用，数据库存储格式
  • 特点: 完整的字段定义，包含 Swagger 注解
  • 应用: Process 模块内部处理、数据持久化
  • Location: 
    common-pipeline/src/main/kotlin/.../BuildFormProperty.kt
  • Purpose: Used internally in microservices, database storage format
  • Features: Complete field definition, including Swagger annotations
  • Application: Internal processing of Process module, data persistence

2. Variable（YAML 模型）

2. Variable (YAML Model)

  • 位置: 
    common-pipeline-yaml/src/main/kotlin/.../yaml/v3/models/Variable.kt
  • 用途: YAML 流水线定义，对外 API 交互
  • 特点: Jackson 注解，支持 JSON 序列化
  • 应用: PAC 流水线、OpenAPI 接口
  • Location: 
    common-pipeline-yaml/src/main/kotlin/.../yaml/v3/models/Variable.kt
  • Purpose: YAML pipeline definition, external API interaction
  • Features: Jackson annotations, supports JSON serialization
  • Application: PAC pipelines, OpenAPI interfaces

3. 转换器（VariableTransfer）

3. Converter (VariableTransfer)

  • 位置: 
    common-pipeline-yaml/src/main/kotlin/.../yaml/transfer/VariableTransfer.kt
  • 职责: 实现 
    BuildFormProperty
    ↔ 
    Variable
    的双向转换
  • 核心方法: 
    • transfer(BuildFormProperty): Variable
      - 内部模型 → YAML 模型
    • transfer(Variable): BuildFormProperty
      - YAML 模型 → 内部模型
  • Location: 
    common-pipeline-yaml/src/main/kotlin/.../yaml/transfer/VariableTransfer.kt
  • Responsibilities: Implements bidirectional conversion between 
    BuildFormProperty
    ↔ 
    Variable
  • Core Methods: 
    • transfer(BuildFormProperty): Variable
      - Internal model → YAML model
    • transfer(Variable): BuildFormProperty
      - YAML model → Internal model

1.3 变量的存储与传递

1.3 Storage and Transfer of Variables

┌─────────────────────────────────────────────────────────────────┐
│                       变量存储与传递架构                          │
└─────────────────────────────────────────────────────────────────┘

  ┌──────────────────┐
  │  流水线配置存储   │  BuildFormProperty
  │  (T_PIPELINE_*) │  存储在 Model JSON 中
  └────────┬─────────┘
           │
           ▼
  ┌──────────────────┐
  │   构建启动       │  读取变量定义
  │  (StartBuild)   │  初始化变量值
  └────────┬─────────┘
           │
           ▼
  ┌──────────────────┐
  │  运行时变量存储   │  T_PIPELINE_BUILD_VAR
  │  (BuildVar)     │  构建级别的变量实例
  └────────┬─────────┘
           │
           ├─────────────────┬─────────────────┐
           ▼                 ▼                 ▼
  ┌────────────────┐ ┌────────────────┐ ┌────────────────┐
  │   Stage 1      │ │   Stage 2      │ │   Stage 3      │
  │  继承 + 新增   │ │  继承 + 更新   │ │  继承 + 输出   │
  └────────────────┘ └────────────────┘ └────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│                       Variable Storage and Transfer Architecture                          │
└─────────────────────────────────────────────────────────────────┘

  ┌──────────────────┐
  │  Pipeline Configuration Storage   │  BuildFormProperty
  │  (T_PIPELINE_*) │  Stored in Model JSON
  └────────┬─────────┘
           │
           ▼
  ┌──────────────────┐
  │   Build Start       │  Read variable definitions
  │  (StartBuild)   │  Initialize variable values
  └────────┬─────────┘
           │
           ▼
  ┌──────────────────┐
  │  Runtime Variable Storage   │  T_PIPELINE_BUILD_VAR
  │  (BuildVar)     │  Build-level variable instances
  └────────┬─────────┘
           │
           ├─────────────────┬─────────────────┐
           ▼                 ▼                 ▼
  ┌────────────────┐ ┌────────────────┐ ┌────────────────┐
  │   Stage 1      │ │   Stage 2      │ │   Stage 3      │
  │  Inherit + Add   │ │  Inherit + Update   │ │  Inherit + Output   │
  └────────────────┘ └────────────────┘ └────────────────┘

二、两大核心主题

II. Two Core Themes

2.1 变量生命周期管理

2.1 Variable Lifecycle Management

覆盖内容：
  • 🔄 变量创建与初始化
  • 📦 变量存储机制
  • 🔀 变量传递与继承
  • 🔄 变量动态更新
  • 🔍 变量查询与调试
典型场景：
  • 理解变量在构建执行过程中的完整流转
  • 开发新的变量初始化逻辑
  • 调试变量传递问题
  • 实现变量的动态更新功能
查阅文档: reference/1-lifecycle.md (1414 行)
Covered Content:
  • 🔄 Variable Creation and Initialization
  • 📦 Variable Storage Mechanism
  • 🔀 Variable Transfer and Inheritance
  • 🔄 Dynamic Variable Update
  • 🔍 Variable Query and Debugging
Typical Scenarios:
  • Understand the complete flow of variables during build execution
  • Develop new variable initialization logic
  • Debug variable transfer issues
  • Implement dynamic variable update functions
Refer to Documentation: reference/1-lifecycle.md (1414 lines)

2.2 变量字段扩展

2.2 Variable Field Extension

覆盖内容：
  • 🆕 新增变量字段
  • 🔧 字段类型定义
  • 🔄 模型转换处理
  • 🎯 前后端字段同步
典型场景：
  • 为变量添加新的属性字段
  • 扩展变量类型
  • 处理历史数据兼容性
  • 同步前后端字段定义
查阅文档: reference/2-extension.md (534 行)
Covered Content:
  • 🆕 Add new variable fields
  • 🔧 Field type definition
  • 🔄 Model conversion processing
  • 🎯 Frontend and backend field synchronization
Typical Scenarios:
  • Add new attribute fields to variables
  • Extend variable types
  • Handle historical data compatibility
  • Synchronize frontend and backend field definitions
Refer to Documentation: reference/2-extension.md (534 lines)

三、使用指南

III. Usage Guide

3.1 场景 1：理解变量生命周期

3.1 Scenario 1: Understand Variable Lifecycle

问题示例：
  • "变量值是如何从流水线配置传递到构建执行的？"
  • "插件如何获取和更新变量值？"
  • "变量的作用域是如何控制的？"
操作： 查阅 reference/1-lifecycle.md
内容包含：
  1. 变量生命周期全景图
  2. 六个关键阶段详解
  3. 变量存储表结构
  4. 变量传递机制
  5. 调试方法与常见问题
Example Questions:
  • "How are variable values transferred from pipeline configurations to build execution?"
  • "How do plugins obtain and update variable values?"
  • "How is the scope of variables controlled?"
Action: Refer to reference/1-lifecycle.md
Content Includes:
  1. Panoramic view of variable lifecycle
  2. Detailed explanation of six key stages
  3. Variable storage table structure
  4. Variable transfer mechanism
  5. Debugging methods and common issues

3.2 场景 2：扩展变量字段

3.2 Scenario 2: Extend Variable Fields

问题示例：
  • "如何为变量添加一个新的 
    allowModify
    字段？"
  • "如何扩展变量的可选值列表？"
  • "如何确保新字段在 YAML 和内部模型之间正确转换？"
操作： 查阅 reference/2-extension.md
内容包含：
  1. 变量字段扩展完整路径
  2. BuildFormProperty 字段定义
  3. Variable (YAML) 字段定义
  4. VariableTransfer 转换器实现
  5. 前端字段同步
  6. 测试与验证
Example Questions:
  • "How to add a new 
    allowModify
    field to variables?"
  • "How to extend the optional value list of variables?"
  • "How to ensure correct conversion of new fields between YAML and internal models?"
Action: Refer to reference/2-extension.md
Content Includes:
  1. Complete path for variable field extension
  2. BuildFormProperty field definition
  3. Variable (YAML) field definition
  4. VariableTransfer converter implementation
  5. Frontend field synchronization
  6. Testing and verification

四、核心类与文件速查

IV. Quick Reference for Core Classes and Files

4.1 变量定义类

4.1 Variable Definition Classes

类名位置用途
BuildFormProperty
common-pipeline/.../BuildFormProperty.kt
后端内部变量模型
Variable
common-pipeline-yaml/.../Variable.kt
YAML 变量模型
VariableTransfer
common-pipeline-yaml/.../VariableTransfer.kt
模型转换器
Class NameLocationPurpose
BuildFormProperty
common-pipeline/.../BuildFormProperty.kt
Backend internal variable model
Variable
common-pipeline-yaml/.../Variable.kt
YAML variable model
VariableTransfer
common-pipeline-yaml/.../VariableTransfer.kt
Model converter

4.2 变量处理服务

4.2 Variable Processing Services

类名位置职责
PipelineVariableService
process/biz-base/.../PipelineVariableService.kt
变量业务逻辑
BuildVariableService
process/biz-base/.../BuildVariableService.kt
构建变量管理
VariableAcrossInfoUtil
process/biz-base/.../VariableAcrossInfoUtil.kt
跨 Job 变量传递
Class NameLocationResponsibility
PipelineVariableService
process/biz-base/.../PipelineVariableService.kt
Variable business logic
BuildVariableService
process/biz-base/.../BuildVariableService.kt
Build variable management
VariableAcrossInfoUtil
process/biz-base/.../VariableAcrossInfoUtil.kt
Cross-Job variable transfer

4.3 数据库表

4.3 Database Tables

表名用途
T_PIPELINE_BUILD_VAR
构建级别的变量实例（运行时）
T_PIPELINE_SETTING
流水线配置（包含变量定义）
T_PIPELINE_MODEL_TASK
插件配置（引用变量）
Table NamePurpose
T_PIPELINE_BUILD_VAR
Build-level variable instances (runtime)
T_PIPELINE_SETTING
Pipeline configuration (includes variable definitions)
T_PIPELINE_MODEL_TASK
Plugin configuration (references variables)

五、开发流程

V. Development Process

5.1 变量功能开发流程

5.1 Variable Function Development Process

1. 明确需求
   ↓
2. 确定涉及的生命周期阶段
   │  → 查阅 reference/1-lifecycle.md
   ↓
3. 是否需要扩展字段？
   ├─ 是 → 查阅 reference/2-extension.md
   │       执行字段扩展流程
   ↓
4. 实现后端逻辑
   │  → 修改 Service/DAO 层
   ↓
5. 前端同步（如需）
   │  → 更新 Vue 组件
   ↓
6. 测试验证
   │  → 单元测试 + 集成测试
   ↓
7. 文档更新
1. Clarify Requirements
   ↓
2. Determine the involved lifecycle stages
   │  → Refer to reference/1-lifecycle.md
   ↓
3. Need to extend fields?
   ├─ Yes → Refer to reference/2-extension.md
   │       Execute field extension process
   ↓
4. Implement Backend Logic
   │  → Modify Service/DAO layers
   ↓
5. Frontend Synchronization (if needed)
   │  → Update Vue components
   ↓
6. Testing and Verification
   │  → Unit tests + Integration tests
   ↓
7. Document Update

5.2 常见开发任务

5.2 Common Development Tasks

任务参考文档关键类
新增变量类型2-extension.md
BuildFormProperty
, 
Variable
变量初始化逻辑1-lifecycle.md
BuildVariableService
跨 Job 传递1-lifecycle.md
VariableAcrossInfoUtil
变量权限控制1-lifecycle.md
PipelineVariableService
变量表达式解析-
utility-components
Skill (reference/2-expression-parser.md)
TaskReference DocumentKey Classes
Add new variable types2-extension.md
BuildFormProperty
, 
Variable
Variable initialization logic1-lifecycle.md
BuildVariableService
Cross-Job transfer1-lifecycle.md
VariableAcrossInfoUtil
Variable permission control1-lifecycle.md
PipelineVariableService
Variable expression parsing-
utility-components
Skill (reference/2-expression-parser.md)

六、与其他 Skill 的关系

VI. Relationship with Other Skills

6.1 依赖的 Skill

6.1 Dependent Skills

  • pipeline-model-architecture
     - 理解变量在 Model 中的位置
  • process-module-architecture
     - 理解变量处理的代码架构
  • utility-components
     (reference/2-expression-parser.md) - 变量引用的表达式解析
  • pipeline-model-architecture
     - Understand the position of variables in the Model
  • process-module-architecture
     - Understand the code architecture of variable processing
  • utility-components
     (reference/2-expression-parser.md) - Expression parsing for variable references

6.2 被引用的场景

6.2 Referenced Scenarios

  • PAC 流水线 → 变量在 YAML 中的定义
  • 流水线编排 → 变量的创建和配置
  • 构建执行 → 变量的初始化和传递
  • 插件开发 → 插件如何读取和更新变量
  • PAC Pipelines → Definition of variables in YAML
  • Pipeline Orchestration → Creation and configuration of variables
  • Build Execution → Initialization and transfer of variables
  • Plugin Development → How plugins read and update variables

七、详细文档导航

VII. Detailed Document Navigation

文档内容行数文件大小
1-lifecycle.md变量生命周期完整流程141478.7KB
2-extension.md变量字段扩展指南53420.2KB
使用建议:
  • 初学者：先阅读本文档了解整体架构，再深入 1-lifecycle.md
  • 开发者：根据具体开发任务直接查阅对应的 reference 文档
  • 调试：优先查阅 1-lifecycle.md 中的调试章节
DocumentContentLinesFile Size
1-lifecycle.mdComplete variable lifecycle process141478.7KB
2-extension.mdVariable field extension guide53420.2KB
Usage Suggestions:
  • Beginners: First read this document to understand the overall architecture, then dive into 1-lifecycle.md
  • Developers: Directly refer to the corresponding reference document according to specific development tasks
  • Debugging: Prioritize checking the debugging section in 1-lifecycle.md

八、常见问题 FAQ

VIII. FAQ

Q1: 变量和参数有什么区别？

Q1: What's the difference between variables and parameters?

A: 在 BK-CI 中，"变量"和"参数"本质上是同一个概念，都是 
BuildFormProperty
。区别在于使用场景：
  • 参数：流水线启动时用户输入的变量
  • 变量：流水线内部定义和使用的变量
A: In BK-CI, "variables" and "parameters" are essentially the same concept, both referring to 
BuildFormProperty
. The difference lies in usage scenarios:
  • Parameters: Variables input by users when starting a pipeline
  • Variables: Variables defined and used internally in the pipeline

Q2: 如何实现跨 Stage 传递变量？

Q2: How to implement cross-Stage variable transfer?

A: 查阅 reference/1-lifecycle.md 中的"阶段四：跨容器传递"章节，了解 
VariableAcrossInfoUtil
的使用。
A: Refer to the "Stage 4: Cross-container Transfer" section in reference/1-lifecycle.md to learn how to use 
VariableAcrossInfoUtil
.

Q3: 新增字段后前端不显示怎么办？

Q3: What to do if the frontend doesn't display after adding a new field?

A: 按照 reference/2-extension.md 的"步骤 4：前端字段同步"章节，确保前端 Vue 组件已同步更新。
A: Follow the "Step 4: Frontend Field Synchronization" section in reference/2-extension.md to ensure that the frontend Vue components have been updated synchronously.

Q4: 变量值为空或不正确？

Q4: Variable values are empty or incorrect?

A: 查阅 reference/1-lifecycle.md 中的"调试方法"章节，使用日志和数据库排查。
A: Refer to the "Debugging Methods" section in reference/1-lifecycle.md and use logs and databases to troubleshoot.

九、版本历史

IX. Version History

版本日期更新内容
2.02025-01整合 
pipeline-variable-lifecycle
和 
pipeline-variable-extension
，采用 reference 结构
1.x2024-12独立的两个 Skill
VersionDateUpdate Content
2.02025-01Integrate 
pipeline-variable-lifecycle
and 
pipeline-variable-extension
, adopt reference structure
1.x2024-12Two independent Skills