monorepo-ci-optimizer
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseMonorepo CI Optimizer
Monorepo CI 优化器
Run only affected builds and tests in monorepos for faster CI.
在单体仓库中仅运行受影响代码的构建和测试,以加速CI流程。
Affected Detection Strategy
受影响代码检测策略
Using Turborepo
使用Turborepo
yaml
undefinedyaml
undefined.github/workflows/ci.yml
.github/workflows/ci.yml
name: CI
on:
pull_request:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Need full history for affected detection
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "pnpm"
- run: pnpm install
- name: Build affected
run: |
pnpm turbo run build --filter='...[origin/main]'
- name: Test affected
run: |
pnpm turbo run test --filter='...[origin/main]'
- name: Lint affected
run: |
pnpm turbo run lint --filter='...[origin/main]'undefinedname: CI
on:
pull_request:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Need full history for affected detection
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "pnpm"
- run: pnpm install
- name: Build affected
run: |
pnpm turbo run build --filter='...[origin/main]'
- name: Test affected
run: |
pnpm turbo run test --filter='...[origin/main]'
- name: Lint affected
run: |
pnpm turbo run lint --filter='...[origin/main]'undefinedUsing Nx
使用Nx
yaml
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: nrwl/nx-set-shas@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
- run: npm ci
- name: Build affected projects
run: npx nx affected:build --base=$NX_BASE --head=$NX_HEAD
- name: Test affected projects
run: npx nx affected:test --base=$NX_BASE --head=$NX_HEAD --parallel=3
- name: Lint affected projects
run: npx nx affected:lint --base=$NX_BASE --head=$NX_HEADyaml
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: nrwl/nx-set-shas@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
- run: npm ci
- name: Build affected projects
run: npx nx affected:build --base=$NX_BASE --head=$NX_HEAD
- name: Test affected projects
run: npx nx affected:test --base=$NX_BASE --head=$NX_HEAD --parallel=3
- name: Lint affected projects
run: npx nx affected:lint --base=$NX_BASE --head=$NX_HEADRemote Caching (Turborepo)
远程缓存(Turborepo)
yaml
- name: Setup Turborepo cache
uses: actions/cache@v3
with:
path: .turbo
key: ${{ runner.os }}-turbo-${{ github.sha }}
restore-keys: |
${{ runner.os }}-turbo-
- name: Build with remote cache
run: pnpm turbo run build
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ secrets.TURBO_TEAM }}yaml
- name: Setup Turborepo cache
uses: actions/cache@v3
with:
path: .turbo
key: ${{ runner.os }}-turbo-${{ github.sha }}
restore-keys: |
${{ runner.os }}-turbo-
- name: Build with remote cache
run: pnpm turbo run build
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ secrets.TURBO_TEAM }}Nx Cloud Integration
Nx Cloud 集成
yaml
- name: Setup Nx Cloud
run: |
npx nx-cloud start-ci-run --distribute-on="3 linux-medium-js"
env:
NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Run affected
run: |
npx nx affected --target=build --parallel=3
npx nx affected --target=test --parallel=3
npx nx affected --target=lint --parallel=3yaml
- name: Setup Nx Cloud
run: |
npx nx-cloud start-ci-run --distribute-on="3 linux-medium-js"
env:
NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Run affected
run: |
npx nx affected --target=build --parallel=3
npx nx affected --target=test --parallel=3
npx nx affected --target=lint --parallel=3Manual Affected Detection
手动受影响代码检测
typescript
// scripts/get-affected.ts
import * as path from "path";
import { execSync } from "child_process";
import * as fs from "fs";
interface Package {
name: string;
path: string;
dependencies: string[];
}
function getChangedFiles(base: string = "origin/main"): string[] {
const output = execSync(`git diff --name-only ${base}...HEAD`, {
encoding: "utf-8",
});
return output.split("\n").filter(Boolean);
}
function getPackages(): Package[] {
const packagesDir = path.join(process.cwd(), "packages");
return fs.readdirSync(packagesDir).map((name) => {
const packageJson = JSON.parse(
fs.readFileSync(path.join(packagesDir, name, "package.json"), "utf-8")
);
return {
name: packageJson.name,
path: `packages/${name}`,
dependencies: [
...Object.keys(packageJson.dependencies || {}),
...Object.keys(packageJson.devDependencies || {}),
],
};
});
}
function getAffectedPackages(): string[] {
const changedFiles = getChangedFiles();
const packages = getPackages();
const affected = new Set<string>();
// Direct changes
changedFiles.forEach((file) => {
packages.forEach((pkg) => {
if (file.startsWith(pkg.path)) {
affected.add(pkg.name);
}
});
});
// Dependent packages
let changed = true;
while (changed) {
changed = false;
packages.forEach((pkg) => {
if (!affected.has(pkg.name)) {
const hasAffectedDep = pkg.dependencies.some((dep) =>
affected.has(dep)
);
if (hasAffectedDep) {
affected.add(pkg.name);
changed = true;
}
}
});
}
return Array.from(affected);
}
// Output for GitHub Actions
const affected = getAffectedPackages();
console.log(`::set-output name=packages::${affected.join(",")}`);
console.log(`Affected packages: ${affected.join(", ")}`);typescript
// scripts/get-affected.ts
import * as path from "path";
import { execSync } from "child_process";
import * as fs from "fs";
interface Package {
name: string;
path: string;
dependencies: string[];
}
function getChangedFiles(base: string = "origin/main"): string[] {
const output = execSync(`git diff --name-only ${base}...HEAD`, {
encoding: "utf-8",
});
return output.split("\n").filter(Boolean);
}
function getPackages(): Package[] {
const packagesDir = path.join(process.cwd(), "packages");
return fs.readdirSync(packagesDir).map((name) => {
const packageJson = JSON.parse(
fs.readFileSync(path.join(packagesDir, name, "package.json"), "utf-8")
);
return {
name: packageJson.name,
path: `packages/${name}`,
dependencies: [
...Object.keys(packageJson.dependencies || {}),
...Object.keys(packageJson.devDependencies || {}),
],
};
});
}
function getAffectedPackages(): string[] {
const changedFiles = getChangedFiles();
const packages = getPackages();
const affected = new Set<string>();
// Direct changes
changedFiles.forEach((file) => {
packages.forEach((pkg) => {
if (file.startsWith(pkg.path)) {
affected.add(pkg.name);
}
});
});
// Dependent packages
let changed = true;
while (changed) {
changed = false;
packages.forEach((pkg) => {
if (!affected.has(pkg.name)) {
const hasAffectedDep = pkg.dependencies.some((dep) =>
affected.has(dep)
);
if (hasAffectedDep) {
affected.add(pkg.name);
changed = true;
}
}
});
}
return Array.from(affected);
}
// Output for GitHub Actions
const affected = getAffectedPackages();
console.log(`::set-output name=packages::${affected.join(",")}`);
console.log(`Affected packages: ${affected.join(", ")}`);Matrix Strategy for Affected Packages
针对受影响包的矩阵策略
yaml
detect-affected:
runs-on: ubuntu-latest
outputs:
packages: ${{ steps.affected.outputs.packages }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- id: affected
run: |
PACKAGES=$(node scripts/get-affected.ts)
echo "packages=$PACKAGES" >> $GITHUB_OUTPUT
test-affected:
needs: detect-affected
if: needs.detect-affected.outputs.packages != ''
runs-on: ubuntu-latest
strategy:
matrix:
package: ${{ fromJSON(needs.detect-affected.outputs.packages) }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "pnpm"
- run: pnpm install
- name: Test ${{ matrix.package }}
run: pnpm --filter ${{ matrix.package }} testyaml
detect-affected:
runs-on: ubuntu-latest
outputs:
packages: ${{ steps.affected.outputs.packages }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- id: affected
run: |
PACKAGES=$(node scripts/get-affected.ts)
echo "packages=$PACKAGES" >> $GITHUB_OUTPUT
test-affected:
needs: detect-affected
if: needs.detect-affected.outputs.packages != ''
runs-on: ubuntu-latest
strategy:
matrix:
package: ${{ fromJSON(needs.detect-affected.outputs.packages) }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "pnpm"
- run: pnpm install
- name: Test ${{ matrix.package }}
run: pnpm --filter ${{ matrix.package }} testWorkspace-aware Caching
工作区感知缓存
yaml
- name: Cache workspace builds
uses: actions/cache@v3
with:
path: |
packages/*/dist
packages/*/node_modules/.cache
key: ${{ runner.os }}-workspace-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('packages/**/*.ts') }}
restore-keys: |
${{ runner.os }}-workspace-${{ hashFiles('**/pnpm-lock.yaml') }}-
${{ runner.os }}-workspace-yaml
- name: Cache workspace builds
uses: actions/cache@v3
with:
path: |
packages/*/dist
packages/*/node_modules/.cache
key: ${{ runner.os }}-workspace-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('packages/**/*.ts') }}
restore-keys: |
${{ runner.os }}-workspace-${{ hashFiles('**/pnpm-lock.yaml') }}-
${{ runner.os }}-workspace-Parallel Execution
并行执行
yaml
- name: Build all packages in parallel
run: pnpm turbo run build --parallel
- name: Test with controlled parallelism
run: pnpm turbo run test --concurrency=3yaml
- name: Build all packages in parallel
run: pnpm turbo run build --parallel
- name: Test with controlled parallelism
run: pnpm turbo run test --concurrency=3Optimization Metrics
优化指标
markdown
undefinedmarkdown
undefinedBefore Optimization
优化前
- Full build: 25 minutes
- All tests: 15 minutes
- Total: 40 minutes
- Runs on every PR
- 完整构建:25分钟
- 全部测试:15分钟
- 总计:40分钟
- 每个PR都会运行
After Affected Detection
启用受影响代码检测后
- Affected build: 5 minutes (80% reduction)
- Affected tests: 3 minutes (80% reduction)
- Total: 8 minutes (80% reduction)
- Only runs necessary checks
- 受影响代码构建:5分钟(减少80%)
- 受影响代码测试:3分钟(减少80%)
- 总计:8分钟(减少80%)
- 仅运行必要的检查
With Remote Caching
启用远程缓存后
- Cache hit build: 30 seconds (98% reduction)
- Cache hit tests: 1 minute (93% reduction)
- Total: 1.5 minutes (96% reduction)
undefined- 缓存命中构建:30秒(减少98%)
- 缓存命中测试:1分钟(减少93%)
- 总计:1.5分钟(减少96%)
undefinedBest Practices
最佳实践
- Fetch full history: for accurate diffs
fetch-depth: 0 - Topological order: Build dependencies first
- Remote caching: Share cache across CI runs
- Parallel execution: Run independent tasks concurrently
- Incremental builds: Only rebuild what changed
- Dependency graph: Track package relationships
- Force full build: On main branch merges
- 拉取完整历史:设置以获取准确的差异对比
fetch-depth: 0 - 拓扑顺序:优先构建依赖项
- 远程缓存:在CI运行之间共享缓存
- 并行执行:同时运行独立任务
- 增量构建:仅重新构建变更的内容
- 依赖图谱:跟踪包之间的依赖关系
- 强制完整构建:在主分支合并时执行
Output Checklist
输出检查清单
- Affected detection implemented
- Turborepo or Nx configured
- Remote caching enabled
- Parallel execution optimized
- Matrix strategy for packages
- Workspace-aware caching
- Dependency graph documented
- Before/after metrics tracked
- 已实现受影响代码检测
- 已配置Turborepo或Nx
- 已启用远程缓存
- 已优化并行执行
- 已针对包配置矩阵策略
- 已启用工作区感知缓存
- 已记录依赖图谱
- 已跟踪优化前后的指标