Back to Details

ai-bug-triage

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
<objective> A hybrid pipeline for bug classification, deduplication, and ticket generation. Deterministic fingerprinting handles deduplication (what LLMs are bad at); LLM handles explanation, severity assessment, and ticket writing (what LLMs are good at).
Key reframe: The LLM is best at explaining and routing, not deduplication. Teach agents to DESIGN the pipeline, not BE the pipeline. </objective>
<目标> 一种用于bug分类、去重和工单生成的混合工作流。确定性指纹处理去重（LLM不擅长的任务）；LLM负责解释、严重度评估和工单撰写（LLM擅长的任务）。
核心重构思路： LLM最适合用于解释和路由，而非去重。应训练Agent设计工作流，而非让Agent成为工作流本身。 </目标>

Discovery Questions

探索问题

Check 
.agents/qa-project-context.md
first — it carries tech stack, component mapping, and known flaky areas that improve classification accuracy. Use it and skip anything already answered there. Then clarify:
  1. What is the failure source?
    • CI pipeline logs (GitHub Actions, GitLab CI, Jenkins, CircleCI)
    • Test framework output (Playwright, Jest, pytest, Vitest)
    • Production error monitoring (Sentry, Datadog, Bugsnag)
    • Manual bug reports from QA or users
  2. What is the ticket destination?
    • Jira, Linear, GitHub Issues, Azure DevOps, Shortcut
    • What fields are required? (component, severity, priority, labels)
    • What workflows exist? (triage board, auto-assignment rules)
  3. What is the deduplication scope?
    • Same test run? Same sprint? Same release? All time?
    • Do you already have fingerprinting? What is the current duplicate rate?
  4. What approval workflow is needed?
    • Auto-create tickets with human review?
    • Suggest tickets for human approval before creation?
    • Auto-close duplicates? (dangerous -- require approval)
  5. What historical data exists?
    • Past bug reports with resolution data?
    • Flaky test history? Known environment issues?
    • Component ownership mapping?
请先查看
.agents/qa-project-context.md
——其中包含技术栈、组件映射和已知不稳定区域信息，可提升分类准确性。若已有答案则跳过该部分，再明确以下问题：
  1. 故障来源是什么？
    • CI流水线日志（GitHub Actions、GitLab CI、Jenkins、CircleCI）
    • 测试框架输出（Playwright、Jest、pytest、Vitest）
    • 生产错误监控（Sentry、Datadog、Bugsnag）
    • QA或用户提交的手动bug报告
  2. 工单目标系统是什么？
    • Jira、Linear、GitHub Issues、Azure DevOps、Shortcut
    • 需要哪些必填字段？（组件、严重度、优先级、标签）
    • 存在哪些工作流？（分流看板、自动分配规则）
  3. 去重范围是什么？
    • 同一次测试运行？同一个迭代？同一个版本？全时段？
    • 您是否已使用指纹识别？当前重复率是多少？
  4. 需要哪些审批工作流？
    • 自动创建工单后由人工审核？
    • 先生成工单建议，经人工批准后再创建？
    • 自动关闭重复工单？（此操作有风险——需审批）
  5. 存在哪些历史数据？
    • 带有解决记录的过往bug报告？
    • 不稳定测试历史？已知环境问题？
    • 组件负责人映射表？

Core Principles

核心原则

  1. Deterministic first, LLM second. Use stable, reproducible fingerprinting for deduplication and clustering. Use LLM only for tasks requiring understanding: severity classification, root cause hypothesis, and human-readable ticket writing.
  2. Normalize before comparing. Raw CI logs are full of timestamps, port numbers, process IDs, and random suffixes that make identical failures look different. Strip all noise before fingerprinting.
  3. Fingerprints are anchored to stable elements. Exception type, top stack frames, test name, error message template, and URL pattern are stable. Timestamps, request IDs, and ephemeral ports are not.
  4. Human approval before destructive actions. Auto-closing a ticket as duplicate or auto-merging reports requires human confirmation. False deduplication wastes more time than manual triage.
  5. Classification drives routing. The value of triage is not the label itself but the routing decision it enables: which team, what priority, what SLA.
  6. Track triage accuracy. Measure how often auto-classification matches human judgment. Below 85% accuracy, the pipeline needs tuning.
  1. 先确定性处理，后LLM处理。 使用稳定、可复现的指纹识别进行去重和聚类。仅在需要理解能力的任务中使用LLM：严重度分类、根因假设和易读工单撰写。
  2. 先标准化再对比。 原始CI日志包含大量时间戳、端口号、进程ID和随机后缀，会导致相同故障看起来不同。指纹识别前需去除所有干扰信息。
  3. 指纹锚定稳定元素。 异常类型、顶部栈帧、测试名称、错误消息模板和URL模式是稳定元素。时间戳、请求ID和临时端口则不是。
  4. 破坏性操作需人工审批。 自动关闭工单为重复项或自动合并报告需要人工确认。错误去重比手动分流更耗时。
  5. 分类驱动路由。 分流的价值不在于标签本身，而在于其支持的路由决策：分配给哪个团队、优先级如何、SLA是什么。
  6. 跟踪分流准确性。 衡量自动分类与人工判断的匹配度。准确率低于85%时，工作流需要调整。

The Pipeline

工作流

CI Log / Error Report
  │
  ▼
Step 1: NORMALIZE
  Strip timestamps, process IDs, ports, random suffixes, ANSI codes
  │
  ▼
Step 2: EXTRACT STABLE ANCHORS
  Exception type, top N stack frames, test name, error message template, URL pattern
  │
  ▼
Step 3: HASH CANONICAL FORM
  Deterministic fingerprint from ordered anchors
  │
  ▼
Step 4: CLUSTER NEAR-DUPLICATES
  Similarity scoring for non-identical but related failures
  │
  ▼
Step 5: LLM CLASSIFY
  Severity, component, suspected root cause, failure category
  │
  ▼
Step 6: LLM GENERATE TICKET
  Title, description, repro steps, evidence, suggested assignee
  │
  ▼
Step 7: HUMAN APPROVAL
  Review before create/close/merge
CI日志 / 错误报告
  │
  ▼
步骤1：标准化
  去除时间戳、进程ID、端口、随机后缀、ANSI代码
  │
  ▼
步骤2：提取稳定锚点
  异常类型、前N个栈帧、测试名称、错误消息模板、URL模式
  │
  ▼
步骤3：哈希规范化格式
  基于有序锚点生成确定性指纹
  │
  ▼
步骤4：聚类近似重复项
  对非完全相同但相关的故障进行相似度评分
  │
  ▼
步骤5：LLM分类
  严重度、组件、疑似根因、故障类别
  │
  ▼
步骤6：LLM生成工单
  标题、描述、复现步骤、证据、建议负责人
  │
  ▼
步骤7：人工审批
  创建/关闭/合并前进行审核

Step 1: Normalize

步骤1：标准化

Strip noise that makes identical failures look different.
Normalization rules (apply in order):
1. Strip ANSI color codes:        \x1b\[[0-9;]*m → ""
2. Strip timestamps:              \d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}[.\d]*Z? → "<TIMESTAMP>"
3. Strip UUIDs:                   [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} → "<UUID>"
4. Strip process IDs:             pid[=: ]\d+ → "pid=<PID>"
5. Strip port numbers:            :\d{4,5}(?=[\s/,)\]]|$) → ":<PORT>"
6. Strip temp file paths:         /tmp/[^\s]+ → "<TMPPATH>"
7. Strip memory addresses:        0x[0-9a-f]{8,16} → "<ADDR>"
8. Strip random suffixes:         [-_][a-z0-9]{6,8}(?=\.) → "<RAND>"
9. Strip request IDs:             (?:request[_-]?id|trace[_-]?id|correlation[_-]?id)[=: ]["']?[a-zA-Z0-9-]+ → "<REQ_ID>"
10. Collapse whitespace:          \s+ → " "
Example:
Before: 2025-03-22T14:32:01.456Z [pid=42891] Error: Connection refused at 127.0.0.1:54321
        request_id=abc-123-def-456
After:  <TIMESTAMP> [pid=<PID>] Error: Connection refused at 127.0.0.1:<PORT>
        <REQ_ID>
Rule 5 strips the port but not the literal loopback IP — 
127.0.0.1
stays in the fingerprint. That's fine for same-host failures, but two runners that bind different hosts (e.g. 
127.0.0.1
vs 
0.0.0.0
) will split into separate fingerprints. If you run heterogeneous hosts, add a rule to normalize bind addresses too.
去除导致相同故障看起来不同的干扰信息。
标准化规则（按顺序执行）：
1. 去除ANSI颜色代码:        \x1b\[[0-9;]*m → ""
2. 去除时间戳:              \d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}[.\d]*Z? → "<TIMESTAMP>"
3. 去除UUID:                   [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} → "<UUID>"
4. 去除进程ID:             pid[=: ]\d+ → "pid=<PID>"
5. 去除端口号:            :\d{4,5}(?=[\s/,)\]]|$) → ":<PORT>"
6. 去除临时文件路径:         /tmp/[^\s]+ → "<TMPPATH>"
7. 去除内存地址:        0x[0-9a-f]{8,16} → "<ADDR>"
8. 去除随机后缀:         [-_][a-z0-9]{6,8}(?=\.) → "<RAND>"
9. 去除请求ID:             (?:request[_-]?id|trace[_-]?id|correlation[_-]?id)[=: ]["']?[a-zA-Z0-9-]+ → "<REQ_ID>"
10. 合并空白字符:          \s+ → " "
示例：
标准化前: 2025-03-22T14:32:01.456Z [pid=42891] Error: Connection refused at 127.0.0.1:54321
        request_id=abc-123-def-456
标准化后:  <TIMESTAMP> [pid=<PID>] Error: Connection refused at 127.0.0.1:<PORT>
        <REQ_ID>
规则5会去除端口，但保留字面回环IP——
127.0.0.1
会留在指纹中。这对于同一主机的故障没问题，但如果两个运行器绑定不同主机（如
127.0.0.1
vs 
0.0.0.0
），会生成不同的指纹。如果您使用异构主机，请添加规则标准化绑定地址。

Step 2: Extract Stable Anchors

步骤2：提取稳定锚点

From the normalized log, extract elements that identify the failure regardless of environment or timing.
Anchor types (in priority order):
AnchorExampleStability
Exception type
TypeError
, 
AssertionError
, 
HTTP 500
Very high
Error message template
Cannot read property 'X' of undefined
High
Top 3 stack frames
at processOrder (order.ts:142)
High
Test name
checkout.spec.ts > completes payment
Very high
URL pattern
POST /api/orders
High
HTTP status code
500
, 
429
, 
503
Very high
Exit code
exit code 1
, 
SIGKILL
High
Assertion diff
Expected: 200, Received: 500
Medium
Extraction rules:
  • Keep function names but strip line numbers (they change with edits)
  • Keep URL paths but strip query parameters and IDs in paths (
    /api/orders/<ID>
    )
  • Keep error message structure but replace dynamic values with placeholders
  • Keep test file and test name exactly as-is
从标准化日志中提取可识别故障的元素，不受环境或时间影响。
锚点类型（按优先级排序）：
锚点示例稳定性
异常类型
TypeError
, 
AssertionError
, 
HTTP 500
极高
错误消息模板
Cannot read property 'X' of undefined
高
前3个栈帧
at processOrder (order.ts:142)
高
测试名称
checkout.spec.ts > completes payment
极高
URL模式
POST /api/orders
高
HTTP状态码
500
, 
429
, 
503
极高
退出码
exit code 1
, 
SIGKILL
高
断言差异
Expected: 200, Received: 500
中等
提取规则：
  • 保留函数名称，但去除行号（代码编辑后会变化）
  • 保留URL路径，但去除查询参数和路径中的ID（
    /api/orders/<ID>
    ）
  • 保留错误消息结构，但将动态值替换为占位符
  • 完全保留测试文件和测试名称

Step 3: Hash Canonical Form

步骤3：哈希规范化格式

Create a deterministic fingerprint from the extracted anchors.
Algorithm:
1. Sort anchors alphabetically by type
2. Concatenate: exception_type + "|" + message_template + "|" + top_frames + "|" + test_name
3. SHA-256 hash the concatenated string
4. Take first 16 hex characters as fingerprint
Fingerprint properties:
  • Same failure always produces same fingerprint (deterministic)
  • Different failures produce different fingerprints (collision-resistant)
  • Minor log format changes do not change fingerprint (stable)
  • Fingerprint is short enough for Jira labels and GitHub tags
Example:
Anchors:
  exception_type: "TypeError"
  message_template: "Cannot read property 'vendorId' of undefined"
  top_frames: "processOrder|groupByVendor|checkout"
  test_name: "checkout.spec.ts > multi-vendor checkout"

Canonical: "TypeError|Cannot read property 'vendorId' of undefined|processOrder|groupByVendor|checkout|checkout.spec.ts > multi-vendor checkout"
Fingerprint: a3f8b2c1e9d04567
基于提取的锚点生成确定性指纹。
算法：
1. 按类型对锚点进行字母排序
2. 拼接字符串：exception_type + "|" + message_template + "|" + top_frames + "|" + test_name
3. 对拼接后的字符串进行SHA-256哈希
4. 取前16个十六进制字符作为指纹
指纹特性：
  • 相同故障始终生成相同指纹（确定性）
  • 不同故障生成不同指纹（抗碰撞）
  • 日志格式的微小变化不会改变指纹（稳定性）
  • 指纹足够短，可用于Jira标签和GitHub标记
示例：
锚点:
  exception_type: "TypeError"
  message_template: "Cannot read property 'vendorId' of undefined"
  top_frames: "processOrder|groupByVendor|checkout"
  test_name: "checkout.spec.ts > multi-vendor checkout"

规范化字符串: "TypeError|Cannot read property 'vendorId' of undefined|processOrder|groupByVendor|checkout|checkout.spec.ts > multi-vendor checkout"
指纹: a3f8b2c1e9d04567

Step 4: Cluster Near-Duplicates

步骤4：聚类近似重复项

Exact fingerprint matching catches identical failures. Similarity scoring catches related failures that differ slightly (same root cause, different manifestation).
Similarity dimensions:
DimensionWeightMatch Criteria
Exception type0.30Exact match
Error message0.25Levenshtein distance < 20% of message length
Stack frames0.25Jaccard similarity of top 5 frames > 0.6
Component/file0.10Same directory or module
Test name0.10Same describe block or test file
Clustering threshold: similarity score > 0.75 = likely duplicate, suggest merge.
Human review required for:
  • Scores between 0.60 and 0.75 (ambiguous)
  • First occurrence of a new fingerprint (no history to compare)
  • Failures in components with known intermittent issues
精确指纹匹配可识别完全相同的故障。相似度评分可识别略有差异但相关的故障（相同根因，不同表现）。
相似度维度：
维度权重匹配标准
异常类型0.30完全匹配
错误消息0.25Levenshtein距离 < 消息长度的20%
栈帧0.25前5个栈帧的Jaccard相似度 > 0.6
组件/文件0.10同一目录或模块
测试名称0.10同一describe块或测试文件
聚类阈值： 相似度评分 > 0.75 = 可能为重复项，建议合并。
需人工审核的情况：
  • 评分在0.60到0.75之间（模糊情况）
  • 新指纹首次出现（无历史数据可对比）
  • 已知存在间歇性问题的组件中的故障

Step 5: LLM Classify

步骤5：LLM分类

After deterministic fingerprinting and clustering, use the LLM to classify the failure. The prompt feeds in exception, message, top 5 stack frames, test name, and CI context, and asks for five fields:
  1. Failure category — 
    test bug | application bug | environment issue | flaky test | build failure
  2. Severity — 
    critical | major | minor | trivial
    (see the severity matrix below)
  3. Component — inferred from stack trace and file paths
  4. Suspected root cause — 1-2 sentence hypothesis
  5. Confidence — 
    high | medium | low
    ; when low, the LLM states what extra information would resolve it
Route low-confidence classifications to human review rather than auto-acting. See 
references/pipeline-prompts-and-integration.md
for the full prompt text and 
references/classification-taxonomy.md
for the bug-category, severity, and component-mapping definitions the prompt should reference.
Failure categories (see references/ci-failure-analysis.md for detail):
CategoryDescriptionTypical Action
Application bugThe app is brokenFile bug ticket, assign to owning team
Test bugThe test is wrongFix the test, no app change needed
Environment issueCI infra / network / service downRetry, notify infra team
Flaky testIntermittent, non-deterministicQuarantine, investigate root cause
Build failureCompilation, dependency, configFix build, usually blocking
完成确定性指纹识别和聚类后，使用LLM对故障进行分类。提示词会输入异常信息、消息、前5个栈帧、测试名称和CI上下文，并要求返回五个字段：
  1. 故障类别 — 
    test bug | application bug | environment issue | flaky test | build failure
  2. 严重度 — 
    critical | major | minor | trivial
    （见下方严重度矩阵）
  3. 组件 — 从栈跟踪和文件路径推断
  4. 疑似根因 — 1-2句假设
  5. 置信度 — 
    high | medium | low
    ；置信度低时，LLM需说明需要哪些额外信息来解决问题
将低置信度分类路由至人工审核，而非自动处理。完整提示词请查看
references/pipeline-prompts-and-integration.md
，提示词应参考的bug类别、严重度和组件映射定义请查看
references/classification-taxonomy.md
。
故障类别（详情请查看references/ci-failure-analysis.md）：
类别描述典型操作
应用bug应用程序损坏创建bug工单，分配给负责团队
测试bug测试用例错误修复测试用例，无需修改应用
环境问题CI基础设施/网络/服务故障重试，通知基础设施团队
不稳定测试间歇性、非确定性故障隔离测试，调查根因
构建失败编译、依赖、配置问题修复构建，通常会阻塞流程

Step 6: LLM Generate Ticket

步骤6：LLM生成工单

Once classified, use the LLM to generate a human-quality bug ticket. The prompt takes the classification plus the normalized error, a log excerpt, and related cluster fingerprints, and produces:
  • Title — concise, searchable, includes component name (under 80 chars)
  • Description — what happened, in plain language (never raw logs)
  • Steps to reproduce — derived from the test name and log context
  • Evidence — relevant log lines, assertion diffs, screenshots if available
  • Suggested labels — 
    [component, severity, failure-category, fingerprint]
  • Suggested assignee — based on component ownership, if known
The 
fingerprint
belongs on the ticket (label and Fingerprint field) so future dedup can match. See 
references/pipeline-prompts-and-integration.md
for the full prompt and the bug report template.
分类完成后，使用LLM生成符合人类阅读习惯的bug工单。提示词会输入分类结果、标准化错误信息、日志片段和相关聚类指纹，生成以下内容：
  • 标题 — 简洁、可搜索，包含组件名称（不超过80字符）
  • 描述 — 用通俗易懂的语言说明故障情况（绝不使用原始日志）
  • 复现步骤 — 从测试名称和日志上下文推导
  • 证据 — 相关日志行、断言差异、可用截图
  • 建议标签 — 
    [component, severity, failure-category, fingerprint]
  • 建议负责人 — 基于组件负责人信息（若已知）
工单中需包含
fingerprint
（标签和指纹字段），以便后续去重匹配。完整提示词和bug报告模板请查看
references/pipeline-prompts-and-integration.md
。

Step 7: Human Approval

步骤7：人工审批

No automated action without review. The pipeline suggests; humans decide.
Approval decisions:
  • Create ticket — New failure, clear root cause, assign to team
  • Merge into existing — Duplicate of known issue, add evidence to existing ticket
  • Quarantine test — Flaky test, not an app bug, quarantine and schedule investigation
  • Retry and monitor — Environment issue, retry CI, alert if persists
  • Dismiss — Known issue already fixed in pending deploy, or test bug with obvious fix
无人工审核不得执行自动化操作。 工作流仅提供建议，最终由人工决策。
审批决策：
  • 创建工单 — 新故障，根因明确，分配给对应团队
  • 合并至现有工单 — 已知问题的重复项，将证据添加至现有工单
  • 隔离测试 — 不稳定测试，非应用bug，隔离并安排调查
  • 重试并监控 — 环境问题，重试CI，若持续则发出警报
  • 驳回 — 已知问题已在待部署版本中修复，或测试bug有明显修复方案

Severity/Priority Matrix

严重度/优先级矩阵

Severity measures impact. Priority measures urgency. They are independent dimensions.
严重度衡量影响范围，优先级衡量紧急程度，二者为独立维度。

Severity Definitions

严重度定义

SeverityDefinitionExamples
CriticalSystem unusable, data loss, security breach, no workaroundPayment processing fails, user data exposed, app crashes on launch
MajorCore feature broken, degraded experience, workaround existsSearch returns wrong results, checkout requires page reload, form data lost on back-button
MinorNon-core feature affected, cosmetic with functional impactSorting does not persist, tooltip clipped on mobile, secondary action fails
TrivialCosmetic only, no functional impactTypo in label, 1px alignment, inconsistent capitalization
严重度定义示例
Critical（致命）系统不可用、数据丢失、安全漏洞、无替代方案支付处理失败、用户数据泄露、应用启动崩溃
Major（严重）核心功能损坏、体验下降、有替代方案搜索返回错误结果、结账需刷新页面、返回按钮导致表单数据丢失
Minor（次要）非核心功能受影响、存在功能性影响的外观问题排序不持久、移动端提示框被截断、次要操作失败
Trivial（轻微）仅外观问题，无功能性影响标签拼写错误、1px对齐问题、大小写不一致

Priority Definitions

优先级定义

PriorityDefinitionSLA (example)
P0Fix immediately, blocks release or productionSame day
P1Fix this sprint, significant user impactThis sprint
P2Fix next sprint, moderate impactNext sprint
P3Fix when convenient, low impactBacklog
优先级定义SLA（示例）
P0立即修复，阻塞发布或生产当日
P1本迭代修复，用户影响显著本迭代
P2下一迭代修复，影响中等下一迭代
P3方便时修复，影响较小待办队列

Severity x Priority Decision Guide

严重度×优先级决策指南

CriticalMajorMinorTrivial
Affects all usersP0P0P1P2
Affects segment (>10%)P0P1P2P3
Affects few users (<10%)P1P1P2P3
Edge case onlyP1P2P3P3
致命严重次要轻微
影响所有用户P0P0P1P2
影响部分用户(>10%)P0P1P2P3
影响少数用户(<10%)P1P1P2P3
仅边缘场景P1P2P3P3

Bug Report Template

Bug报告模板

Use the same template for any bug report, whether auto-generated or human-written. It carries the defect heading, severity/priority/component/environment/fingerprint/reporter metadata, then Description, Steps to Reproduce, Expected/Actual Behavior, Evidence, Frequency, Suggested Root Cause, and Related Issues. See 
references/pipeline-prompts-and-integration.md
for the full copy-paste Markdown template.
所有bug报告（自动生成或人工撰写）均使用同一模板。包含缺陷标题、严重度/优先级/组件/环境/指纹/报告人元数据，以及描述、复现步骤、预期/实际行为、证据、出现频率、建议根因和相关问题。完整可复制的Markdown模板请查看
references/pipeline-prompts-and-integration.md
。

Deduplication Patterns

去重模式

PatternDetectionAction
Exact duplicateSame fingerprintMerge into existing ticket, add evidence
Near-duplicateSame cluster (similarity > 0.75)Link tickets, suggest merge for human review
Same root cause, different symptomSame exception type + overlapping frames in different testsCreate parent ticket linking symptom tickets
Regression of fixed bugFingerprint matches closed ticketReopen ticket, flag as regression, increase priority
Flaky recurrenceSame fingerprint intermittently across CI runsTag as flaky, quarantine if rate > 10%
模式检测方式操作
完全重复相同指纹合并至现有工单，添加证据
近似重复同一聚类（相似度>0.75）关联工单，建议人工审核后合并
相同根因，不同症状相同异常类型 + 不同测试中栈帧重叠创建父工单关联所有症状工单
已修复bug复发指纹匹配已关闭工单重新打开工单，标记为回归，提高优先级
不稳定测试复发同一指纹在CI运行中间歇性出现标记为不稳定，若出现率>10%则隔离

CI Failure Analysis

CI故障分析

See 
references/ci-failure-analysis.md
for comprehensive patterns. Key decision: consistent failure = test bug or app bug; intermittent failure = flaky test or environment; multiple failures at once = environment or shared component; build failure = code or dependency issue.
全面模式请查看
references/ci-failure-analysis.md
。关键决策：持续故障=测试bug或应用bug；间歇性故障=不稳定测试或环境问题；同时出现多个故障=环境或共享组件问题；构建失败=代码或依赖问题。

Integration Patterns

集成模式

The pipeline output is tracker-agnostic: Step 6 produces title, description, labels, severity, and component that map to any tracker's fields. See 
references/pipeline-prompts-and-integration.md
for the 
gh issue create
/ fingerprint-dedup commands, the GitHub Actions "triage on failure" workflow, and notes on Jira/Linear/Azure DevOps REST/GraphQL integration.
工作流输出与跟踪系统无关：步骤6生成的标题、描述、标签、严重度和组件可映射到任何跟踪系统的字段。GitHub Issues的
gh issue create
/指纹去重命令、GitHub Actions「故障时分流」工作流，以及Jira/Linear/Azure DevOps REST/GraphQL集成说明，请查看
references/pipeline-prompts-and-integration.md
。

Buy vs Build

外购 vs 自研

Before implementing the full pipeline, check whether a hosted platform already covers the work you'd be doing. Several tools now ship AI-driven test triage that overlaps directly with Steps 4-6.
PlatformCoversNotes
Trunk Flaky TestsFingerprinting, clustering, severity routing, native PR comments + webhooksDedicated Agents feature for triage; documented Quarantining workflow — the closest off-the-shelf analog to this skill's pipeline
CloudBees Smart TestsFingerprinting, ML-based prioritization, Test Impact AnalysisFormerly Launchable — agents searching old docs may find the old name
Datadog Test OptimizationFlaky Test Management (Auto Retries, Early Flake Detection, Failed Test Replay), Test Impact AnalysisBits AI Dev Agent now auto-generates fix PRs and Flaky Test Policies auto-quarantine-then-disable after 30 days; pairs with Datadog APM if you're already on Datadog
SealightsQuality intelligence and test-impact gatingEnterprise; strongest in regulated industries
Use the in-skill pipeline when (a) you need on-prem or air-gapped deployment, (b) your tracker integration is exotic, or (c) you want an explicit AI-prompt audit trail for compliance. Otherwise, buying is usually cheaper than rebuilding fingerprinting + clustering.
在实施完整工作流之前，请检查是否已有托管平台可覆盖您要做的工作。目前有多个工具提供AI驱动的测试分流功能，与步骤4-6直接重叠。
平台覆盖范围说明
Trunk Flaky Tests指纹识别、聚类、严重度路由、原生PR评论+Webhooks专用于分流的Agents功能；有文档记录的隔离工作流——与本技能工作流最接近的现成工具
CloudBees Smart Tests指纹识别、基于ML的优先级排序、测试影响分析前身为Launchable——查找旧文档的Agent可能会看到旧名称
Datadog Test Optimization不稳定测试管理（自动重试、早期不稳定检测、失败测试重放）、测试影响分析Bits AI Dev Agent现在可自动生成修复PR，不稳定测试策略会在30天后自动隔离并禁用；若已使用Datadog，可与Datadog APM搭配使用
Sealights质量智能和测试影响门禁企业级；在受监管行业表现突出
以下场景建议使用本技能工作流：(a) 需要本地部署或离线部署；(b) 跟踪系统集成需求特殊；(c) 因合规要求需要明确的AI提示审计轨迹。否则，外购通常比重新构建指纹识别+聚类更划算。

Model selection cost note

模型选择成本说明

Use Sonnet 4.6 / Haiku 4.5 for classification (Step 5) — it's cheap and capable enough. Escalate to Opus 4.8 only when the cluster is novel, the failure is ambiguous, or the suggested root cause has low confidence. Burning Opus on every triage is wasteful.
分类（步骤5）使用Sonnet 4.6 / Haiku 4.5即可——成本低且足够胜任。仅在聚类为新类型、故障模糊或建议根因置信度低时，才升级到Opus 4.8。每次分流都使用Opus会造成浪费。

Anti-Patterns

反模式

1. Using LLM for Deduplication

1. 使用LLM进行去重

LLMs are non-deterministic. The same two errors compared twice may get different similarity scores. Use deterministic fingerprinting for deduplication; use LLM only for explaining and classifying.
LLM具有非确定性。同一两个错误对比两次可能得到不同的相似度评分。使用确定性指纹识别进行去重；仅使用LLM进行解释和分类。

2. Auto-Closing Without Review

2. 无审核自动关闭工单

Automatically closing a ticket as "duplicate" based on fingerprint matching can merge distinct issues. Always require human confirmation for close/merge actions.
仅基于指纹匹配自动关闭工单为「重复项」可能会合并不同问题。关闭/合并操作始终需要人工确认。

3. Over-Classifying Severity

3. 过度分类严重度

If everything is "critical," nothing is. Follow the severity matrix strictly. A cosmetic typo is trivial even if it annoys someone.
如果所有问题都被标记为「致命」，则没有真正的优先级。严格遵循严重度矩阵。即使有人不满，外观拼写错误也属于轻微问题。

4. Ignoring Environment Failures

4. 忽略环境故障

Labeling all failures as "app bug" when many are CI infrastructure issues (Docker OOM, network timeout, disk full). Classify environment issues separately -- they need different remediation.
将所有故障标记为「应用bug」，而实际上很多是CI基础设施问题（Docker内存不足、网络超时、磁盘已满）。单独分类环境问题——它们需要不同的修复措施。

5. No Feedback Loop

5. 无反馈循环

Building the pipeline once and never measuring accuracy. Track: auto-classification accuracy, false duplicate rate, ticket quality ratings from developers.
构建工作流后不再衡量准确性。跟踪：自动分类准确率、错误去重率、开发人员对工单质量的评分。

6. Raw Logs in Tickets

6. 工单中使用原始日志

Pasting 500 lines of raw CI output into a bug ticket. Normalize, extract relevant lines, and present the 5-10 lines that matter.
将500行原始CI输出粘贴到bug工单中。应标准化日志，提取相关行，仅展示关键的5-10行。

7. Fingerprinting Without Normalization

7. 未经标准化直接指纹识别

Hashing raw log lines produces unstable fingerprints that change every run — the same failure gets a different fingerprint each time, so dedup never fires. Normalization is mandatory: always normalize before fingerprinting (Step 1), never fingerprint raw logs.
对原始日志行进行哈希会生成不稳定的指纹，每次运行都会变化——相同故障每次生成不同的指纹，导致去重永远无法生效。标准化是必需的：指纹识别前必须先标准化（步骤1），绝不对原始日志进行指纹识别。

8. No Component Ownership Mapping

8. 无组件负责人映射

Classification without routing is useless. Maintain a component-to-team mapping so that classified bugs reach the right people.
仅分类不路由毫无意义。维护组件到团队的映射表，确保分类后的bug能送达正确的负责人。

Verification

验证

The fingerprinter is the load-bearing piece — prove it before trusting any dedup. Run the 5 stability assertions from 
references/ci-failure-analysis.md
against your implementation:
  1. Same error, different timestamps → same fingerprint.
  2. Same error, different PIDs and ports → same fingerprint.
  3. Same error, different line numbers (code edited) → same fingerprint.
  4. Different errors (e.g. 
    TypeError
    vs 
    RangeError
    ) → different fingerprints.
  5. Same exception type, different message property (
    'name'
    vs 
    'email'
    ) → different fingerprints.
Tests 1-3 must collapse to one hash; tests 4-5 must produce distinct hashes. A failure here means normalization (Step 1) is leaking noise into the hash or stripping a stable anchor — fix that before tuning clustering. The clustering weights (0.30 / 0.25 / 0.25 / 0.10 / 0.10) must sum to 1.00.
指纹识别是核心组件——在信任任何去重功能之前，请先验证。针对您的实现运行
references/ci-failure-analysis.md
中的5项稳定性断言：
  1. 相同错误，不同时间戳 → 相同指纹。
  2. 相同错误，不同PID和端口 → 相同指纹。
  3. 相同错误，不同行号（代码已编辑） → 相同指纹。
  4. 不同错误（如
    TypeError
    vs 
    RangeError
    ） → 不同指纹。
  5. 相同异常类型，不同消息属性（
    'name'
    vs 
    'email'
    ） → 不同指纹。
测试1-3必须生成同一个哈希；测试4-5必须生成不同的哈希。若测试失败，说明标准化（步骤1）将干扰信息带入了哈希，或去除了稳定锚点——在调整聚类之前需修复此问题。聚类权重（0.30 / 0.25 / 0.25 / 0.10 / 0.10）总和必须为1.00。

Done When

完成标准

  • Every failure in 
    failures.json
    has non-null 
    severity
    , 
    component
    , and 
    category
    fields.
  • The 5 fingerprint stability assertions above pass (tests 1-3 same hash, tests 4-5 distinct).
  • Duplicates are merged or linked, each pointing to the canonical ticket's fingerprint label.
  • Every P0/P1 ticket has an assignee set.
  • Auto-classification accuracy is measured and recorded (target > 85%; see 
    qa-metrics
    ).
  • failures.json
    中的每个故障都有非空的
    severity
    、
    component
    和
    category
    字段。
  • 上述5项指纹稳定性断言全部通过（测试1-3哈希相同，测试4-5哈希不同）。
  • 重复项已合并或关联，均指向标准工单的指纹标签。
  • 每个P0/P1工单都已设置负责人。
  • 自动分类准确率已测量并记录（目标>85%；请查看
    qa-metrics
    ）。

Related Skills

相关技能

  • qa-metrics
     — Track triage accuracy, duplicate rates, mean time to classification, and defect escape rates.
  • ci-cd-integration
     — Pipeline configuration for running triage on test failures, parallel execution, and reporting.
  • test-reliability
     — Runtime per-test healing and quarantine for a single flaky locator. Triage classifies failures; test-reliability fixes one test live.
  • observability-driven-testing
     — Goes the other direction: turns production telemetry into new test designs. Use it when prod errors should spawn tests, not tickets.
  • qa-project-context
     — Project context that improves classification accuracy: component map, known issues, ownership.
  • ai-test-generation
     — Generate regression tests from triaged bug reports.
  • qa-metrics
     — 跟踪分流准确率、重复率、平均分类时间和缺陷逃逸率。
  • ci-cd-integration
     — 测试故障时分流、并行执行和报告的流水线配置。
  • test-reliability
     — 单个不稳定定位器的运行时测试修复和隔离。分流负责分类故障；test-reliability负责实时修复单个测试。
  • observability-driven-testing
     — 反向流程：将生产遥测数据转化为新测试设计。当生产错误应生成测试而非工单时使用。
  • qa-project-context
     — 提升分类准确性的项目上下文：组件映射、已知问题、负责人信息。
  • ai-test-generation
     — 从已分流的bug报告生成回归测试。

Reference Files (in 
references/
)

参考文件（位于
references/
）

  • pipeline-prompts-and-integration.md — LLM classification + ticket-generation prompts (Steps 5-6), the full bug report Markdown template, and tracker integration code (GitHub Issues, CI workflow, Jira/Linear notes).
  • classification-taxonomy.md — Bug categories, severity definitions, component mapping rules, and root cause categories.
  • ci-failure-analysis.md — CI log parsing patterns, failure category decision tree, fingerprinting algorithm detail.
  • pipeline-prompts-and-integration.md — LLM分类+工单生成提示词（步骤5-6）、完整bug报告Markdown模板、跟踪系统集成代码（GitHub Issues、CI工作流、Jira/Linear说明）。
  • classification-taxonomy.md — bug类别、严重度定义、组件映射规则和根因类别。
  • ci-failure-analysis.md — CI日志解析模式、故障类别决策树、指纹识别算法详情。