Loading...
Loading...
Compare original and translation side by side
./testing-handbook../testing-handbook~/testing-handbookhttps://github.com/trailofbits/testing-handbook./testing-handbook../testing-handbook~/testing-handbookhttps://github.com/trailofbits/testing-handbookPhase 0: Setup Phase 1: Discovery
┌─────────────────┐ ┌─────────────────┐
│ Locate handbook │ → │ Analyze handbook│
│ - Find or clone │ │ - Scan sections │
│ - Confirm path │ │ - Classify types│
└─────────────────┘ └─────────────────┘
↓ ↓
Phase 3: Generation Phase 2: Planning
┌─────────────────┐ ┌─────────────────┐
│ TWO-PASS GEN │ ← │ Generate plan │
│ Pass 1: Content │ │ - New skills │
│ Pass 2: X-refs │ │ - Updates │
│ - Write to gen/ │ │ - Present user │
└─────────────────┘ └─────────────────┘
↓
Phase 4: Testing Phase 5: Finalize
┌─────────────────┐ ┌─────────────────┐
│ Validate skills │ → │ Post-generation │
│ - Run validator │ │ - Update README │
│ - Test activation│ │ - Update X-refs │
│ - Fix issues │ │ - Self-improve │
└─────────────────┘ └─────────────────┘Phase 0: 准备阶段 Phase 1: 发现阶段
┌─────────────────┐ ┌─────────────────┐
│ 定位手册位置 │ → │ 分析手册内容 │
│ - 查找或克隆 │ │ - 扫描章节 │
│ - 确认路径 │ │ - 分类技能类型 │
└─────────────────┘ └─────────────────┘
↓ ↓
Phase 3: 生成阶段 Phase 2: 规划阶段
┌─────────────────┐ ┌─────────────────┐
│ 双阶段生成流程 │ ← │ 生成执行计划 │
│ 第一阶段:内容生成 │ │ - 新技能开发 │
│ 第二阶段:交叉引用 │ │ - 技能更新 │
│ - 写入gen/目录 │ │ - 提交用户确认 │
└─────────────────┘ └─────────────────┘
↓
Phase 4: 测试阶段 Phase 5: 收尾阶段
┌─────────────────┐ ┌─────────────────┐
│ 验证技能有效性 │ → │ 生成后处理 │
│ - 运行验证脚本 │ │ - 更新README文件 │
│ - 测试技能激活 │ │ - 更新交叉引用 │
│ - 修复问题 │ │ - 自我优化改进 │
└─────────────────┘ └─────────────────┘plugins/testing-handbook-skills/skills/[skill-name]/*plugins/testing-handbook-skills/skills/testing-handbook-generator/*README.mdplugins/property-based-testing/plugins/static-analysis/testing-handbook-skills/plugins/testing-handbook-skills/skills/[skill-name]/*plugins/testing-handbook-skills/skills/testing-handbook-generator/*README.mdplugins/property-based-testing/plugins/static-analysis/testing-handbook-skills/| Handbook Section | Skill Type | Template |
|---|---|---|
| Tool Skill | tool-skill.md |
| Fuzzer Skill | fuzzer-skill.md |
| Technique Skill | technique-skill.md |
| Domain Skill | domain-skill.md |
| Tool Skill | tool-skill.md |
| 手册章节 | 技能类型 | 模板文件 |
|---|---|---|
| 工具类技能 | tool-skill.md |
| 模糊测试工具技能 | fuzzer-skill.md |
| 方法类技能 | technique-skill.md |
| 领域类技能 | domain-skill.md |
| 工具类技能 | tool-skill.md |
| Signal | Indicates |
|---|---|
| Major tool/topic |
| Numbered files (00-, 10-, 20-) | Structured content |
| Methodology content |
| Has external links |
| 信号特征 | 代表含义 |
|---|---|
包含 | 主要工具/主题 |
| 带编号的文件(00-、10-、20-) | 结构化内容 |
| 方法论相关内容 |
| 包含外部链接 |
| Signal | Action |
|---|---|
| Skip section |
| Empty directory | Skip section |
| Template/placeholder file | Skip section |
GUI-only tool (e.g., | Skip section (Claude cannot operate GUI tools) |
| 信号特征 | 处理动作 |
|---|---|
前置元数据中包含 | 跳过该章节 |
| 空目录 | 跳过该章节 |
| 模板/占位文件 | 跳过该章节 |
纯GUI工具(如 | 跳过该章节(Claude无法操作GUI工具) |
├─ Need to analyze handbook and build plan?
│ └─ Read: discovery.md
│ (Handbook analysis methodology, plan format)
│
├─ Spawning skill generation agents?
│ └─ Read: agent-prompt.md
│ (Full prompt template, variable reference, validation checklist)
│
├─ Generating a specific skill type?
│ └─ Read appropriate template:
│ ├─ Tool (Semgrep, CodeQL) → templates/tool-skill.md
│ ├─ Fuzzer (libFuzzer, AFL++) → templates/fuzzer-skill.md
│ ├─ Technique (harness, coverage) → templates/technique-skill.md
│ └─ Domain (crypto, web) → templates/domain-skill.md
│
├─ Validating generated skills?
│ └─ Run: scripts/validate-skills.py
│ Then read: testing.md for activation testing
│
├─ Finalizing after generation?
│ └─ See: Post-Generation Tasks below
│ (Update main README, update Skills Cross-Reference, self-improvement)
│
└─ Quick generation from specific section?
└─ Use Quick Reference above, apply template directly├─ 是否需要分析手册并制定计划?
│ └─ 阅读:discovery.md
│ (手册分析方法论、计划格式)
│
├─ 是否需要启动技能生成Agent?
│ └─ 阅读:agent-prompt.md
│ (完整提示模板、变量参考、验证检查清单)
│
├─ 是否生成特定类型的技能?
│ └─ 阅读对应模板:
│ ├─ 工具类(Semgrep、CodeQL)→ templates/tool-skill.md
│ ├─ 模糊测试工具类(libFuzzer、AFL++)→ templates/fuzzer-skill.md
│ ├─ 方法类(测试 harness、覆盖率分析)→ templates/technique-skill.md
│ └─ 领域类(加密、Web安全)→ templates/domain-skill.md
│
├─ 是否验证生成的技能?
│ └─ 运行:scripts/validate-skills.py
│ 然后阅读:testing.md了解激活测试方法
│
├─ 生成完成后是否需要收尾?
│ └─ 查看:下方的生成后处理任务
│ (更新主README、更新技能交叉引用、自我优化)
│
└─ 是否基于特定章节快速生成技能?
└─ 使用上方快速参考,直接应用对应模板Pass 1 - Generating 5 skills in parallel:
├─ Agent 1: libfuzzer (fuzzer) → skills/libfuzzer/SKILL.md
├─ Agent 2: aflpp (fuzzer) → skills/aflpp/SKILL.md
├─ Agent 3: semgrep (tool) → skills/semgrep/SKILL.md
├─ Agent 4: harness-writing (technique) → skills/harness-writing/SKILL.md
└─ Agent 5: wycheproof (domain) → skills/wycheproof/SKILL.md
Each agent uses: pass=1 (content only, Related Skills left empty)## Related Skills\n\n<!-- PASS2: populate after all skills exist -->references: DEFERRED第一阶段 - 并行生成5个技能:
├─ Agent 1: libfuzzer(模糊测试工具)→ skills/libfuzzer/SKILL.md
├─ Agent 2: aflpp(模糊测试工具)→ skills/aflpp/SKILL.md
├─ Agent 3: semgrep(工具)→ skills/semgrep/SKILL.md
├─ Agent 4: harness-writing(方法)→ skills/harness-writing/SKILL.md
└─ Agent 5: wycheproof(领域)→ skills/wycheproof/SKILL.md
每个Agent使用参数: pass=1(仅生成内容,相关技能章节留空)## Related Skills\n\n<!-- PASS2: populate after all skills exist -->references: DEFERREDPass 2 - Populating cross-references:
├─ Read all generated skill names from skills/*/SKILL.md
├─ For each skill, determine related skills based on:
│ ├─ related_sections from discovery (handbook structure)
│ ├─ Skill type relationships (fuzzers → techniques)
│ └─ Explicit mentions in content
└─ Update each SKILL.md's Related Skills sectionls -d skills/*/SKILL.md第二阶段 - 填充交叉引用:
├─ 从skills/*/SKILL.md收集所有已生成的技能名称
├─ 为每个技能确定相关技能,依据:
│ ├─ 发现阶段的related_sections(手册结构)
│ ├─ 技能类型关系(模糊测试工具 → 方法)
│ └─ 内容中的明确提及
└─ 更新每个SKILL.md的“相关技能”章节ls -d skills/*/SKILL.mdpasspass| Failure Type | Detection | Recovery Action |
|---|---|---|
| Agent crashed | No output report | Re-run single agent with same inputs |
| Validation failed | Output report shows errors | Check gaps/warnings, manually patch or re-run |
| Wrong skill type | Content doesn't match template | Re-run with corrected |
| Missing content | Output report lists gaps | Accept if minor, or provide additional |
| Pass 2 broken ref | Validator shows missing skill | Check if skill was skipped, update reference |
| 故障类型 | 检测方式 | 恢复动作 |
|---|---|---|
| Agent崩溃 | 无输出报告 | 使用相同输入重新运行单个Agent |
| 验证失败 | 输出报告显示错误 | 检查缺口/警告,手动修复或重新运行 |
| 技能类型错误 | 内容与模板不匹配 | 使用修正后的 |
| 内容缺失 | 输出报告列出缺口 | 若为小缺口则接受,或提供额外 |
| 第二阶段出现断链 | 验证脚本显示缺失技能 | 检查技能是否被跳过,更新引用 |
undefinedundefined
**Regeneration workflow:**
1. Re-read the handbook section for fresh content
2. Apply the appropriate template
3. Write to `skills/{skill-name}/SKILL.md` (overwrites existing)
4. Re-run Pass 2 for that skill only to update cross-references
5. Run validator on the single skill: `uv run scripts/validate-skills.py --skill {skill-name}`
**重新生成工作流:**
1. 重新读取手册章节获取最新内容
2. 应用对应模板
3. 写入`skills/{skill-name}/SKILL.md`(覆盖现有文件)
4. 仅对该技能重新运行第二阶段以更新交叉引用
5. 对单个技能运行验证:`uv run scripts/validate-skills.py --skill {skill-name}`skills/[skill-name]/SKILL.mdskills/[skill-name]/SKILL.mduv run scripts/validate-skills.pyREADME.mdREADME.mduv run scripts/validate-skills.pyREADME.mdREADME.mdREADME.mdtesting-handbook-skillstesting-handbook-generator| Plugin | Description | Author |
|--------|-------------|--------|
| ... other plugins ... |
| [testing-handbook-skills](plugins/testing-handbook-skills/) | Meta-skill that generates skills from the Testing Handbook | Paweł Płatek |
| [libfuzzer](plugins/testing-handbook-skills/skills/libfuzzer/) | Coverage-guided fuzzing with libFuzzer for C/C++ | testing-handbook-generator |
| [aflpp](plugins/testing-handbook-skills/skills/aflpp/) | Multi-core fuzzing with AFL++ | testing-handbook-generator |
| [semgrep](plugins/testing-handbook-skills/skills/semgrep/) | Fast static analysis for finding bugs | testing-handbook-generator |README.mdtesting-handbook-skillstesting-handbook-generator| 插件 | 描述 | 作者 |
|--------|-------------|--------|
| ... 其他插件 ... |
| [testing-handbook-skills](plugins/testing-handbook-skills/) | 从测试手册生成技能的元技能 | Paweł Płatek |
| [libfuzzer](plugins/testing-handbook-skills/skills/libfuzzer/) | 用于C/C++的覆盖率引导模糊测试工具libFuzzer | testing-handbook-generator |
| [aflpp](plugins/testing-handbook-skills/skills/aflpp/) | 多核模糊测试工具AFL++ | testing-handbook-generator |
| [semgrep](plugins/testing-handbook-skills/skills/semgrep/) | 快速静态分析工具,用于发现漏洞 | testing-handbook-generator |README.mdSKILL.md## Related Skills-->-.->| Relationship | Arrow Style | Example |
|---|---|---|
| Fuzzer → Technique | | |
| Tool → Tool (alternative) | | |
| Fuzzer → Fuzzer (alternative) | | |
| Technique → Technique | | |
validate-skills.pyREADME.mdSKILL.md## Related Skills-->-.->| 关系类型 | 箭头样式 | 示例 |
|---|---|---|
| 模糊测试工具 → 方法 | | |
| 工具 → 工具(替代) | | |
| 模糊测试工具 → 模糊测试工具(替代) | | |
| 方法 → 方法 | | |
validate-skills.pySKILL.mdtemplates/*.mddiscovery.mdtesting.mdIssue: libFuzzer skill missing sanitizer flags table
Fix: Updated templates/fuzzer-skill.md to include ## Compiler Flags sectionSKILL.mdtemplates/*.mddiscovery.mdtesting.md问题:libFuzzer技能缺失 sanitizer 标志表格
修复:更新templates/fuzzer-skill.md以添加## 编译器标志章节User: "Generate skills from the testing handbook"
1. Locate handbook (check common locations, ask user, or clone)
2. Read discovery.md for methodology
3. Scan handbook at {handbook_path}/content/docs/
4. Build candidate list with types
5. Present plan to user
6. On approval, generate each skill using appropriate template
7. Validate generated skills
8. Update main README.md with generated skills table
9. Update README.md Skills Cross-Reference graph from Related Skills sections
10. Self-improve: note any template/discovery issues for future runs
11. Report results用户:"从测试手册生成技能"
1. 定位手册(检查常见位置、询问用户或克隆)
2. 阅读discovery.md了解方法论
3. 扫描手册目录{handbook_path}/content/docs/
4. 构建带类型的候选技能列表
5. 向用户提交计划
6. 获得批准后,使用对应模板生成每个技能
7. 验证生成的技能
8. 更新主README.md的生成技能表格
9. 根据“相关技能”章节更新README.md的技能交叉引用图
10. 自我优化:记录模板/发现环节的问题以便未来改进
11. 报告结果User: "Create a skill for the libFuzzer section"
1. Read /testing-handbook/content/docs/fuzzing/c-cpp/10-libfuzzer/
2. Identify type: Fuzzer Skill
3. Read templates/fuzzer-skill.md
4. Extract content, apply template
5. Write to skills/libfuzzer/SKILL.md
6. Validate and report用户:"为libFuzzer章节创建技能"
1. 读取 /testing-handbook/content/docs/fuzzing/c-cpp/10-libfuzzer/
2. 识别类型:模糊测试工具技能
3. 阅读templates/fuzzer-skill.md
4. 提取内容,应用模板
5. 写入skills/libfuzzer/SKILL.md
6. 验证并报告