testing-handbook-generator
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseTesting Handbook Skill Generator
测试手册技能生成器
Generate and maintain Claude Code skills from the Trail of Bits Testing Handbook.
基于Trail of Bits测试手册生成并维护Claude Code技能。
When to Use
使用场景
Invoke this skill when:
- Creating new security testing skills from handbook content
- User mentions "testing handbook", "appsec.guide", or asks about generating skills
- Bulk skill generation or refresh is needed
Do NOT use for:
- General security testing questions (use the generated skills)
- Non-handbook skill creation
在以下场景调用本技能:
- 基于手册内容创建新的安全测试技能时
- 用户提及“测试手册”“appsec.guide”或询问技能生成相关问题时
- 需要批量生成或更新技能时
请勿在以下场景使用:
- 通用安全测试问题(请使用已生成的技能)
- 非基于手册的技能创建
Handbook Location
手册位置
The skill needs the Testing Handbook repository. See discovery.md for full details.
Quick reference: Check , , → ask user → clone as last resort.
./testing-handbook../testing-handbook~/testing-handbookRepository:
https://github.com/trailofbits/testing-handbook本技能需要Testing Handbook代码仓库。详情请查看discovery.md。
快速参考: 检查、、 → 询问用户 → 最后再考虑克隆仓库。
./testing-handbook../testing-handbook~/testing-handbook代码仓库:
https://github.com/trailofbits/testing-handbookWorkflow Overview
工作流概述
Phase 0: Setup Phase 1: Discovery
┌─────────────────┐ ┌─────────────────┐
│ Locate handbook │ → │ Analyze handbook│
│ - Find or clone │ │ - Scan sections │
│ - Confirm path │ │ - Classify types│
└─────────────────┘ └─────────────────┘
↓ ↓
Phase 3: Generation Phase 2: Planning
┌─────────────────┐ ┌─────────────────┐
│ TWO-PASS GEN │ ← │ Generate plan │
│ Pass 1: Content │ │ - New skills │
│ Pass 2: X-refs │ │ - Updates │
│ - Write to gen/ │ │ - Present user │
└─────────────────┘ └─────────────────┘
↓
Phase 4: Testing Phase 5: Finalize
┌─────────────────┐ ┌─────────────────┐
│ Validate skills │ → │ Post-generation │
│ - Run validator │ │ - Update README │
│ - Test activation│ │ - Update X-refs │
│ - Fix issues │ │ - Self-improve │
└─────────────────┘ └─────────────────┘Phase 0: 准备阶段 Phase 1: 发现阶段
┌─────────────────┐ ┌─────────────────┐
│ 定位手册位置 │ → │ 分析手册内容 │
│ - 查找或克隆 │ │ - 扫描章节 │
│ - 确认路径 │ │ - 分类技能类型 │
└─────────────────┘ └─────────────────┘
↓ ↓
Phase 3: 生成阶段 Phase 2: 规划阶段
┌─────────────────┐ ┌─────────────────┐
│ 双阶段生成流程 │ ← │ 生成执行计划 │
│ 第一阶段:内容生成 │ │ - 新技能开发 │
│ 第二阶段:交叉引用 │ │ - 技能更新 │
│ - 写入gen/目录 │ │ - 提交用户确认 │
└─────────────────┘ └─────────────────┘
↓
Phase 4: 测试阶段 Phase 5: 收尾阶段
┌─────────────────┐ ┌─────────────────┐
│ 验证技能有效性 │ → │ 生成后处理 │
│ - 运行验证脚本 │ │ - 更新README文件 │
│ - 测试技能激活 │ │ - 更新交叉引用 │
│ - 修复问题 │ │ - 自我优化改进 │
└─────────────────┘ └─────────────────┘Scope Restrictions
范围限制
ONLY modify these locations:
- - Generated skills (as siblings to testing-handbook-generator)
plugins/testing-handbook-skills/skills/[skill-name]/* - - Self-improvement
plugins/testing-handbook-skills/skills/testing-handbook-generator/* - Repository root - Add generated skills to table
README.md
NEVER modify or analyze:
- Other plugins (,
plugins/property-based-testing/, etc.)plugins/static-analysis/ - Other skills outside this plugin
Do not scan or pull into context any skills outside of . Generate skills based solely on handbook content and resources referenced from it.
testing-handbook-skills/仅可修改以下位置:
- - 生成的技能文件(与testing-handbook-generator同级)
plugins/testing-handbook-skills/skills/[skill-name]/* - - 自我优化相关文件
plugins/testing-handbook-skills/skills/testing-handbook-generator/* - 仓库根目录- 将生成的技能添加到表格中
README.md
绝对禁止修改或分析:
- 其他插件(、
plugins/property-based-testing/等)plugins/static-analysis/ - 本插件以外的其他技能
请勿扫描或引入以外的任何技能到上下文。仅基于手册内容及其中引用的资源生成技能。
testing-handbook-skills/Quick Reference
快速参考
Section → Skill Type Mapping
章节 → 技能类型映射
| Handbook Section | Skill Type | Template |
|---|---|---|
| Tool Skill | tool-skill.md |
| Fuzzer Skill | fuzzer-skill.md |
| Technique Skill | technique-skill.md |
| Domain Skill | domain-skill.md |
| Tool Skill | tool-skill.md |
| 手册章节 | 技能类型 | 模板文件 |
|---|---|---|
| 工具类技能 | tool-skill.md |
| 模糊测试工具技能 | fuzzer-skill.md |
| 方法类技能 | technique-skill.md |
| 领域类技能 | domain-skill.md |
| 工具类技能 | tool-skill.md |
Skill Candidate Signals
技能候选信号
| Signal | Indicates |
|---|---|
| Major tool/topic |
| Numbered files (00-, 10-, 20-) | Structured content |
| Methodology content |
| Has external links |
| 信号特征 | 代表含义 |
|---|---|
包含 | 主要工具/主题 |
| 带编号的文件(00-、10-、20-) | 结构化内容 |
| 方法论相关内容 |
| 包含外部链接 |
Exclusion Signals
排除信号
| Signal | Action |
|---|---|
| Skip section |
| Empty directory | Skip section |
| Template/placeholder file | Skip section |
GUI-only tool (e.g., | Skip section (Claude cannot operate GUI tools) |
| 信号特征 | 处理动作 |
|---|---|
前置元数据中包含 | 跳过该章节 |
| 空目录 | 跳过该章节 |
| 模板/占位文件 | 跳过该章节 |
纯GUI工具(如 | 跳过该章节(Claude无法操作GUI工具) |
Decision Tree
决策树
Starting skill generation?
├─ Need to analyze handbook and build plan?
│ └─ Read: discovery.md
│ (Handbook analysis methodology, plan format)
│
├─ Spawning skill generation agents?
│ └─ Read: agent-prompt.md
│ (Full prompt template, variable reference, validation checklist)
│
├─ Generating a specific skill type?
│ └─ Read appropriate template:
│ ├─ Tool (Semgrep, CodeQL) → templates/tool-skill.md
│ ├─ Fuzzer (libFuzzer, AFL++) → templates/fuzzer-skill.md
│ ├─ Technique (harness, coverage) → templates/technique-skill.md
│ └─ Domain (crypto, web) → templates/domain-skill.md
│
├─ Validating generated skills?
│ └─ Run: scripts/validate-skills.py
│ Then read: testing.md for activation testing
│
├─ Finalizing after generation?
│ └─ See: Post-Generation Tasks below
│ (Update main README, update Skills Cross-Reference, self-improvement)
│
└─ Quick generation from specific section?
└─ Use Quick Reference above, apply template directly开始技能生成?
├─ 是否需要分析手册并制定计划?
│ └─ 阅读:discovery.md
│ (手册分析方法论、计划格式)
│
├─ 是否需要启动技能生成Agent?
│ └─ 阅读:agent-prompt.md
│ (完整提示模板、变量参考、验证检查清单)
│
├─ 是否生成特定类型的技能?
│ └─ 阅读对应模板:
│ ├─ 工具类(Semgrep、CodeQL)→ templates/tool-skill.md
│ ├─ 模糊测试工具类(libFuzzer、AFL++)→ templates/fuzzer-skill.md
│ ├─ 方法类(测试 harness、覆盖率分析)→ templates/technique-skill.md
│ └─ 领域类(加密、Web安全)→ templates/domain-skill.md
│
├─ 是否验证生成的技能?
│ └─ 运行:scripts/validate-skills.py
│ 然后阅读:testing.md了解激活测试方法
│
├─ 生成完成后是否需要收尾?
│ └─ 查看:下方的生成后处理任务
│ (更新主README、更新技能交叉引用、自我优化)
│
└─ 是否基于特定章节快速生成技能?
└─ 使用上方快速参考,直接应用对应模板Two-Pass Generation (Phase 3)
双阶段生成流程(第三阶段)
Generation uses a two-pass approach to solve forward reference problems (skills referencing other skills that don't exist yet).
生成流程采用双阶段方式解决前向引用问题(技能引用尚未生成的其他技能)。
Pass 1: Content Generation (Parallel)
第一阶段:内容生成(并行)
Generate all skills in parallel without the Related Skills section:
Pass 1 - Generating 5 skills in parallel:
├─ Agent 1: libfuzzer (fuzzer) → skills/libfuzzer/SKILL.md
├─ Agent 2: aflpp (fuzzer) → skills/aflpp/SKILL.md
├─ Agent 3: semgrep (tool) → skills/semgrep/SKILL.md
├─ Agent 4: harness-writing (technique) → skills/harness-writing/SKILL.md
└─ Agent 5: wycheproof (domain) → skills/wycheproof/SKILL.md
Each agent uses: pass=1 (content only, Related Skills left empty)Pass 1 agents:
- Generate all sections EXCEPT Related Skills
- Leave a placeholder:
## Related Skills\n\n<!-- PASS2: populate after all skills exist --> - Output report includes
references: DEFERRED
并行生成所有技能,但不包含“相关技能”章节:
第一阶段 - 并行生成5个技能:
├─ Agent 1: libfuzzer(模糊测试工具)→ skills/libfuzzer/SKILL.md
├─ Agent 2: aflpp(模糊测试工具)→ skills/aflpp/SKILL.md
├─ Agent 3: semgrep(工具)→ skills/semgrep/SKILL.md
├─ Agent 4: harness-writing(方法)→ skills/harness-writing/SKILL.md
└─ Agent 5: wycheproof(领域)→ skills/wycheproof/SKILL.md
每个Agent使用参数: pass=1(仅生成内容,相关技能章节留空)第一阶段Agent要求:
- 生成所有章节,除了“相关技能”
- 留下占位符:
## Related Skills\n\n<!-- PASS2: populate after all skills exist --> - 输出报告包含
references: DEFERRED
Pass 2: Cross-Reference Population (Sequential)
第二阶段:交叉引用填充(串行)
After all Pass 1 agents complete, run Pass 2 to populate Related Skills:
Pass 2 - Populating cross-references:
├─ Read all generated skill names from skills/*/SKILL.md
├─ For each skill, determine related skills based on:
│ ├─ related_sections from discovery (handbook structure)
│ ├─ Skill type relationships (fuzzers → techniques)
│ └─ Explicit mentions in content
└─ Update each SKILL.md's Related Skills sectionPass 2 process:
- Collect all generated skill names:
ls -d skills/*/SKILL.md - For each skill, identify related skills using the mapping from discovery
- Edit each SKILL.md to replace the placeholder with actual links
- Validate cross-references exist (no broken links)
所有第一阶段Agent完成后,执行第二阶段填充“相关技能”章节:
第二阶段 - 填充交叉引用:
├─ 从skills/*/SKILL.md收集所有已生成的技能名称
├─ 为每个技能确定相关技能,依据:
│ ├─ 发现阶段的related_sections(手册结构)
│ ├─ 技能类型关系(模糊测试工具 → 方法)
│ └─ 内容中的明确提及
└─ 更新每个SKILL.md的“相关技能”章节第二阶段流程:
- 收集所有生成的技能名称:
ls -d skills/*/SKILL.md - 利用发现阶段的映射为每个技能识别相关技能
- 编辑每个SKILL.md,用实际链接替换占位符
- 验证交叉引用是否存在(无断链)
Agent Prompt Template
Agent提示模板
See agent-prompt.md for the full prompt template with:
- Variable substitution reference (including variable)
pass - Pre-write validation checklist
- Hugo shortcode conversion rules
- Line count splitting rules
- Error handling guidance
- Output report format
查看**agent-prompt.md**获取完整提示模板,包括:
- 变量替换参考(含变量)
pass - 预写入验证检查清单
- Hugo短代码转换规则
- 行数拆分规则
- 错误处理指南
- 输出报告格式
Collecting Results
结果收集
After Pass 1: Aggregate output reports, verify all skills generated.
After Pass 2: Run validator to check cross-references.
第一阶段后:汇总输出报告,验证所有技能是否生成。
第二阶段后:运行验证脚本检查交叉引用。
Handling Agent Failures
Agent故障处理
If an agent fails or produces invalid output:
| Failure Type | Detection | Recovery Action |
|---|---|---|
| Agent crashed | No output report | Re-run single agent with same inputs |
| Validation failed | Output report shows errors | Check gaps/warnings, manually patch or re-run |
| Wrong skill type | Content doesn't match template | Re-run with corrected |
| Missing content | Output report lists gaps | Accept if minor, or provide additional |
| Pass 2 broken ref | Validator shows missing skill | Check if skill was skipped, update reference |
Important: Do NOT re-run the entire parallel batch for a single agent failure. Fix individual failures independently.
如果Agent失败或生成无效输出:
| 故障类型 | 检测方式 | 恢复动作 |
|---|---|---|
| Agent崩溃 | 无输出报告 | 使用相同输入重新运行单个Agent |
| 验证失败 | 输出报告显示错误 | 检查缺口/警告,手动修复或重新运行 |
| 技能类型错误 | 内容与模板不匹配 | 使用修正后的 |
| 内容缺失 | 输出报告列出缺口 | 若为小缺口则接受,或提供额外 |
| 第二阶段出现断链 | 验证脚本显示缺失技能 | 检查技能是否被跳过,更新引用 |
重要提示: 单个Agent失败时请勿重新运行整个并行批次。独立修复单个故障。
Single-Skill Regeneration
单技能重新生成
To regenerate a single skill without re-running the entire batch:
undefined无需重新运行整个批次即可重新生成单个技能:
undefinedRegenerate single skill (Pass 1 - content only)
重新生成单个技能(第一阶段 - 仅内容)
"Use testing-handbook-generator to regenerate the {skill-name} skill from section {section_path}"
"使用testing-handbook-generator从章节{section_path}重新生成{skill-name}技能"
Example:
示例:
"Use testing-handbook-generator to regenerate the libfuzzer skill from section fuzzing/c-cpp/10-libfuzzer"
**Regeneration workflow:**
1. Re-read the handbook section for fresh content
2. Apply the appropriate template
3. Write to `skills/{skill-name}/SKILL.md` (overwrites existing)
4. Re-run Pass 2 for that skill only to update cross-references
5. Run validator on the single skill: `uv run scripts/validate-skills.py --skill {skill-name}`"使用testing-handbook-generator从章节fuzzing/c-cpp/10-libfuzzer重新生成libfuzzer技能"
**重新生成工作流:**
1. 重新读取手册章节获取最新内容
2. 应用对应模板
3. 写入`skills/{skill-name}/SKILL.md`(覆盖现有文件)
4. 仅对该技能重新运行第二阶段以更新交叉引用
5. 对单个技能运行验证:`uv run scripts/validate-skills.py --skill {skill-name}`Output Location
输出位置
Generated skills are written to:
skills/[skill-name]/SKILL.mdEach skill gets its own directory for potential supporting files (as siblings to testing-handbook-generator).
生成的技能将写入:
skills/[skill-name]/SKILL.md每个技能拥有独立目录,可存放潜在支持文件(与testing-handbook-generator同级)。
Quality Checklist
质量检查清单
Before delivering generated skills:
- All handbook sections analyzed (Phase 1)
- Plan presented to user before generation (Phase 2)
- Parallel agents launched - one per skill (Phase 3)
- Templates applied correctly per skill type
- Validator passes:
uv run scripts/validate-skills.py - Activation testing passed - see testing.md
- Main updated with generated skills table
README.md - Skills Cross-Reference graph updated
README.md - Self-improvement notes captured
- User notified with summary
交付生成的技能前需完成:
- 所有手册章节已分析(第一阶段)
- 生成前已向用户提交计划并确认(第二阶段)
- 已启动并行Agent - 每个技能对应一个Agent(第三阶段)
- 已根据技能类型正确应用模板
- 验证脚本通过:
uv run scripts/validate-skills.py - 激活测试通过 - 查看testing.md
- 主已更新生成技能表格
README.md - 技能交叉引用图已更新
README.md - 已记录自我优化笔记
- 已向用户发送结果摘要
Post-Generation Tasks
生成后处理任务
1. Update Main README
1. 更新主README文件
After generating skills, update the repository's main to list them.
README.mdFormat: Add generated skills to the same "Available Plugins" table, directly after . Use plain text as the author (no link).
testing-handbook-skillstesting-handbook-generatorExample:
markdown
| Plugin | Description | Author |
|--------|-------------|--------|
| ... other plugins ... |
| [testing-handbook-skills](plugins/testing-handbook-skills/) | Meta-skill that generates skills from the Testing Handbook | Paweł Płatek |
| [libfuzzer](plugins/testing-handbook-skills/skills/libfuzzer/) | Coverage-guided fuzzing with libFuzzer for C/C++ | testing-handbook-generator |
| [aflpp](plugins/testing-handbook-skills/skills/aflpp/) | Multi-core fuzzing with AFL++ | testing-handbook-generator |
| [semgrep](plugins/testing-handbook-skills/skills/semgrep/) | Fast static analysis for finding bugs | testing-handbook-generator |生成技能后,更新仓库主以列出这些技能。
README.md格式: 将生成的技能添加到“可用插件”表格中,紧跟在之后。使用纯文本作为作者(无需链接)。
testing-handbook-skillstesting-handbook-generator示例:
markdown
| 插件 | 描述 | 作者 |
|--------|-------------|--------|
| ... 其他插件 ... |
| [testing-handbook-skills](plugins/testing-handbook-skills/) | 从测试手册生成技能的元技能 | Paweł Płatek |
| [libfuzzer](plugins/testing-handbook-skills/skills/libfuzzer/) | 用于C/C++的覆盖率引导模糊测试工具libFuzzer | testing-handbook-generator |
| [aflpp](plugins/testing-handbook-skills/skills/aflpp/) | 多核模糊测试工具AFL++ | testing-handbook-generator |
| [semgrep](plugins/testing-handbook-skills/skills/semgrep/) | 快速静态分析工具,用于发现漏洞 | testing-handbook-generator |2. Update Skills Cross-Reference
2. 更新技能交叉引用
After generating skills, update the 's Skills Cross-Reference section with the mermaid graph showing skill relationships.
README.mdProcess:
- Read each generated skill's and extract its
SKILL.mdsection## Related Skills - Build the mermaid graph with nodes grouped by skill type (Fuzzers, Techniques, Tools, Domain)
- Add edges based on the Related Skills relationships:
- Solid arrows () for primary technique dependencies
--> - Dashed arrows () for alternative tool suggestions
-.->
- Solid arrows (
- Replace the existing mermaid code block in README.md
Edge classification:
| Relationship | Arrow Style | Example |
|---|---|---|
| Fuzzer → Technique | | |
| Tool → Tool (alternative) | | |
| Fuzzer → Fuzzer (alternative) | | |
| Technique → Technique | | |
Validation: After updating, run to verify all referenced skills exist.
validate-skills.py生成技能后,更新的技能交叉引用章节,使用mermaid图展示技能关系。
README.md流程:
- 读取每个生成技能的并提取其
SKILL.md章节## Related Skills - 构建mermaid图,按技能类型(模糊测试工具、方法、工具、领域)分组节点
- 根据“相关技能”关系添加边:
- 实线箭头 () 表示主要方法依赖
--> - 虚线箭头 () 表示替代工具建议
-.->
- 实线箭头 (
- 替换README.md中现有的mermaid代码块
边分类:
| 关系类型 | 箭头样式 | 示例 |
|---|---|---|
| 模糊测试工具 → 方法 | | |
| 工具 → 工具(替代) | | |
| 模糊测试工具 → 模糊测试工具(替代) | | |
| 方法 → 方法 | | |
验证: 更新完成后,运行验证所有引用的技能是否存在。
validate-skills.py3. Self-Improvement
3. 自我优化
After each generation run, reflect on what could improve future runs.
Capture improvements to:
- Templates (missing sections, better structure)
- Discovery logic (missed patterns, false positives)
- Content extraction (shortcodes not handled, formatting issues)
Update process:
- Note issues encountered during generation
- Identify patterns that caused problems
- Update relevant files:
- - Workflow, decision tree, quick reference updates
SKILL.md - - Template improvements
templates/*.md - - Detection logic updates
discovery.md - - New validation checks
testing.md
- Document the improvement in commit message
Example self-improvement:
Issue: libFuzzer skill missing sanitizer flags table
Fix: Updated templates/fuzzer-skill.md to include ## Compiler Flags section每次生成运行后,反思如何改进未来的运行。
需记录的改进点:
- 模板(缺失章节、更优结构)
- 发现逻辑(遗漏模式、误报)
- 内容提取(未处理的短代码、格式问题)
更新流程:
- 记录生成过程中遇到的问题
- 识别导致问题的模式
- 更新相关文件:
- - 工作流、决策树、快速参考更新
SKILL.md - - 模板改进
templates/*.md - - 检测逻辑更新
discovery.md - - 新增验证检查项
testing.md
- 在提交信息中记录改进内容
自我优化示例:
问题:libFuzzer技能缺失 sanitizer 标志表格
修复:更新templates/fuzzer-skill.md以添加## 编译器标志章节Example Usage
使用示例
Full Discovery and Generation
完整发现与生成流程
User: "Generate skills from the testing handbook"
1. Locate handbook (check common locations, ask user, or clone)
2. Read discovery.md for methodology
3. Scan handbook at {handbook_path}/content/docs/
4. Build candidate list with types
5. Present plan to user
6. On approval, generate each skill using appropriate template
7. Validate generated skills
8. Update main README.md with generated skills table
9. Update README.md Skills Cross-Reference graph from Related Skills sections
10. Self-improve: note any template/discovery issues for future runs
11. Report results用户:"从测试手册生成技能"
1. 定位手册(检查常见位置、询问用户或克隆)
2. 阅读discovery.md了解方法论
3. 扫描手册目录{handbook_path}/content/docs/
4. 构建带类型的候选技能列表
5. 向用户提交计划
6. 获得批准后,使用对应模板生成每个技能
7. 验证生成的技能
8. 更新主README.md的生成技能表格
9. 根据“相关技能”章节更新README.md的技能交叉引用图
10. 自我优化:记录模板/发现环节的问题以便未来改进
11. 报告结果Single Section Generation
单章节生成
User: "Create a skill for the libFuzzer section"
1. Read /testing-handbook/content/docs/fuzzing/c-cpp/10-libfuzzer/
2. Identify type: Fuzzer Skill
3. Read templates/fuzzer-skill.md
4. Extract content, apply template
5. Write to skills/libfuzzer/SKILL.md
6. Validate and report用户:"为libFuzzer章节创建技能"
1. 读取 /testing-handbook/content/docs/fuzzing/c-cpp/10-libfuzzer/
2. 识别类型:模糊测试工具技能
3. 阅读templates/fuzzer-skill.md
4. 提取内容,应用模板
5. 写入skills/libfuzzer/SKILL.md
6. 验证并报告Tips
注意事项
Do:
- Always present plan before generating
- Use appropriate template for skill type
- Preserve code blocks exactly
- Validate after generation
Don't:
- Generate without user approval
- Skip fetching non-video external resources (use WebFetch)
- Fetch video URLs (YouTube, Vimeo - titles only)
- Include handbook images directly
- Skip validation step
- Exceed 500 lines per SKILL.md
For first-time use: Start with discovery.md to understand the handbook analysis process.
For template reference: See templates/ directory for skill type templates.
For validation: See testing.md for quality assurance methodology.
请遵循:
- 生成前务必向用户提交计划并确认
- 根据技能类型使用合适的模板
- 完全保留代码块内容
- 生成后进行验证
请勿:
- 未获得用户批准即开始生成
- 跳过获取非视频类外部资源(使用WebFetch)
- 获取视频URL(YouTube、Vimeo - 仅保留标题)
- 直接包含手册图片
- 跳过验证步骤
- 单个SKILL.md文件超过500行
首次使用: 从discovery.md开始,了解手册分析流程。
模板参考: 查看templates/目录获取各技能类型模板。
验证参考: 查看testing.md了解质量保证方法论。