coverage-analysis
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseCoverage Analysis
覆盖率分析
Coverage analysis is essential for understanding which parts of your code are exercised during fuzzing. It helps identify fuzzing blockers like magic value checks and tracks the effectiveness of harness improvements over time.
覆盖率分析对于了解fuzzing期间代码的哪些部分被执行至关重要。它有助于识别诸如魔术值检查之类的fuzzing阻碍因素,并跟踪harness改进的长期有效性。
Overview
概述
Code coverage during fuzzing serves two critical purposes:
- Assessing harness effectiveness: Understand which parts of your application are actually executed by your fuzzing harnesses
- Tracking fuzzing progress: Monitor how coverage changes when updating harnesses, fuzzers, or the system under test (SUT)
Coverage is a proxy for fuzzer capability and performance. While coverage is not ideal for measuring fuzzer performance in absolute terms, it reliably indicates whether your harness works effectively in a given setup.
Fuzzing期间的代码覆盖率有两个关键用途:
- 评估harness有效性:了解应用程序的哪些部分实际被你的fuzzing harness执行
- 跟踪fuzzing进度:在更新harness、fuzzer或被测系统(SUT)时,监控覆盖率的变化
覆盖率是衡量fuzzer能力和性能的一个指标。尽管从绝对意义上讲,覆盖率并不是衡量fuzzer性能的理想指标,但它可以可靠地表明你的harness在特定设置下是否有效工作。
Key Concepts
核心概念
| Concept | Description |
|---|---|
| Coverage instrumentation | Compiler flags that track which code paths are executed |
| Corpus coverage | Coverage achieved by running all test cases in a fuzzing corpus |
| Magic value checks | Hard-to-discover conditional checks that block fuzzer progress |
| Coverage-guided fuzzing | Fuzzing strategy that prioritizes inputs that discover new code paths |
| Coverage report | Visual or textual representation of executed vs. unexecuted code |
| 概念 | 描述 |
|---|---|
| Coverage instrumentation | 用于跟踪哪些代码路径被执行的编译器标志 |
| Corpus coverage | 运行fuzzing corpus中所有测试用例所达到的覆盖率 |
| Magic value checks | 难以发现的条件检查,会阻碍fuzzer的进展 |
| Coverage-guided fuzzing | 优先选择能发现新代码路径的输入的fuzzing策略 |
| Coverage report | 已执行代码与未执行代码的可视化或文本表示 |
When to Apply
适用场景
Apply this technique when:
- Starting a new fuzzing campaign to establish a baseline
- Fuzzer appears to plateau without finding new paths
- After harness modifications to verify improvements
- When migrating between different fuzzers
- Identifying areas requiring dictionary entries or seed inputs
- Debugging why certain code paths aren't reached
Skip this technique when:
- Fuzzing campaign is actively finding crashes
- Coverage infrastructure isn't set up yet
- Working with extremely large codebases where full coverage reports are impractical
- Fuzzer's internal coverage metrics are sufficient for your needs
适用于以下场景:
- 启动新的fuzzing活动以建立基准
- Fuzzer进入平台期,无法发现新路径
- 修改harness后验证改进效果
- 在不同fuzzer之间迁移时
- 识别需要字典条目或种子输入的区域
- 调试某些代码路径无法被触及的原因
以下场景可跳过:
- Fuzzing活动正在积极发现崩溃问题
- 尚未搭建覆盖率基础设施
- 处理超大型代码库,生成完整覆盖率报告不切实际
- Fuzzer的内部覆盖率指标已满足需求
Quick Reference
快速参考
| Task | Command/Pattern |
|---|---|
| LLVM coverage instrumentation (C/C++) | |
| GCC coverage instrumentation | |
| cargo-fuzz coverage (Rust) | |
| Generate LLVM profile data | |
| LLVM coverage report | |
| LLVM HTML report | |
| gcovr HTML report | |
| 任务 | 命令/模式 |
|---|---|
| LLVM覆盖率插桩(C/C++) | |
| GCC覆盖率插桩 | |
| cargo-fuzz覆盖率分析(Rust) | |
| 生成LLVM profile数据 | |
| 生成LLVM覆盖率报告 | |
| 生成LLVM HTML报告 | |
| 生成gcovr HTML报告 | |
Ideal Coverage Workflow
理想的覆盖率工作流
The following workflow represents best practices for integrating coverage analysis into your fuzzing campaigns:
[Fuzzing Campaign]
|
v
[Generate Corpus]
|
v
[Coverage Analysis]
|
+---> Coverage Increased? --> Continue fuzzing with larger corpus
|
+---> Coverage Decreased? --> Fix harness or investigate SUT changes
|
+---> Coverage Plateaued? --> Add dictionary entries or seed inputsKey principle: Use the corpus generated after each fuzzing campaign to calculate coverage, rather than real-time fuzzer statistics. This approach provides reproducible, comparable measurements across different fuzzing tools.
以下工作流是将覆盖率分析整合到fuzzing活动中的最佳实践:
[Fuzzing活动]
|
v
[生成Corpus]
|
v
[覆盖率分析]
|
+---> 覆盖率提升? --> 使用更大的corpus继续fuzzing
|
+---> 覆盖率下降? --> 修复harness或调查SUT变更
|
+---> 覆盖率停滞? --> 添加字典条目或种子输入核心原则:使用每次fuzzing活动后生成的corpus来计算覆盖率,而非fuzzer的实时统计数据。这种方法可在不同fuzzing工具间提供可复现、可对比的测量结果。
Step-by-Step
分步指南
Step 1: Build with Coverage Instrumentation
步骤1:通过覆盖率插桩构建
Choose your instrumentation method based on toolchain:
LLVM/Clang (C/C++):
bash
clang++ -fprofile-instr-generate -fcoverage-mapping \
-O2 -DNO_MAIN \
main.cc harness.cc execute-rt.cc -o fuzz_execGCC (C/C++):
bash
g++ -ftest-coverage -fprofile-arcs \
-O2 -DNO_MAIN \
main.cc harness.cc execute-rt.cc -o fuzz_exec_gcovRust:
bash
rustup toolchain install nightly --component llvm-tools-preview
cargo +nightly fuzz coverage fuzz_target_1根据工具链选择插桩方式:
LLVM/Clang(C/C++):
bash
clang++ -fprofile-instr-generate -fcoverage-mapping \
-O2 -DNO_MAIN \
main.cc harness.cc execute-rt.cc -o fuzz_execGCC(C/C++):
bash
g++ -ftest-coverage -fprofile-arcs \
-O2 -DNO_MAIN \
main.cc harness.cc execute-rt.cc -o fuzz_exec_gcovRust:
bash
rustup toolchain install nightly --component llvm-tools-preview
cargo +nightly fuzz coverage fuzz_target_1Step 2: Create Execution Runtime (C/C++ only)
步骤2:创建执行运行时(仅C/C++)
For C/C++ projects, create a runtime that executes your corpus:
cpp
// execute-rt.cc
#include <stdio.h>
#include <stdlib.h>
#include <dirent.h>
#include <stdint.h>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
void load_file_and_test(const char *filename) {
FILE *file = fopen(filename, "rb");
if (file == NULL) {
printf("Failed to open file: %s\n", filename);
return;
}
fseek(file, 0, SEEK_END);
long filesize = ftell(file);
rewind(file);
uint8_t *buffer = (uint8_t*) malloc(filesize);
if (buffer == NULL) {
printf("Failed to allocate memory for file: %s\n", filename);
fclose(file);
return;
}
long read_size = (long) fread(buffer, 1, filesize, file);
if (read_size != filesize) {
printf("Failed to read file: %s\n", filename);
free(buffer);
fclose(file);
return;
}
LLVMFuzzerTestOneInput(buffer, filesize);
free(buffer);
fclose(file);
}
int main(int argc, char **argv) {
if (argc != 2) {
printf("Usage: %s <directory>\n", argv[0]);
return 1;
}
DIR *dir = opendir(argv[1]);
if (dir == NULL) {
printf("Failed to open directory: %s\n", argv[1]);
return 1;
}
struct dirent *entry;
while ((entry = readdir(dir)) != NULL) {
if (entry->d_type == DT_REG) {
char filepath[1024];
snprintf(filepath, sizeof(filepath), "%s/%s", argv[1], entry->d_name);
load_file_and_test(filepath);
}
}
closedir(dir);
return 0;
}对于C/C++项目,创建一个用于执行corpus的运行时:
cpp
// execute-rt.cc
#include <stdio.h>
#include <stdlib.h>
#include <dirent.h>
#include <stdint.h>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
void load_file_and_test(const char *filename) {
FILE *file = fopen(filename, "rb");
if (file == NULL) {
printf("Failed to open file: %s\n", filename);
return;
}
fseek(file, 0, SEEK_END);
long filesize = ftell(file);
rewind(file);
uint8_t *buffer = (uint8_t*) malloc(filesize);
if (buffer == NULL) {
printf("Failed to allocate memory for file: %s\n", filename);
fclose(file);
return;
}
long read_size = (long) fread(buffer, 1, filesize, file);
if (read_size != filesize) {
printf("Failed to read file: %s\n", filename);
free(buffer);
fclose(file);
return;
}
LLVMFuzzerTestOneInput(buffer, filesize);
free(buffer);
fclose(file);
}
int main(int argc, char **argv) {
if (argc != 2) {
printf("Usage: %s <directory>\n", argv[0]);
return 1;
}
DIR *dir = opendir(argv[1]);
if (dir == NULL) {
printf("Failed to open directory: %s\n", argv[1]);
return 1;
}
struct dirent *entry;
while ((entry = readdir(dir)) != NULL) {
if (entry->d_type == DT_REG) {
char filepath[1024];
snprintf(filepath, sizeof(filepath), "%s/%s", argv[1], entry->d_name);
load_file_and_test(filepath);
}
}
closedir(dir);
return 0;
}Step 3: Execute on Corpus
步骤3:在Corpus上执行
LLVM (C/C++):
bash
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec corpus/GCC (C/C++):
bash
./fuzz_exec_gcov corpus/Rust:
Coverage data is automatically generated when running .
cargo fuzz coverageLLVM(C/C++):
bash
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec corpus/GCC(C/C++):
bash
./fuzz_exec_gcov corpus/Rust:
运行时会自动生成覆盖率数据。
cargo fuzz coverageStep 4: Process Coverage Data
步骤4:处理覆盖率数据
LLVM:
bash
undefinedLLVM:
bash
undefinedMerge raw profile data
Merge raw profile data
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
Generate text report
Generate text report
llvm-cov report ./fuzz_exec
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
llvm-cov report ./fuzz_exec
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
Generate HTML report
Generate HTML report
llvm-cov show ./fuzz_exec
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
-format=html -output-dir fuzz_html/
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
-format=html -output-dir fuzz_html/
**GCC with gcovr:**
```bashllvm-cov show ./fuzz_exec
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
-format=html -output-dir fuzz_html/
-instr-profile=fuzz.profdata
-ignore-filename-regex='harness.cc|execute-rt.cc'
-format=html -output-dir fuzz_html/
**GCC + gcovr:**
```bashInstall gcovr (via pip for latest version)
Install gcovr (via pip for latest version)
python3 -m venv venv
source venv/bin/activate
pip3 install gcovr
python3 -m venv venv
source venv/bin/activate
pip3 install gcovr
Generate report
Generate report
gcovr --gcov-executable "llvm-cov gcov"
--exclude harness.cc --exclude execute-rt.cc
--root . --html-details -o coverage.html
--exclude harness.cc --exclude execute-rt.cc
--root . --html-details -o coverage.html
**Rust:**
```bashgcovr --gcov-executable "llvm-cov gcov"
--exclude harness.cc --exclude execute-rt.cc
--root . --html-details -o coverage.html
--exclude harness.cc --exclude execute-rt.cc
--root . --html-details -o coverage.html
**Rust:**
```bashInstall required tools
Install required tools
cargo install cargo-binutils rustfilt
cargo install cargo-binutils rustfilt
Create HTML generation script
Create HTML generation script
cat <<'EOF' > ./generate_html
#!/bin/sh
if [ $# -lt 1 ]; then
echo "Error: Name of fuzz target is required."
echo "Usage: $0 fuzz_target [sources...]"
exit 1
fi
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt
"target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET"
-instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata"
-show-line-counts-or-regions -show-instantiations
-format=html -o fuzz_html/ $SRC_FILTER EOF chmod +x ./generate_html
"target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET"
-instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata"
-show-line-counts-or-regions -show-instantiations
-format=html -o fuzz_html/ $SRC_FILTER EOF chmod +x ./generate_html
cat <<'EOF' > ./generate_html
#!/bin/sh
if [ $# -lt 1 ]; then
echo "Error: Name of fuzz target is required."
echo "Usage: $0 fuzz_target [sources...]"
exit 1
fi
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt
"target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET"
-instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata"
-show-line-counts-or-regions -show-instantiations
-format=html -o fuzz_html/ $SRC_FILTER EOF chmod +x ./generate_html
"target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET"
-instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata"
-show-line-counts-or-regions -show-instantiations
-format=html -o fuzz_html/ $SRC_FILTER EOF chmod +x ./generate_html
Generate HTML report
Generate HTML report
./generate_html fuzz_target_1 src/lib.rs
undefined./generate_html fuzz_target_1 src/lib.rs
undefinedStep 5: Analyze Results
步骤5:分析结果
Review the coverage report to identify:
- Uncovered code blocks: Areas that may need better seed inputs or dictionary entries
- Magic value checks: Conditional statements with hardcoded values that block progress
- Dead code: Functions that may not be reachable through your harness
- Coverage changes: Compare against baseline to track improvements or regressions
查看覆盖率报告以识别:
- 未覆盖的代码块:可能需要更好的种子输入或字典条目的区域
- 魔术值检查:包含硬编码值的条件语句,会阻碍fuzzing进展
- 死代码:无法通过harness触及的函数
- 覆盖率变化:与基准对比,跟踪改进或退化情况
Common Patterns
常见模式
Pattern: Identifying Magic Values
模式:识别魔术值
Problem: Fuzzer cannot discover paths guarded by magic value checks.
Coverage reveals:
cpp
// Coverage shows this block is never executed
if (buf == 0x7F454C46) { // ELF magic number
// start parsing buf
}Solution: Add magic values to dictionary file:
undefined问题:Fuzzer无法发现受魔术值检查保护的路径。
覆盖率分析揭示:
cpp
// Coverage shows this block is never executed
if (buf == 0x7F454C46) { // ELF magic number
// start parsing buf
}解决方案:将魔术值添加到字典文件:
undefinedmagic.dict
magic.dict
"\x7F\x45\x4C\x46"
undefined"\x7F\x45\x4C\x46"
undefinedPattern: Handling Crashing Inputs
模式:处理崩溃输入
Problem: Coverage generation fails when corpus contains crashing inputs.
Before:
bash
./fuzz_exec corpus/ # Crashes on bad input, no coverage generatedAfter:
cpp
// Fork before executing to isolate crashes
int main(int argc, char **argv) {
// ... directory opening code ...
while ((entry = readdir(dir)) != NULL) {
if (entry->d_type == DT_REG) {
pid_t pid = fork();
if (pid == 0) {
// Child process - crash won't affect parent
char filepath[1024];
snprintf(filepath, sizeof(filepath), "%s/%s", argv[1], entry->d_name);
load_file_and_test(filepath);
exit(0);
} else {
// Parent waits for child
waitpid(pid, NULL, 0);
}
}
}
}问题:当corpus包含崩溃输入时,覆盖率生成失败。
改进前:
bash
./fuzz_exec corpus/ # Crashes on bad input, no coverage generated改进后:
cpp
// Fork before executing to isolate crashes
int main(int argc, char **argv) {
// ... directory opening code ...
while ((entry = readdir(dir)) != NULL) {
if (entry->d_type == DT_REG) {
pid_t pid = fork();
if (pid == 0) {
// Child process - crash won't affect parent
char filepath[1024];
snprintf(filepath, sizeof(filepath), "%s/%s", argv[1], entry->d_name);
load_file_and_test(filepath);
exit(0);
} else {
// Parent waits for child
waitpid(pid, NULL, 0);
}
}
}
}Pattern: CMake Integration
模式:CMake集成
Use Case: Adding coverage builds to CMake projects.
cmake
project(FuzzingProject)
cmake_minimum_required(VERSION 3.0)使用场景:为CMake项目添加覆盖率构建。
cmake
project(FuzzingProject)
cmake_minimum_required(VERSION 3.0)Main binary
Main binary
add_executable(program main.cc)
add_executable(program main.cc)
Fuzzing binary
Fuzzing binary
add_executable(fuzz main.cc harness.cc)
target_compile_definitions(fuzz PRIVATE NO_MAIN=1)
target_compile_options(fuzz PRIVATE -g -O2 -fsanitize=fuzzer)
target_link_libraries(fuzz -fsanitize=fuzzer)
add_executable(fuzz main.cc harness.cc)
target_compile_definitions(fuzz PRIVATE NO_MAIN=1)
target_compile_options(fuzz PRIVATE -g -O2 -fsanitize=fuzzer)
target_link_libraries(fuzz -fsanitize=fuzzer)
Coverage execution binary
Coverage execution binary
add_executable(fuzz_exec main.cc harness.cc execute-rt.cc)
target_compile_definitions(fuzz_exec PRIVATE NO_MAIN)
target_compile_options(fuzz_exec PRIVATE -O2 -fprofile-instr-generate -fcoverage-mapping)
target_link_libraries(fuzz_exec -fprofile-instr-generate)
Build:
```bash
cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ .
cmake --build . --target fuzz_execadd_executable(fuzz_exec main.cc harness.cc execute-rt.cc)
target_compile_definitions(fuzz_exec PRIVATE NO_MAIN)
target_compile_options(fuzz_exec PRIVATE -O2 -fprofile-instr-generate -fcoverage-mapping)
target_link_libraries(fuzz_exec -fprofile-instr-generate)
构建:
```bash
cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ .
cmake --build . --target fuzz_execAdvanced Usage
高级用法
Tips and Tricks
技巧与窍门
| Tip | Why It Helps |
|---|---|
Use LLVM 18+ with | Organizes large reports by directory structure instead of flat file list |
| Export to lcov format for better HTML | |
| Compare coverage across campaigns | Store |
| Filter harness code from reports | Use |
| Automate coverage in CI/CD | Generate coverage reports automatically after scheduled fuzzing runs |
| Use gcovr 5.1+ for Clang 14+ | Older gcovr versions have compatibility issues with recent LLVM |
| 技巧 | 作用 |
|---|---|
使用LLVM 18+及 | 按目录结构组织大型报告,而非扁平文件列表 |
| 导出为lcov格式以生成更优HTML报告 | |
| 跨活动对比覆盖率 | 为 |
| 从报告中过滤harness代码 | 使用 |
| 在CI/CD中自动化覆盖率分析 | 定期fuzzing运行后自动生成覆盖率报告 |
| 为Clang 14+使用gcovr 5.1+ | 旧版gcovr与新版LLVM存在兼容性问题 |
Incremental Coverage Updates
增量覆盖率更新
GCC's gcov instrumentation incrementally updates files across multiple runs. This is useful for tracking coverage as you add test cases:
.gcdabash
undefinedGCC的gcov插桩可在多次运行中增量更新文件。这在添加测试用例时跟踪覆盖率非常有用:
.gcdabash
undefinedFirst run
First run
./fuzz_exec_gcov corpus_batch_1/
gcovr --html coverage_v1.html
./fuzz_exec_gcov corpus_batch_1/
gcovr --html coverage_v1.html
Second run (adds to existing coverage)
Second run (adds to existing coverage)
./fuzz_exec_gcov corpus_batch_2/
gcovr --html coverage_v2.html
./fuzz_exec_gcov corpus_batch_2/
gcovr --html coverage_v2.html
Start fresh
Start fresh
gcovr --delete # Remove .gcda files
./fuzz_exec_gcov corpus/
undefinedgcovr --delete # Remove .gcda files
./fuzz_exec_gcov corpus/
undefinedHandling Large Codebases
处理大型代码库
For projects with hundreds of source files:
-
Filter by prefix: Only generate reports for relevant directoriesbash
llvm-cov show ./fuzz_exec -instr-profile=fuzz.profdata /path/to/src/ -
Use directory coverage: Group by directory to reduce clutter (LLVM 18+)bash
llvm-cov show -show-directory-coverage -format=html -output-dir html/ -
Generate JSON for programmatic analysis:bash
llvm-cov export -format=lcov > coverage.json
对于包含数百个源文件的项目:
-
按前缀过滤:仅为相关目录生成报告bash
llvm-cov show ./fuzz_exec -instr-profile=fuzz.profdata /path/to/src/ -
使用目录覆盖率:按目录分组以减少混乱(LLVM 18+)bash
llvm-cov show -show-directory-coverage -format=html -output-dir html/ -
导出为JSON以进行程序化分析:bash
llvm-cov export -format=lcov > coverage.json
Differential Coverage
差异覆盖率
Compare coverage between two fuzzing campaigns:
bash
undefined对比两次fuzzing活动的覆盖率:
bash
undefinedCampaign 1
Campaign 1
LLVM_PROFILE_FILE=campaign1.profraw ./fuzz_exec corpus1/
llvm-profdata merge -sparse campaign1.profraw -o campaign1.profdata
LLVM_PROFILE_FILE=campaign1.profraw ./fuzz_exec corpus1/
llvm-profdata merge -sparse campaign1.profraw -o campaign1.profdata
Campaign 2
Campaign 2
LLVM_PROFILE_FILE=campaign2.profraw ./fuzz_exec corpus2/
llvm-profdata merge -sparse campaign2.profraw -o campaign2.profdata
LLVM_PROFILE_FILE=campaign2.profraw ./fuzz_exec corpus2/
llvm-profdata merge -sparse campaign2.profraw -o campaign2.profdata
Compare
Compare
llvm-cov show ./fuzz_exec
-instr-profile=campaign2.profdata
-instr-profile=campaign1.profdata
-show-line-counts-or-regions
-instr-profile=campaign2.profdata
-instr-profile=campaign1.profdata
-show-line-counts-or-regions
undefinedllvm-cov show ./fuzz_exec
-instr-profile=campaign2.profdata
-instr-profile=campaign1.profdata
-show-line-counts-or-regions
-instr-profile=campaign2.profdata
-instr-profile=campaign1.profdata
-show-line-counts-or-regions
undefinedAnti-Patterns
反模式
| Anti-Pattern | Problem | Correct Approach |
|---|---|---|
| Using fuzzer-reported coverage for comparisons | Different fuzzers calculate coverage differently, making cross-tool comparison meaningless | Use dedicated coverage tools (llvm-cov, gcovr) for reproducible measurements |
| Generating coverage with optimizations | | Use |
| Not filtering harness code | Harness coverage inflates numbers and obscures SUT coverage | Use |
| Mixing LLVM and GCC instrumentation | Incompatible formats cause parsing failures | Stick to one toolchain for coverage builds |
| Ignoring crashing inputs | Crashes prevent coverage generation, hiding real coverage data | Fix crashes first, or use process forking to isolate them |
| Not tracking coverage over time | One-time coverage checks miss regressions and improvements | Store coverage data with timestamps and track trends |
| 反模式 | 问题 | 正确做法 |
|---|---|---|
| 使用fuzzer报告的覆盖率进行对比 | 不同fuzzer的覆盖率计算方式不同,跨工具对比无意义 | 使用专用覆盖率工具(llvm-cov、gcovr)获取可复现的测量结果 |
| 带优化选项生成覆盖率 | | 覆盖率构建使用 |
| 未过滤harness代码 | Harness的覆盖率会夸大数值并掩盖SUT的覆盖率 | 使用 |
| 混合使用LLVM与GCC插桩 | 格式不兼容会导致解析失败 | 统一使用同一工具链进行覆盖率构建 |
| 忽略崩溃输入 | 崩溃会阻止覆盖率生成,隐藏真实的覆盖率数据 | 先修复崩溃,或使用进程分叉隔离崩溃 |
| 未跟踪长期覆盖率变化 | 单次覆盖率检查无法发现退化与改进 | 为覆盖率数据添加时间戳并跟踪趋势 |
Tool-Specific Guidance
工具特定指南
libFuzzer
libFuzzer
libFuzzer uses LLVM's SanitizerCoverage by default for guiding fuzzing, but you need separate instrumentation for generating reports.
Build for coverage:
bash
clang++ -fprofile-instr-generate -fcoverage-mapping \
-O2 -DNO_MAIN \
main.cc harness.cc execute-rt.cc -o fuzz_execExecute corpus and generate report:
bash
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec corpus/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov show ./fuzz_exec -instr-profile=fuzz.profdata -format=html -output-dir html/Integration tips:
- Don't use for coverage builds (it conflicts with profile instrumentation)
-fsanitize=fuzzer - Reuse the same harness function () with a different main function
LLVMFuzzerTestOneInput - Use the flag to exclude harness code from coverage reports
-ignore-filename-regex - Consider using llvm-cov's flag for template-heavy C++ code
-show-instantiation
libFuzzer默认使用SanitizerCoverage提供反馈;覆盖率分析用于评估harness的有效性。
为覆盖率构建:
bash
clang++ -fprofile-instr-generate -fcoverage-mapping \
-O2 -DNO_MAIN \
main.cc harness.cc execute-rt.cc -o fuzz_exec执行corpus并生成报告:
bash
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec corpus/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov show ./fuzz_exec -instr-profile=fuzz.profdata -format=html -output-dir html/集成技巧:
- 覆盖率构建不要使用(与profile插桩冲突)
-fsanitize=fuzzer - 复用相同的harness函数(),搭配不同的主函数
LLVMFuzzerTestOneInput - 使用参数从覆盖率报告中排除harness代码
-ignore-filename-regex - 对于重度使用模板的C++代码,可考虑使用llvm-cov的参数
-show-instantiation
AFL++
AFL++
AFL++ provides its own coverage feedback mechanism, but for detailed reports use standard LLVM/GCC tools.
Build for coverage with LLVM:
bash
clang++ -fprofile-instr-generate -fcoverage-mapping \
-O2 main.cc harness.cc execute-rt.cc -o fuzz_execBuild for coverage with GCC:
bash
AFL_USE_ASAN=0 afl-gcc -ftest-coverage -fprofile-arcs \
main.cc harness.cc execute-rt.cc -o fuzz_exec_gcovExecute and generate report:
bash
undefinedAFL++提供自有覆盖率反馈机制,但如需详细报告需使用标准LLVM/GCC工具。
使用LLVM进行覆盖率构建:
bash
clang++ -fprofile-instr-generate -fcoverage-mapping \
-O2 main.cc harness.cc execute-rt.cc -o fuzz_exec使用GCC进行覆盖率构建:
bash
AFL_USE_ASAN=0 afl-gcc -ftest-coverage -fprofile-arcs \
main.cc harness.cc execute-rt.cc -o fuzz_exec_gcov执行并生成报告:
bash
undefinedLLVM approach
LLVM approach
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec afl_output/queue/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov report ./fuzz_exec -instr-profile=fuzz.profdata
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec afl_output/queue/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov report ./fuzz_exec -instr-profile=fuzz.profdata
GCC approach
GCC approach
./fuzz_exec_gcov afl_output/queue/
gcovr --html-details -o coverage.html
**Integration tips:**
- Don't use AFL++'s instrumentation (`afl-clang-fast`) for coverage builds
- Use standard compilers with coverage flags instead
- AFL++'s `queue/` directory contains your corpus
- AFL++'s built-in coverage statistics are useful for real-time monitoring but not for detailed analysis./fuzz_exec_gcov afl_output/queue/
gcovr --html-details -o coverage.html
**集成技巧:**
- 覆盖率构建不要使用AFL++的插桩(`afl-clang-fast`)
- 改用标准编译器搭配覆盖率标志
- AFL++的`queue/`目录包含你的corpus
- AFL++的内置覆盖率统计适用于实时监控,但不适用于详细分析cargo-fuzz (Rust)
cargo-fuzz(Rust)
cargo-fuzz provides built-in coverage generation using LLVM tools.
Install prerequisites:
bash
rustup toolchain install nightly --component llvm-tools-preview
cargo install cargo-binutils rustfiltGenerate coverage data:
bash
cargo +nightly fuzz coverage fuzz_target_1Create HTML report script:
bash
cat <<'EOF' > ./generate_html
#!/bin/sh
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt \
"target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET" \
-instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata" \
-show-line-counts-or-regions -show-instantiations \
-format=html -o fuzz_html/ $SRC_FILTER
EOF
chmod +x ./generate_htmlGenerate report:
bash
./generate_html fuzz_target_1 src/lib.rsIntegration tips:
- Always use the nightly toolchain for coverage
- The flag makes function names readable
-Xdemangler=rustfilt - Filter by source files (e.g., ) to focus on crate code
src/lib.rs - Use and
-show-line-counts-or-regionsfor better Rust-specific output-show-instantiations - Corpus is located in
fuzz/corpus/<target>/
cargo-fuzz使用LLVM工具提供内置的覆盖率生成功能。
安装前置依赖:
bash
rustup toolchain install nightly --component llvm-tools-preview
cargo install cargo-binutils rustfilt生成覆盖率数据:
bash
cargo +nightly fuzz coverage fuzz_target_1创建HTML报告脚本:
bash
cat <<'EOF' > ./generate_html
#!/bin/sh
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt \
"target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET" \
-instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata" \
-show-line-counts-or-regions -show-instantiations \
-format=html -o fuzz_html/ $SRC_FILTER
EOF
chmod +x ./generate_html生成报告:
bash
./generate_html fuzz_target_1 src/lib.rs集成技巧:
- 覆盖率分析始终使用nightly工具链
- 参数可让函数名称更易读
-Xdemangler=rustfilt - 按源文件过滤(如)以聚焦于 crate 代码
src/lib.rs - 使用和
-show-line-counts-or-regions参数获取更适合Rust的输出-show-instantiations - Corpus位于目录
fuzz/corpus/<target>/
honggfuzz
honggfuzz
honggfuzz works with standard LLVM/GCC coverage instrumentation.
Build for coverage:
bash
undefinedhonggfuzz可与标准LLVM/GCC覆盖率插桩配合使用。
为覆盖率构建:
bash
undefinedUse standard compiler, not honggfuzz compiler
Use standard compiler, not honggfuzz compiler
clang -fprofile-instr-generate -fcoverage-mapping
-O2 harness.c execute-rt.c -o fuzz_exec
-O2 harness.c execute-rt.c -o fuzz_exec
**Execute corpus:**
```bash
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec honggfuzz_workspace/Integration tips:
- Don't use for coverage builds
hfuzz-clang - honggfuzz corpus is typically in a workspace directory
- Use the same LLVM workflow as libFuzzer
clang -fprofile-instr-generate -fcoverage-mapping
-O2 harness.c execute-rt.c -o fuzz_exec
-O2 harness.c execute-rt.c -o fuzz_exec
**执行corpus:**
```bash
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec honggfuzz_workspace/集成技巧:
- 覆盖率构建不要使用
hfuzz-clang - honggfuzz的corpus通常位于工作区目录
- 使用与libFuzzer相同的LLVM工作流
Troubleshooting
故障排除
| Issue | Cause | Solution |
|---|---|---|
| Profile wasn't generated or wrong path | Verify |
| Mismatch between binary and profile data | Rebuild binary with same flags used during execution |
| Coverage reports show 0% | Wrong binary used for report generation | Use the instrumented binary, not the fuzzing binary |
| | Add |
| Crashes prevent coverage generation | Corpus contains crashing inputs | Filter crashes or use forking approach to isolate failures |
| Coverage decreases after harness change | Harness now skips certain code paths | Review harness logic; may need to support more input formats |
| HTML report is flat file list | Using older LLVM version | Upgrade to LLVM 18+ and use |
| Mixing LLVM and GCC coverage | Rebuild everything with same toolchain |
| 问题 | 原因 | 解决方案 |
|---|---|---|
| 未生成profile或路径错误 | 确认已设置 |
| 二进制文件与profile数据不匹配 | 使用执行时相同的标志重新构建二进制文件 |
| 覆盖率报告显示0% | 生成报告时使用了错误的二进制文件 | 使用带插桩的二进制文件,而非fuzzing二进制文件 |
| | 添加 |
| 崩溃阻止覆盖率生成 | Corpus包含崩溃输入 | 过滤崩溃输入,或使用分叉方法隔离故障 |
| 修改harness后覆盖率下降 | Harness现在跳过了某些代码路径 | 检查harness逻辑;可能需要支持更多输入格式 |
| HTML报告为扁平文件列表 | 使用了旧版LLVM | 升级到LLVM 18+并使用 |
| 混合使用了LLVM与GCC覆盖率 | 使用同一工具链重新构建所有内容 |
Related Skills
相关技能
Tools That Use This Technique
使用该技术的工具
| Skill | How It Applies |
|---|---|
| libfuzzer | Uses SanitizerCoverage for feedback; coverage analysis evaluates harness effectiveness |
| aflpp | Uses edge coverage for feedback; detailed analysis requires separate instrumentation |
| cargo-fuzz | Built-in |
| honggfuzz | Uses edge coverage; analyze with standard LLVM/GCC tools |
| 技能 | 应用方式 |
|---|---|
| libfuzzer | 使用SanitizerCoverage提供反馈;覆盖率分析用于评估harness有效性 |
| aflpp | 使用边缘覆盖率提供反馈;详细分析需要单独的插桩 |
| cargo-fuzz | 为Rust项目提供内置的 |
| honggfuzz | 使用边缘覆盖率;通过标准LLVM/GCC工具进行分析 |
Related Techniques
相关技术
| Skill | Relationship |
|---|---|
| fuzz-harness-writing | Coverage reveals which code paths harness reaches; guides harness improvements |
| fuzzing-dictionaries | Coverage identifies magic value checks that need dictionary entries |
| corpus-management | Coverage analysis helps curate corpora by identifying redundant test cases |
| sanitizers | Coverage helps verify sanitizer-instrumented code is actually executed |
| 技能 | 关联关系 |
|---|---|
| fuzz-harness-writing | 覆盖率分析揭示harness触及的代码路径;指导harness改进 |
| fuzzing-dictionaries | 覆盖率分析识别需要字典条目的魔术值检查 |
| corpus-management | 覆盖率分析帮助整理corpus,识别冗余测试用例 |
| sanitizers | 覆盖率分析帮助验证带sanitizer插桩的代码是否被实际执行 |
Resources
资源
Key External Resources
关键外部资源
LLVM Source-Based Code Coverage
Comprehensive guide to LLVM's profile instrumentation, including advanced features like branch coverage, region coverage, and integration with existing build systems. Covers compiler flags, runtime behavior, and profile data formats.
llvm-cov Command Guide
Detailed CLI reference for llvm-cov commands including , , and . Documents all filtering options, output formats, and integration with llvm-profdata.
showreportexportgcovr Documentation
Complete guide to gcovr tool for generating coverage reports from gcov data. Covers HTML themes, filtering options, multi-directory projects, and CI/CD integration patterns.
SanitizerCoverage Documentation
Low-level documentation for LLVM's SanitizerCoverage instrumentation. Explains inline 8-bit counters, PC tables, and how fuzzers use coverage feedback for guidance.
On the Evaluation of Fuzzer Performance
Research paper examining limitations of coverage as a fuzzing performance metric. Argues for more nuanced evaluation methods beyond simple code coverage percentages.
LLVM Source-Based Code Coverage
LLVM profile插桩的全面指南,包括分支覆盖率、区域覆盖率等高级功能,以及与现有构建系统的集成。涵盖编译器标志、运行时行为和profile数据格式。
llvm-cov Command Guide
llvm-cov命令的详细CLI参考,包括、和子命令。记录了所有过滤选项、输出格式以及与llvm-profdata的集成。
showreportexportgcovr Documentation
使用gcov数据生成覆盖率报告的gcovr工具完整指南。涵盖HTML主题、过滤选项、多目录项目和CI/CD集成模式。
SanitizerCoverage Documentation
LLVM SanitizerCoverage插桩的底层文档。解释了内联8位计数器、PC表以及fuzzer如何使用覆盖率反馈进行引导。
On the Evaluation of Fuzzer Performance
研究论文,探讨了覆盖率作为fuzzing性能指标的局限性。主张采用更细致的评估方法,而非简单的代码覆盖率百分比。
Video Resources
视频资源
Not applicable - coverage analysis is primarily a tooling and workflow topic best learned through documentation and hands-on practice.
不适用 - 覆盖率分析主要是工具和工作流主题,通过文档和实践操作学习效果最佳。