mantis-structural-index
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseStructural Code Index Builder
结构代码索引构建器
This is an optional first-class stage in the Pass Lifecycle Contract. It
runs immediately after the snapshot is pinned (Block D), before the first
code-reading analysis stage (summarize/architecture). It only needs
+ and must not depend on architecture/KB.
CODE_ROOTSNAPSHOT_ID这是Pass Lifecycle Contract中的一个可选一级阶段。它会在快照固定完成后(Block D)立即运行,早于首个代码读取分析阶段(摘要/架构分析)。它仅需 + ,且不得依赖架构/知识库(KB)。
CODE_ROOTSNAPSHOT_IDSystem Goal
系统目标
Structural Code Index Builder. Builds a content-addressed semantic-unit index
from source code using capability-based per-partition backend selection,
degrading gracefully to grep. Provides ,
, and call-site awareness to improve LLM
reasoning quality during discovery — supplementing (never replacing) grep-based
call-site discovery with structural data. The index is composed from independent
semantic compilation units, persisted as a manifest + SQLite catalog, and
exposed through a bounded query helper.
find_callers(symbol)get_function_boundary(file, line)结构代码索引构建器。基于能力的分区后端选择机制,从源代码构建基于内容寻址的语义单元索引,当无法使用更优方案时优雅降级为grep。提供、方法以及调用站点感知能力,以提升发现阶段的LLM推理质量——用结构化数据补充(绝不替代)基于grep的调用站点发现。该索引由独立的语义编译单元组成,以清单(manifest)+ SQLite目录形式持久化,并通过受限查询助手对外暴露。
find_callers(symbol)get_function_boundary(file, line)Command Definition
命令定义
- Command:
/mantis-structural-index - Description: Build a content-addressed semantic-unit index from source
code under .
CODE_ROOT - Arguments (optional; supplied by the orchestrator, consumed by Block A):
/
--snapshot_root/--snapshot_id. All absent → MODE-OFF/legacy mode (reads source from the current directory, writes index to--state_root)../workspace/kb/structural_index/
- 命令:
/mantis-structural-index - 描述: 从下的源代码构建基于内容寻址的语义单元索引。
CODE_ROOT - 参数(可选;由编排器提供,供Block A使用): /
--snapshot_root/--snapshot_id。若所有参数均缺失→进入MODE-OFF/传统模式(从当前目录读取源代码,将索引写入--state_root)。./workspace/kb/structural_index/
Input/Output Contract
输入/输出约定
- Reads:
- (to read
workspace/.mantis_state.jsonfor provenance checking and snapshot-aware rebuild logic).active_snapshot - (to check
workspace/kb/structural_index/manifest.jsonfor reuse-on-match idempotency — primary check).snapshot_id - (backward-compat provenance check if manifest absent).
workspace/kb/structural_index.jsonl - (content-addressed cache for incremental unit reuse).
workspace/kb/structural_index/units/ - and sidecar
workspace/kb/structural_index/native/files (prebuilt index attachments and metadata manifests).provenance.json - CODE_ROOT source files (via generated helper script — the script parses all
source files under ).
CODE_ROOT
- Writes:
- (STATE-RELATIVE — atomic commit point, written LAST).
workspace/kb/structural_index/manifest.json - (STATE-RELATIVE — query-optimized serving store).
workspace/kb/structural_index/catalog.sqlite - (STATE-RELATIVE — content-addressed immutable unit outputs).
workspace/kb/structural_index/units/ - (STATE-RELATIVE — partitioned serving data for large corpora).
workspace/kb/structural_index/shards/ - (STATE-RELATIVE — prebuilt index attachments: SCIP, Kythe, LSIF).
workspace/kb/structural_index/native/ - (STATE-RELATIVE — temporary objects during build).
workspace/kb/structural_index/tmp/ - (STATE-RELATIVE — compatibility pointer; full export below threshold).
workspace/kb/structural_index.jsonl - (STATE-RELATIVE — the builder helper script).
workspace/helpers/build_structural_index.py - (STATE-RELATIVE — the query helper script).
workspace/helpers/query_structural_index.py
- Preconditions:
- Source files must exist under . If
CODE_ROOTis not resolved (MODE-OFF and no readableCODE_ROOT), build against the current directory withactive_snapshotset tosnapshot_id. Do NOT skip — this is the standalone-efficiency case."unknown"
- Source files must exist under
- Inert until wired: This skill returns an empty index until a caller
invokes it (the harness, , or
mantis-plan). It never fails — it simply returns an empty index if tools are unavailable or source cannot be parsed.mantis-researcher - Idempotency Guarantee:
- Read-only on . Writes only to STATE-RELATIVE paths. Re-running with the same
CODE_ROOTandCODE_ROOTreuses the existing index (manifestSNAPSHOT_IDmatch) rather than rebuilding — except in MODE-OFF, where it always rebuilds. Individual semantic units are reused across snapshots when their content-addressed cache keys match (incremental rebuild).snapshot_id
- Read-only on
- 读取:
- (读取
workspace/.mantis_state.json以进行溯源检查和快照感知的重建逻辑)。active_snapshot - (检查
workspace/kb/structural_index/manifest.json以实现匹配时复用的幂等性——主要检查项)。snapshot_id - (若清单缺失,用于向后兼容的溯源检查)。
workspace/kb/structural_index.jsonl - (基于内容寻址的缓存,用于增量单元复用)。
workspace/kb/structural_index/units/ - 及附属
workspace/kb/structural_index/native/文件(预构建的索引附件和元数据清单)。provenance.json - CODE_ROOT下的源代码文件(通过生成的助手脚本读取——该脚本解析CODE_ROOT下所有源代码文件)。
- 写入:
- (与状态相关——原子提交点,最后写入)。
workspace/kb/structural_index/manifest.json - (与状态相关——查询优化的服务存储)。
workspace/kb/structural_index/catalog.sqlite - (与状态相关——基于内容寻址的不可变单元输出)。
workspace/kb/structural_index/units/ - (与状态相关——针对大型语料库的分区服务数据)。
workspace/kb/structural_index/shards/ - (与状态相关——预构建的索引附件:SCIP、Kythe、LSIF)。
workspace/kb/structural_index/native/ - (与状态相关——构建过程中的临时对象)。
workspace/kb/structural_index/tmp/ - (与状态相关——兼容指针;低于阈值时导出完整内容)。
workspace/kb/structural_index.jsonl - (与状态相关——构建助手脚本)。
workspace/helpers/build_structural_index.py - (与状态相关——查询助手脚本)。
workspace/helpers/query_structural_index.py
- 前置条件:
- CODE_ROOT下必须存在源代码文件。若CODE_ROOT未解析(MODE-OFF且无可读的),则针对当前目录构建,
active_snapshot设为snapshot_id。请勿跳过——这是独立高效的使用场景。"unknown"
- CODE_ROOT下必须存在源代码文件。若CODE_ROOT未解析(MODE-OFF且无可读的
- 未触发时无操作: 该技能在被调用前(由 harness、或
mantis-plan调用)会返回空索引。它永远不会失败——若工具不可用或源代码无法解析,仅返回空索引。mantis-researcher - 幂等性保证:
- 对CODE_ROOT仅执行读取操作。仅写入与状态相关的路径。若使用相同的CODE_ROOT和SNAPSHOT_ID重新运行,会复用现有索引(清单匹配)而非重建——但MODE-OFF模式除外,该模式下始终重建。当语义单元的内容寻址缓存键匹配时,可跨快照复用单个语义单元(增量重建)。
snapshot_id
- 对CODE_ROOT仅执行读取操作。仅写入与状态相关的路径。若使用相同的CODE_ROOT和SNAPSHOT_ID重新运行,会复用现有索引(清单
Instructions
操作步骤
Step 0: Locator Resolution (run first)
步骤0:定位器解析(首先运行)
LOCATOR RESOLUTION (before reading ANY target code or artifact):
0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect),
you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from
state for provenance/annotation; NEVER stop merely because a code root is unset.
1. Determine CODE_ROOT, in this priority order:
a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root.
It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback
(used when a caller hands you a prepared tree, e.g. a patched shadow).
b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it.
c. Else read state_root/workspace/.mantis_state.json (state_root from
--state_root if passed, else ./workspace/... relative to the current dir)
-> active_snapshot.root / .snapshot_id / .snapshot_pinned.
d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory,
treat snapshot_pinned = false (MODE-OFF). Do NOT stop.
2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a):
verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing
or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is
deliberately mutated and is sentinel-EXEMPT.)
3. PATH FIELDS:
- SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files
that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry
containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is
NOT of the form <existing-path>:<integer> is a non-source LOCATOR
(symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL
line-range and line-existence logic.
- STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT):
kb_references, repro_file_path, reattack_file_path, helper scripts, report
files, and all state/findings JSON.
4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that
compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy
(mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may
cd into CODE_ROOT.
5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command
run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT
(the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT
lacks .git/.hg/.repo.
6. Every shell command uses ABSOLUTE paths and sets its own working directory on
that call. Do NOT assume the working directory persists between calls.This is a CODE-READING stage — it reads target source files under CODE_ROOT via
the helper script. Block A step 0's findings-only skip does NOT apply.
LOCATOR RESOLUTION (before reading ANY target code or artifact):
0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect),
you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from
state for provenance/annotation; NEVER stop merely because a code root is unset.
1. Determine CODE_ROOT, in this priority order:
a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root.
It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback
(used when a caller hands you a prepared tree, e.g. a patched shadow).
b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it.
c. Else read state_root/workspace/.mantis_state.json (state_root from
--state_root if passed, else ./workspace/... relative to the current dir)
-> active_snapshot.root / .snapshot_id / .snapshot_pinned.
d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory,
treat snapshot_pinned = false (MODE-OFF). Do NOT stop.
2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a):
verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing
or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is
deliberately mutated and is sentinel-EXEMPT.)
3. PATH FIELDS:
- SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files
that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry
containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is
NOT of the form <existing-path>:<integer> is a non-source LOCATOR
(symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL
line-range and line-existence logic.
- STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT):
kb_references, repro_file_path, reattack_file_path, helper scripts, report
files, and all state/findings JSON.
4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that
compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy
(mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may
cd into CODE_ROOT.
5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command
run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT
(the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT
lacks .git/.hg/.repo.
6. Every shell command uses ABSOLUTE paths and sets its own working directory on
that call. Do NOT assume the working directory persists between calls.这是一个代码读取阶段——它通过助手脚本读取CODE_ROOT下的目标源代码。Block A步骤0中的仅发现结果跳过规则不适用于此阶段。
Step 1: Idempotency / Freshness Check (MANDATORY FIRST)
步骤1:幂等性/新鲜度检查(必须首先执行)
MANDATORY FIRST STEP: Before writing any build scripts, probing backends, or
extracting symbols, check for MODE-OFF or an existing manifest:
- MODE-OFF check (FIRST): In MODE-OFF (is
SNAPSHOT_IDor absent) → always rebuild. Do NOT reuse a previous"unknown"index, because the live tree is mutable and"unknown"is a constant (not a freshness signal). Skip directly to Step 2 (individual units whose content has not changed may still hit the content-addressed cache)."unknown" - If exists, read its
workspace/kb/structural_index/manifest.jsonfield.snapshot_id - If matches the current
snapshot_id(andSNAPSHOT_IDis NOTSNAPSHOT_ID) → reuse the index immediately and STOP. Do NOT invoke Step 2, probe backends, or write build scripts. This bounds cost across retries/crash-resume."unknown" - If is absent but
manifest.jsonexists (backward compat), read its provenance header (workspace/kb/structural_index.jsonl,_provenancekeys). Ifsnapshot_idmatches (and is NOTsnapshot_id) → reuse and STOP. Otherwise proceed to rebuild."unknown" - If differs → proceed to rebuild (Steps 2–5). For incremental reuse: before rebuilding a semantic unit, check
SNAPSHOT_IDfor a content-addressed cache hit (see Content-Addressed Cache Key below). A cache hit reuses the unit output without re-extraction.units/
必须首先执行的步骤: 在编写任何构建脚本、探测后端或提取符号之前,检查是否处于MODE-OFF模式或存在现有清单:
- MODE-OFF检查(首先执行): 若处于MODE-OFF模式(为
SNAPSHOT_ID或缺失)→ 始终重建。请勿复用之前的"unknown"索引,因为实时代码树是可变的,"unknown"是常量(而非新鲜度信号)。直接跳至步骤2(内容未更改的单个单元仍可命中基于内容寻址的缓存)。"unknown" - 若存在,读取其
workspace/kb/structural_index/manifest.json字段。snapshot_id - 若与当前
snapshot_id匹配(且SNAPSHOT_ID不为SNAPSHOT_ID)→ 立即复用索引并停止操作。请勿执行步骤2、探测后端或编写构建脚本。这可限制重试/崩溃恢复场景下的成本。"unknown" - 若缺失但
manifest.json存在(向后兼容),读取其溯源头(workspace/kb/structural_index.jsonl、_provenance键)。若snapshot_id匹配(且不为snapshot_id)→ 复用并停止操作。否则继续重建。"unknown" - 若不同→继续重建(步骤2–5)。对于增量复用:重建语义单元前,检查
SNAPSHOT_ID是否存在内容寻址缓存命中(见下文基于内容寻址的缓存键)。缓存命中时可复用单元输出,无需重新提取。units/
Step 2: Select Backend per Partition
步骤2:按分区选择后端
Partition the codebase into semantic units (see Per-Language Semantic Units
below). For each unit, select the most precise backend available in this
environment using a capability-based, per-partition decision — not a single
global ladder. The following are examples, not an exhaustive enum. The
helper probes each tier per partition and selects the highest available:
- Snapshot-matched SCIP / LSIF / Kythe / clangd-static / remote index
(precision: ) — if a pre-built index matching the current
semanticorsnapshot_idis available. Most precise: full type-aware cross-reference, call hierarchy, and hover/signature data.root_fingerprint - Compiler / typechecker-backed extractor (precision: ) — if
typecheck, build context, or typechecker is available. Type-accurate symbol resolution and call edges.compile_commands.json - Language-aware AST extraction (precision: ) — if
ast,tree-sitter, or a language-specific parser is available. Full AST parsing: function boundaries, call expressions, signatures.ast-grep - Symbol-only extraction (precision: ) — if
symbol-onlyor equivalent is onctags. Symbol table only (function definitions, locations — no call graph). Call-site extraction uses a lightweight regex pass within known function boundaries.PATH - Heuristic fallback (precision: ) — Python stdlib regex pass over source files. Identifies function definitions and call patterns using language-agnostic heuristics. Less precise but zero-dependency.
heuristic - Coverage-only manifest + lexical fallback (precision: ) — grep; no structural index is written. Manifest records
coverage-only. Consumers fall back to grep-based discovery (today's behavior byte-for-byte).status: "empty"
Native Index Probing & Resolution Rules:
- Probe instruction: Before evaluating per-partition backends, probe
and subdirectories
workspace/kb/structural_index/native/for prebuilt index files.native/{scip,lsif,kythe}/ - Snapshot-declaration convention: Because native formats (SCIP, LSIF,
Kythe) do not embed snapshot identity directly in their binary payload,
prebuilt indexes MUST declare their target snapshot using a sidecar
manifest located at
provenance.jsonorworkspace/kb/structural_index/native/provenance.json. The manifest contains an array of attachments:native/<kind>/provenance.json. A native index is matched if its declared[{"kind": "scip|lsif|kythe", "path": "...", "snapshot_id": "...", "root_fingerprint": "...", "language": "...", "indexer": "...", "precision": "semantic", "files": [...]}]equalssnapshot_id(whenSNAPSHOT_ID) or itsSNAPSHOT_ID != "unknown"matches the workspace's calculated root fingerprint. If theroot_fingerprintarray is absent or empty, treat the native index as covering no individual files directly (record infilesbut do not updatemanifest.native_indexesrows; fall through to lower tiers for all files).coverage - Record-and-Defer Ingestion Rule: Parsing raw binary native indexes (e.g.
SCIP protobuf) in pure Python without dependencies is costly and complex.
Option A builder scripts MUST detect matching prebuilt native indexes, record
their entries in , and set the
manifest.native_indexestablecoverage(e.g.,backendor"scip") for all files listed in the provenance manifest. If"scip-clangd"is NOT populated with symbols from the native index (raw binary deferred to harness/MCP readers), setcatalog.sqliteandcoverage.status = "deferred"(withprecision = "deferred"). This prevents the query helper from claiming an un-ingested partition is "authoritative empty" atindexed_files = 0precision, ensuring consumers run the mandatory grep fallback. WhensemanticIS populated (e.g., via Option B pre-ingestion orcatalog.sqlite), setscip-to-sqliteandprecision = "semantic".status = "indexed"
LSP is NOT equivalent to SCIP / LSIF. LSP is an interactive protocol whose
workspace state may be partial or mutable. Use it only when the server can
demonstrate snapshot identity AND complete workspace coverage. A running
language server does not automatically qualify as a backend.
semanticSCIP merging. SCIP explicitly permits merging complementary information from
indexers with different precision levels. Its format records indexer version and
per-document language metadata. The builder MAY merge results from multiple
indexers (e.g., a SCIP index for Go + tree-sitter for Python) within a single
catalog.
The determinism lives in a runtime-generated versioned helper
(, , grep-and-regenerate
on reuse) that probes and selects backends per partition. No shipped binaries;
air-gapped-safe.
build_structural_index.py# MANTIS_HELPER_VERSION = 5Deterministic partial coverage. On very large source trees, a full rebuild
can dominate stage-0 wall-clock. Replace discovery-order truncation with a
deterministic priority queue:
- Explicit target files and symbols (from , if available).
plan.json - Changed units and known reverse dependencies (from Block E diff + reverse-dependency edges in the existing catalog).
- Containing packages/modules and direct imports of tier 1+2 units.
- Remaining units in normalized path order.
Apply deterministic (default: 10000) or (default:
500MB) bounds to that queue. Persist deferred units in the manifest so another
invocation can resume. Statuses: , , , .
max_unitsmax_source_bytescompletepartialemptyfailedA consumer should only interpret "no callers" strongly when the relevant
partition is complete, snapshot-matched, covered, and semantically precise.
Otherwise the answer is "no indexed callers", followed by the existing lexical
fallback. The index is HINT-only, so partial coverage is safe (grep remains
authoritative).
将代码库划分为语义单元(见下文按语言划分的语义单元)。针对每个单元,使用基于能力的分区决策选择当前环境中可用的最精准后端——而非单一全局优先级。以下是示例,而非详尽枚举。助手会针对每个分区探测各层级并选择最高可用层级:
- 快照匹配的SCIP / LSIF / Kythe / clangd-static / 远程索引(精度:)——若存在与当前
semantic或snapshot_id匹配的预构建索引。精度最高:支持完整的类型感知交叉引用、调用层级以及悬停/签名数据。root_fingerprint - 编译器/类型检查器驱动的提取器(精度:)——若存在
typecheck、构建上下文或类型检查器。可实现类型准确的符号解析和调用边。compile_commands.json - 感知语言的AST提取(精度:)——若
ast、tree-sitter或特定语言解析器可用。支持完整AST解析:函数边界、调用表达式、签名。ast-grep - 仅符号提取(精度:)——若
symbol-only或等效工具在ctags中。仅提供符号表(函数定义、位置——无调用图)。调用站点提取通过已知函数边界内的轻量级正则扫描实现。PATH - 启发式回退(精度:)——通过Python标准库正则扫描源代码文件。使用语言无关的启发式方法识别函数定义和调用模式。精度较低但无依赖。
heuristic - 仅覆盖清单+词法回退(精度:)——仅使用grep;不写入结构索引。清单记录
coverage-only。消费者回退到基于grep的发现(与当前行为完全一致)。status: "empty"
原生索引探测与解析规则:
- 探测指令: 在评估分区后端之前,探测及其子目录
workspace/kb/structural_index/native/以查找预构建索引文件。native/{scip,lsif,kythe}/ - 快照声明约定: 由于原生格式(SCIP、LSIF、Kythe)未在其二进制负载中直接嵌入快照标识,预构建索引必须通过附属的清单声明其目标快照,该清单位于
provenance.json或workspace/kb/structural_index/native/provenance.json。清单包含一个附件数组:native/<kind>/provenance.json。若原生索引声明的[{"kind": "scip|lsif|kythe", "path": "...", "snapshot_id": "...", "root_fingerprint": "...", "language": "...", "indexer": "...", "precision": "semantic", "files": [...]}]等于snapshot_id(当SNAPSHOT_ID时),或其SNAPSHOT_ID != "unknown"与工作区计算的根指纹匹配,则该索引匹配。若root_fingerprint数组缺失或为空,则认为原生索引不直接覆盖任何单个文件(在files中记录,但不更新manifest.native_indexes行;所有文件均回退到更低层级)。coverage - 记录并延迟摄取规则: 在纯Python环境中无依赖地解析原始二进制原生索引(如SCIP protobuf)成本高昂且复杂。Option A构建脚本必须检测匹配的预构建原生索引,在中记录其条目,并为溯源清单中列出的所有文件设置
manifest.native_indexes表的coverage(例如backend或"scip")。若"scip-clangd"未填充来自原生索引的符号(原始二进制延迟到harness/MCP读取器处理),则设置catalog.sqlite和coverage.status = "deferred"(precision = "deferred")。这可防止查询助手声称未摄取的分区在indexed_files = 0精度下“权威为空”,确保消费者执行强制grep回退。若semantic已填充(例如通过Option B预摄取或catalog.sqlite),则设置scip-to-sqlite和precision = "semantic"。status = "indexed"
LSP不等同于SCIP / LSIF。 LSP是交互式协议,其工作区状态可能不完整或可变。仅当服务器可证明快照标识且覆盖完整工作区时,才可使用LSP。运行中的语言服务器并不自动符合后端的要求。
semanticSCIP合并。 SCIP明确允许合并来自不同精度级别索引器的互补信息。其格式记录了索引器版本和每个文档的语言元数据。构建器可在单个目录中合并来自多个索引器的结果(例如Go的SCIP索引 + Python的tree-sitter索引)。
确定性体现在运行时生成的版本化助手(,,复用前通过grep检查并重新生成)中,该助手会探测并按分区选择后端。无预装二进制文件;支持离线环境。
build_structural_index.py# MANTIS_HELPER_VERSION = 5确定性部分覆盖。 在超大型源代码树上,完整重建可能占据阶段0的主要耗时。将发现顺序截断替换为确定性优先级队列:
- 明确的目标文件和符号(若可用)。
plan.json - 已更改的单元和已知反向依赖(来自Block E差异 + 现有目录中的反向依赖边)。
- 包含的包/模块以及一级+二级单元的直接导入。
- 剩余单元按标准化路径顺序排列。
为该队列应用确定性的(默认:10000)或(默认:500MB)限制。将延迟处理的单元持久化到清单中,以便下次调用可继续。状态包括:、、、。
max_unitsmax_source_bytescompletepartialemptyfailed仅当相关分区已完成、快照匹配、已覆盖且语义精准时,消费者才可将“无调用者”的结果视为可靠。否则结果应为“无索引调用者”,随后执行现有词法回退。索引仅作为提示,因此部分覆盖是安全的(grep仍为权威来源)。
Step 3: Write and Run Helper Scripts
步骤3:编写并运行助手脚本
Two runtime-generated helpers are used. Both follow the grep-and-regenerate
pattern: before reuse, grep the first line for the version marker; if absent or
a different integer, REGENERATE.
使用两个运行时生成的助手脚本。两者均遵循grep并重新生成的模式:复用前,grep第一行查找版本标记;若缺失或版本号不同,则重新生成。
Builder: build_structural_index.py
build_structural_index.py构建器:build_structural_index.py
build_structural_index.py- Write the builder to . The FIRST LINE MUST be exactly
workspace/helpers/build_structural_index.py. Before reusing an existing helper, grep its first lines for# MANTIS_HELPER_VERSION = 5; if that marker is absent or a different integer, REGENERATE the helper.MANTIS_HELPER_VERSION = 5 - The builder partitions the codebase into semantic units (see Per-Language
Semantic Units), computes content-addressed cache keys, checks for reuse, selects a backend per partition (Step 2), extracts symbols + edges, and writes results to
units/.catalog.sqlite - The builder must extract:
- Symbols: ,
symbol_id,name,qualified_name,namespace,language,file_path,start_line,end_line,kind,signature,backend.precision - Call edges: ,
caller_id,callee_id,callee_name,file_path,line.edge_kind - Function boundaries: ,
symbol_id,file_path,start_line,end_line,signature.language - Coverage: ,
file_path,indexed,backend,precision,unit_cache_key.status
- Symbols:
- The builder writes the manifest LAST (Step 4) and the compatibility pointer.
- The builder MUST use ABSOLUTE paths and set its own working directory (Block
A step 6). It MUST NOT write anything under when
CODE_ROOTis true (Block A step 4). Backends that produce sidecar files (ctagssnapshot_pinned, cscopetags, clangd cache) MUST be redirected to STATE-RELATIVE paths:cscope.out,ctags -f <state>/helpers/tags,cscope -f <state>/helpers/cscope.out. Read-only LSP/SCIP queries to a running server need no redirect.CLANGD_INDEX_STORAGE=<state>/helpers/
- 将构建器写入。第一行必须精确为
workspace/helpers/build_structural_index.py。复用现有助手前,grep其前几行查找# MANTIS_HELPER_VERSION = 5;若标记缺失或版本号不同,则重新生成助手。MANTIS_HELPER_VERSION = 5 - 构建器将代码库划分为语义单元(见按语言划分的语义单元),计算基于内容寻址的缓存键,检查是否可复用,按分区选择后端(步骤2),提取符号+边,并将结果写入
units/。catalog.sqlite - 构建器必须提取:
- 符号:、
symbol_id、name、qualified_name、namespace、language、file_path、start_line、end_line、kind、signature、backend。precision - 调用边:、
caller_id、callee_id、callee_name、file_path、line。edge_kind - 函数边界:、
symbol_id、file_path、start_line、end_line、signature。language - 覆盖情况:、
file_path、indexed、backend、precision、unit_cache_key。status
- 符号:
- 构建器最后写入清单(步骤4)和兼容指针。
- 构建器必须使用绝对路径并设置自身工作目录(Block A步骤6)。当为true时,不得写入CODE_ROOT下的任何内容(Block A步骤4)。会生成附属文件的后端(如ctags的
snapshot_pinned、cscope的tags、clangd缓存)必须重定向到与状态相关的路径:cscope.out、ctags -f <state>/helpers/tags、cscope -f <state>/helpers/cscope.out。对运行中服务器的只读LSP/SCIP查询无需重定向。CLANGD_INDEX_STORAGE=<state>/helpers/
Query helper: query_structural_index.py
query_structural_index.py查询助手:query_structural_index.py
query_structural_index.py- Write the query helper to . The FIRST LINE MUST be exactly
workspace/helpers/query_structural_index.py. Before reusing an existing helper, grep its first lines for# MANTIS_HELPER_VERSION = 5; if absent or a different integer, REGENERATE.MANTIS_HELPER_VERSION = 5 - The query helper provides bounded, paginated operations against
(or a remote endpoint — identical API). It IS the consumption contract (see Query Interface below).
catalog.sqlite - If is absent but
catalog.sqliteexists, the query helper falls back to linear scanning of the JSONL file (slower but functional). If both are absent, it returns empty results withstructural_index.jsonl.coverage.partition_status = "empty"
- 将查询助手写入。第一行必须精确为
workspace/helpers/query_structural_index.py。复用现有助手前,grep其前几行查找# MANTIS_HELPER_VERSION = 5;若缺失或版本号不同,则重新生成。MANTIS_HELPER_VERSION = 5 - 查询助手提供针对(或远程端点——API完全一致)的受限分页操作。它是消费约定的核心(见下文查询接口)。
catalog.sqlite - 若缺失但
catalog.sqlite存在,查询助手会回退到线性扫描JSONL文件(速度较慢但可用)。若两者均缺失,则返回空结果,且structural_index.jsonl。coverage.partition_status = "empty"
Step 4: Write Manifest and Commit
步骤4:写入清单并提交
- Write the manifest to first. Then atomically rename it to
workspace/kb/structural_index/tmp/manifest.json. This is the atomic commit point — the manifest is written LAST, after all units, catalog, and coverage data are written.workspace/kb/structural_index/manifest.json - Write a compatibility pointer to (STATE-RELATIVE — NEVER under
workspace/kb/structural_index.jsonl). Below a configurable threshold (default: 10K records), emit a complete JSONL export (provenance header + all records). Above the threshold, emit only a provenance header withCODE_ROOTandcompat_pointer.full_export = falseset. Large consumers MUST use the query interface.compat_pointer.symbol_count - The manifest field records which backend was used per language (e.g.,
provider.backend_versions), so consumers know the precision level.{"go": {"backend_name": "scip-clangd", "precision": "semantic"}, "python": {"backend_name": "tree-sitter", "precision": "ast"}} - Interrupted builds leave unreferenced temp objects in without corrupting the last published index. On resume, check
tmp/status andmanifest.json(Step 1).snapshot_id
- 首先将清单写入。然后原子重命名为
workspace/kb/structural_index/tmp/manifest.json。这是原子提交点——清单最后写入,在所有单元、目录和覆盖数据写入完成之后。workspace/kb/structural_index/manifest.json - 将兼容指针写入(与状态相关——绝不写入CODE_ROOT下)。低于可配置阈值(默认:10000条记录)时,导出完整的JSONL内容(溯源头 + 所有记录)。高于阈值时,仅写入包含
workspace/kb/structural_index.jsonl和compat_pointer.full_export = false的溯源头。大型消费者必须使用查询接口。compat_pointer.symbol_count - 清单的字段记录每种语言使用的后端(例如
provider.backend_versions),以便消费者了解精度级别。{"go": {"backend_name": "scip-clangd", "precision": "semantic"}, "python": {"backend_name": "tree-sitter", "precision": "ast"}} - 中断的构建会在中留下未引用的临时对象,但不会损坏最后发布的索引。恢复时,检查
tmp/的状态和manifest.json(步骤1)。snapshot_id
Step 5: Return Results / Notify Caller
步骤5:返回结果/通知调用者
- Return the path to ,
manifest.json, and the query helper. Include a summary (unit count, symbol count, call-edge count, backends used, coverage status).catalog.sqlite - If the index is empty (no tools available or no source files found), notify the caller: "Structural index is empty — structural context unavailable."
- Do not notify the user directly — this skill is invoked as a sub-agent by the harness, planner, or researcher.
- 返回、
manifest.json和查询助手的路径。包含摘要(单元数量、符号数量、调用边数量、使用的后端、覆盖状态)。catalog.sqlite - 若索引为空(无可用工具或未找到源代码文件),通知调用者:“结构索引为空——结构上下文不可用。”
- 请勿直接通知用户——该技能由harness、规划器或研究者作为子代理调用。
Index Schema and On-Disk Contract
索引架构与磁盘约定
On-Disk Layout
磁盘布局
workspace/kb/structural_index/
├── manifest.json # Atomic commit point — written LAST
├── catalog.sqlite # Query-optimized serving store (both directions indexed)
├── units/ # Content-addressed immutable unit outputs
│ └── ab/cd/abcdef... # sha256 prefix sharding (2+2 hex dirs)
├── shards/ # Partitioned serving data (large corpora)
│ └── shard_0000.sqlite
├── native/ # Prebuilt index attachments (SCIP, Kythe, LSIF)
│ ├── provenance.json # Prebuilt index provenance manifest
│ ├── scip/
│ └── kythe/
└── tmp/ # Temporary objects during build
workspace/helpers/
├── build_structural_index.py # Builder (MANTIS_HELPER_VERSION = 5)
└── query_structural_index.py # Query helper (MANTIS_HELPER_VERSION = 5)
workspace/kb/structural_index.jsonl # Compatibility pointerworkspace/kb/structural_index/
├── manifest.json # 原子提交点 —— 最后写入
├── catalog.sqlite # 查询优化的服务存储(双向索引)
├── units/ # 基于内容寻址的不可变单元输出
│ └── ab/cd/abcdef... # sha256前缀分片(2+2十六进制目录)
├── shards/ # 分区服务数据(大型语料库)
│ └── shard_0000.sqlite
├── native/ # 预构建索引附件(SCIP、Kythe、LSIF)
│ ├── provenance.json # 预构建索引溯源清单
│ ├── scip/
│ └── kythe/
└── tmp/ # 构建过程中的临时对象
workspace/helpers/
├── build_structural_index.py # 构建器(MANTIS_HELPER_VERSION = 5)
└── query_structural_index.py # 查询助手(MANTIS_HELPER_VERSION = 5)
workspace/kb/structural_index.jsonl # 兼容指针Manifest Schema (manifest.json
)
manifest.json清单架构(manifest.json
)
manifest.jsonjson
{
"schema_version": 1,
"snapshot_id": "<SNAPSHOT_ID or 'unknown'>",
"root_fingerprint": "<sha256 of sorted (path, content_sha256) for all source files>",
"status": "complete|partial|empty|failed",
"provider": {
"kind": "local-build|baseline+overlay|remote",
"catalog": "catalog.sqlite",
"backend_versions": {
"cpp": {"backend_name": "tree-sitter", "backend_version": "0.20.8", "precision": "ast"},
"go": {"backend_name": "scip-clangd", "backend_version": "0.2.3", "precision": "semantic"}
}
},
"units": {"total": 0, "reused": 0, "rebuild": 0, "failed": 0},
"coverage": {"total_files": 0, "indexed_files": 0, "failed_files": 0, "deferred_files": 0},
"shards": [{"id": "", "path": "", "checksum": "", "partition_key": "", "symbol_count": 0, "edge_count": 0}],
"deferred_units": [{"unit_id": "", "language": "", "files": [], "priority": 4, "reason": ""}],
"native_indexes": [{"kind": "scip", "path": "", "snapshot_id": "", "root_fingerprint": "", "language": "", "indexer": "", "precision": ""}],
"baseline": {"source": "ci|local|none", "snapshot_id": "", "manifest_path": ""},
"overlay": {"units_added": 0, "units_modified": 0, "files": []},
"compat_pointer": {"path": "structural_index.jsonl", "full_export": true, "symbol_count": 0},
"created_at": "<ISO 8601>",
"build_duration_ms": 0
}Atomic commit: The manifest is written LAST (atomic rename from ).
Interrupted builds leave unreferenced temp objects without corrupting the last
published index.
tmp/json
{
"schema_version": 1,
"snapshot_id": "<SNAPSHOT_ID or 'unknown'>",
"root_fingerprint": "<sha256 of sorted (path, content_sha256) for all source files>",
"status": "complete|partial|empty|failed",
"provider": {
"kind": "local-build|baseline+overlay|remote",
"catalog": "catalog.sqlite",
"backend_versions": {
"cpp": {"backend_name": "tree-sitter", "backend_version": "0.20.8", "precision": "ast"},
"go": {"backend_name": "scip-clangd", "backend_version": "0.2.3", "precision": "semantic"}
}
},
"units": {"total": 0, "reused": 0, "rebuild": 0, "failed": 0},
"coverage": {"total_files": 0, "indexed_files": 0, "failed_files": 0, "deferred_files": 0},
"shards": [{"id": "", "path": "", "checksum": "", "partition_key": "", "symbol_count": 0, "edge_count": 0}],
"deferred_units": [{"unit_id": "", "language": "", "files": [], "priority": 4, "reason": ""}],
"native_indexes": [{"kind": "scip", "path": "", "snapshot_id": "", "root_fingerprint": "", "language": "", "indexer": "", "precision": ""}],
"baseline": {"source": "ci|local|none", "snapshot_id": "", "manifest_path": ""},
"overlay": {"units_added": 0, "units_modified": 0, "files": []},
"compat_pointer": {"path": "structural_index.jsonl", "full_export": true, "symbol_count": 0},
"created_at": "<ISO 8601>",
"build_duration_ms": 0
}原子提交: 清单最后写入(从原子重命名)。中断的构建会留下未引用的临时对象,但不会损坏最后发布的索引。
tmp/SQLite Catalog Schema (catalog.sqlite
)
catalog.sqliteSQLite目录架构(catalog.sqlite
)
catalog.sqlitesql
CREATE TABLE IF NOT EXISTS schema_meta (
key TEXT PRIMARY KEY, value TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS symbols (
symbol_id TEXT PRIMARY KEY,
name TEXT NOT NULL,
qualified_name TEXT NOT NULL,
namespace TEXT,
language TEXT NOT NULL,
file_path TEXT NOT NULL,
start_line INTEGER NOT NULL,
end_line INTEGER,
kind TEXT NOT NULL,
signature TEXT,
backend TEXT NOT NULL,
precision TEXT NOT NULL CHECK (precision IN ('semantic','typecheck','ast','symbol-only','heuristic','deferred','coverage-only')),
corpus TEXT DEFAULT 'default',
partition_key TEXT,
unit_cache_key TEXT,
source_layer TEXT NOT NULL DEFAULT 'baseline'
);
CREATE TABLE IF NOT EXISTS call_edges (
edge_id INTEGER PRIMARY KEY AUTOINCREMENT,
caller_id TEXT NOT NULL,
callee_id TEXT,
callee_name TEXT NOT NULL,
file_path TEXT NOT NULL,
line INTEGER NOT NULL,
edge_kind TEXT NOT NULL CHECK (edge_kind IN ('direct','indirect','virtual','macro','unresolved')),
corpus TEXT DEFAULT 'default',
partition_key TEXT,
source_layer TEXT NOT NULL DEFAULT 'baseline',
FOREIGN KEY (caller_id) REFERENCES symbols(symbol_id),
FOREIGN KEY (callee_id) REFERENCES symbols(symbol_id)
);
CREATE TABLE IF NOT EXISTS function_boundaries (
symbol_id TEXT PRIMARY KEY,
file_path TEXT NOT NULL,
start_line INTEGER NOT NULL,
end_line INTEGER NOT NULL,
signature TEXT,
language TEXT NOT NULL,
FOREIGN KEY (symbol_id) REFERENCES symbols(symbol_id)
);
CREATE TABLE IF NOT EXISTS coverage (
file_path TEXT NOT NULL,
indexed INTEGER NOT NULL DEFAULT 0,
backend TEXT,
precision TEXT,
unit_cache_key TEXT,
status TEXT NOT NULL DEFAULT 'pending',
corpus TEXT DEFAULT 'default',
partition_key TEXT,
PRIMARY KEY (file_path, corpus)
);
CREATE TABLE IF NOT EXISTS unit_cache (
cache_key TEXT PRIMARY KEY,
unit_id TEXT NOT NULL,
language TEXT NOT NULL,
extractor_name TEXT NOT NULL,
extractor_version TEXT NOT NULL,
file_count INTEGER NOT NULL,
symbol_count INTEGER NOT NULL,
edge_count INTEGER NOT NULL,
source_bytes INTEGER NOT NULL,
created_at TEXT NOT NULL,
snapshot_id TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS deferred_units (
unit_id TEXT PRIMARY KEY,
language TEXT NOT NULL,
files TEXT NOT NULL,
priority INTEGER NOT NULL,
reason TEXT NOT NULL,
cache_key TEXT,
created_at TEXT NOT NULL
);
-- Indexes: both directions!
CREATE INDEX IF NOT EXISTS idx_symbols_name ON symbols(name);
CREATE INDEX IF NOT EXISTS idx_symbols_qualified ON symbols(qualified_name);
CREATE INDEX IF NOT EXISTS idx_symbols_file ON symbols(file_path, start_line);
CREATE INDEX IF NOT EXISTS idx_symbols_lang ON symbols(language);
CREATE INDEX IF NOT EXISTS idx_edges_caller ON call_edges(caller_id);
CREATE INDEX IF NOT EXISTS idx_edges_callee ON call_edges(callee_id);
CREATE INDEX IF NOT EXISTS idx_edges_callee_name ON call_edges(callee_name);
CREATE INDEX IF NOT EXISTS idx_edges_file ON call_edges(file_path, line);
CREATE INDEX IF NOT EXISTS idx_boundaries_file ON function_boundaries(file_path, start_line, end_line);
CREATE INDEX IF NOT EXISTS idx_coverage_status ON coverage(status);
CREATE INDEX IF NOT EXISTS idx_deferred_priority ON deferred_units(priority);Partitioning: When symbols exceed 500K or edges exceed 2M, partition into
shard SQLite files under by (stable
hash). Each shard has the same schema. The query helper routes queries to the
correct shard(s) based on the manifest.
shards/lang:{language}:bucket:{NN}sql
CREATE TABLE IF NOT EXISTS schema_meta (
key TEXT PRIMARY KEY, value TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS symbols (
symbol_id TEXT PRIMARY KEY,
name TEXT NOT NULL,
qualified_name TEXT NOT NULL,
namespace TEXT,
language TEXT NOT NULL,
file_path TEXT NOT NULL,
start_line INTEGER NOT NULL,
end_line INTEGER,
kind TEXT NOT NULL,
signature TEXT,
backend TEXT NOT NULL,
precision TEXT NOT NULL CHECK (precision IN ('semantic','typecheck','ast','symbol-only','heuristic','deferred','coverage-only')),
corpus TEXT DEFAULT 'default',
partition_key TEXT,
unit_cache_key TEXT,
source_layer TEXT NOT NULL DEFAULT 'baseline'
);
CREATE TABLE IF NOT EXISTS call_edges (
edge_id INTEGER PRIMARY KEY AUTOINCREMENT,
caller_id TEXT NOT NULL,
callee_id TEXT,
callee_name TEXT NOT NULL,
file_path TEXT NOT NULL,
line INTEGER NOT NULL,
edge_kind TEXT NOT NULL CHECK (edge_kind IN ('direct','indirect','virtual','macro','unresolved')),
corpus TEXT DEFAULT 'default',
partition_key TEXT,
source_layer TEXT NOT NULL DEFAULT 'baseline',
FOREIGN KEY (caller_id) REFERENCES symbols(symbol_id),
FOREIGN KEY (callee_id) REFERENCES symbols(symbol_id)
);
CREATE TABLE IF NOT EXISTS function_boundaries (
symbol_id TEXT PRIMARY KEY,
file_path TEXT NOT NULL,
start_line INTEGER NOT NULL,
end_line INTEGER NOT NULL,
signature TEXT,
language TEXT NOT NULL,
FOREIGN KEY (symbol_id) REFERENCES symbols(symbol_id)
);
CREATE TABLE IF NOT EXISTS coverage (
file_path TEXT NOT NULL,
indexed INTEGER NOT NULL DEFAULT 0,
backend TEXT,
precision TEXT,
unit_cache_key TEXT,
status TEXT NOT NULL DEFAULT 'pending',
corpus TEXT DEFAULT 'default',
partition_key TEXT,
PRIMARY KEY (file_path, corpus)
);
CREATE TABLE IF NOT EXISTS unit_cache (
cache_key TEXT PRIMARY KEY,
unit_id TEXT NOT NULL,
language TEXT NOT NULL,
extractor_name TEXT NOT NULL,
extractor_version TEXT NOT NULL,
file_count INTEGER NOT NULL,
symbol_count INTEGER NOT NULL,
edge_count INTEGER NOT NULL,
source_bytes INTEGER NOT NULL,
created_at TEXT NOT NULL,
snapshot_id TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS deferred_units (
unit_id TEXT PRIMARY KEY,
language TEXT NOT NULL,
files TEXT NOT NULL,
priority INTEGER NOT NULL,
reason TEXT NOT NULL,
cache_key TEXT,
created_at TEXT NOT NULL
);
-- 索引:双向!
CREATE INDEX IF NOT EXISTS idx_symbols_name ON symbols(name);
CREATE INDEX IF NOT EXISTS idx_symbols_qualified ON symbols(qualified_name);
CREATE INDEX IF NOT EXISTS idx_symbols_file ON symbols(file_path, start_line);
CREATE INDEX IF NOT EXISTS idx_symbols_lang ON symbols(language);
CREATE INDEX IF NOT EXISTS idx_edges_caller ON call_edges(caller_id);
CREATE INDEX IF NOT EXISTS idx_edges_callee ON call_edges(callee_id);
CREATE INDEX IF NOT EXISTS idx_edges_callee_name ON call_edges(callee_name);
CREATE INDEX IF NOT EXISTS idx_edges_file ON call_edges(file_path, line);
CREATE INDEX IF NOT EXISTS idx_boundaries_file ON function_boundaries(file_path, start_line, end_line);
CREATE INDEX IF NOT EXISTS idx_coverage_status ON coverage(status);
CREATE INDEX IF NOT EXISTS idx_deferred_priority ON deferred_units(priority);分区: 当符号超过500K或边超过2M时,按(稳定哈希)将数据分区到下的分片SQLite文件中。每个分片具有相同的架构。查询助手根据清单将查询路由到正确的分片。
lang:{language}:bucket:{NN}shards/Content-Addressed Cache Key
基于内容寻址的缓存键
python
cache_key = sha256(
schema_version +
extractor_name@version +
language +
compile_context_digest +
ordered_input_content_digests +
dependency_or_interface_digest
)Critical: is NOT in the cache key. It goes in provenance only.
This permits reuse across commits, branches, and snapshots with identical units.
snapshot_idpython
cache_key = sha256(
schema_version +
extractor_name@version +
language +
compile_context_digest +
ordered_input_content_digests +
dependency_or_interface_digest
)关键: 不在缓存键中。仅在溯源信息中记录。这允许跨提交、分支和具有相同单元的快照复用缓存。
snapshot_idCanonical Symbol IDs
规范符号ID
Native IDs (when semantic backend is available):
- SCIP:
scip:{symbol} - Kythe:
kythe:{uri} - clangd:
clangd:{usr}
Fallback IDs (when no semantic backend):
fallback:{language}:{file_path}:{sha256(qualified_name|start_line|signature)[:16]}This distinguishes namespaces, overloads, methods, local functions, and
duplicate names across languages or repositories.
原生ID(当语义后端可用时):
- SCIP:
scip:{symbol} - Kythe:
kythe:{uri} - clangd:
clangd:{usr}
回退ID(当无语义后端时):
fallback:{language}:{file_path}:{sha256(qualified_name|start_line|signature)[:16]}这可区分不同语言或仓库中的命名空间、重载、方法、本地函数和重复名称。
Edge Kinds
调用边类型
Edge kinds are kept explicitly separate — never silently merged:
- — statically resolved call to a known symbol
direct - — function pointer, closure, callback
indirect - — virtual method dispatch (runtime-resolved)
virtual - — call introduced by macro expansion
macro - — callee name found but no symbol_id resolved
unresolved
调用边类型明确区分——绝不静默合并:
- —— 静态解析到已知符号的调用
direct - —— 函数指针、闭包、回调
indirect - —— 虚方法调度(运行时解析)
virtual - —— 宏展开引入的调用
macro - —— 找到被调用者名称但未解析到symbol_id
unresolved
Per-Language Semantic Units
按语言划分的语义单元
| Language | Unit Type | Compile Context | Dependency Digest |
|---|---|---|---|
| C/C++ | Compilation unit | compile_commands.json entry | sha256 of transitive header interfaces |
| Go | Package | go.mod + build tags | sha256 of imported packages' exported API |
| Rust | Crate | Cargo.toml + features | sha256 of extern crate signatures |
| Java/Kotlin | Compiler batch | module path + classpath | sha256 of imported class signatures |
| TypeScript | Project | tsconfig.json | sha256 of imported module type declarations |
| Fallback | Individual file | None | Empty string |
Fallback Granularity Rule: For the fallback tier (regex, AST parser, or
lightweight crawler without a language build system), each source file MUST be
its own independent semantic unit. Never bundle multiple source files into a
single fallback unit. Per-file unit isolation is essential for content-addressed
cache efficiency — when one file changes in a future snapshot, only that single
file's unit invalidates while all unchanged files hit the cache
().
reused = N-1| 语言 | 单元类型 | 编译上下文 | 依赖摘要 |
|---|---|---|---|
| C/C++ | 编译单元 | compile_commands.json条目 | 传递性头文件接口的sha256值 |
| Go | 包 | go.mod + 构建标签 | 导入包导出API的sha256值 |
| Rust | Crate | Cargo.toml + 特性 | 外部crate签名的sha256值 |
| Java/Kotlin | 编译器批次 | 模块路径 + 类路径 | 导入类签名的sha256值 |
| TypeScript | 项目 | tsconfig.json | 导入模块类型声明的sha256值 |
| 回退 | 单个文件 | 无 | 空字符串 |
回退粒度规则: 对于回退层级(正则、AST解析器或无语言构建系统的轻量级爬虫),每个源代码文件必须作为独立的语义单元。绝不能将多个源代码文件捆绑到单个回退单元中。按文件划分单元的隔离性对于基于内容寻址的缓存效率至关重要——当未来快照中某个文件更改时,仅该文件的单元失效,所有未更改的文件均可命中缓存()。
reused = N-1Baseline + Delta Overlay
基线+增量覆盖
- Static baseline: Built once in CI for the complete snapshot.
- Incremental cache: Immutable unit outputs reused across snapshots.
- Delta overlay: Changed/target files indexed locally, layered over baseline.
- Remote serving: Optional when global index is too costly locally.
At query time, the query helper merges baseline + overlay results. Overlay
symbols supersede baseline symbols for the same file. The manifest records
with baseline manifest reference and
overlay unit count.
provider.kind = "baseline+overlay"- 静态基线: 在CI中为完整快照构建一次。
- 增量缓存: 不可变单元输出可跨快照复用。
- 增量覆盖: 本地索引已更改/目标文件,叠加在基线上。
- 远程服务: 当全局索引本地成本过高时可选。
查询时,查询助手会合并基线+覆盖结果。覆盖符号会替代基线中同一文件的符号。清单记录,包含基线清单引用和覆盖单元数量。
provider.kind = "baseline+overlay"Compatibility JSONL Export
兼容JSONL导出
The old format is retained as a compatibility export
only — it is NOT the query contract. Below the threshold (default 10K records),
a complete JSONL export is emitted. Above the threshold, only a provenance
pointer is written. Large consumers MUST use the query interface.
structural_index.jsonlLine 1 — provenance header:
json
{"_provenance": true, "snapshot_id": "abc123", "tool": "tree-sitter"}Lines 2+ — structural records (one per line, discriminator):
_typejson
{"_type": "function", "key": "src/parser.c:parse_input", "start_line": 45, "end_line": 120, "signature": "int parse_input(char *buf, size_t len)", "calls": ["malloc", "validate_input", "memcpy"]}
{"_type": "call_edge", "caller": "parse_input", "callee": "malloc", "file": "src/parser.c", "line": 78}
{"_type": "call_edge", "caller": "main", "callee": "parse_input", "file": "src/main.c", "line": 203}Provenance header fields:
| Field | Type | Description |
|---|---|---|
| bool | Always |
| string | |
| string | Backend used (e.g. |
Record types:
| Description | Key fields |
|---|---|---|
| Function definition with boundary and signature | |
| A call from caller to callee at file:line | |
旧的格式仅作为兼容导出保留——它不是查询约定。低于阈值(默认10000条记录)时,导出完整的JSONL内容。高于阈值时,仅写入溯源指针。大型消费者必须使用查询接口。
structural_index.jsonl第1行——溯源头:
json
{"_provenance": true, "snapshot_id": "abc123", "tool": "tree-sitter"}第2行及以后——结构记录(每行一条,作为区分符):
_typejson
{"_type": "function", "key": "src/parser.c:parse_input", "start_line": 45, "end_line": 120, "signature": "int parse_input(char *buf, size_t len)", "calls": ["malloc", "validate_input", "memcpy"]}
{"_type": "call_edge", "caller": "parse_input", "callee": "malloc", "file": "src/parser.c", "line": 78}
{"_type": "call_edge", "caller": "main", "callee": "parse_input", "file": "src/main.c", "line": 203}溯源头字段:
| 字段 | 类型 | 描述 |
|---|---|---|
| bool | 始终为 |
| string | 构建索引所针对的 |
| string | 使用的后端(例如 |
记录类型:
| 描述 | 关键字段 |
|---|---|---|
| 包含边界和签名的函数定义 | |
| 文件某一行中调用者到被调用者的调用关系 | |
Snapshot Safety
快照安全性
- Build from , not live tree (when pinned). When the snapshot is pinned, the structural index is built from the pinned
CODE_ROOT, ensuring it reflects the exact bytes the pipeline is analyzing.CODE_ROOT - Reuse-on-match. If already carries the current
manifest.json(andSNAPSHOT_IDis NOTSNAPSHOT_ID— in MODE-OFF, always rebuild), reuse it — do not rebuild. Rebuild when"unknown"differs or isSNAPSHOT_ID. Individual units may still be reused from the content-addressed cache. This bounds cost across retries/crash-resume."unknown" - Manifest atomicity. The manifest is written LAST via atomic rename from
. Interrupted builds leave unreferenced temp objects without corrupting the last published index. On resume, check
tmp/status andmanifest.json.snapshot_id - STALE flag in HALT mode. When is false (HALT), the index may be built from the unpinned
snapshot_pinnedbut is marked as potentially stale. Consumers treat structural hints as advisory.CODE_ROOT - MODE-OFF: build, do not skip. When is absent (MODE-OFF), build against the current directory (cwd) with provenance
active_snapshotset tosnapshot_id. This is the standalone-efficiency case — the index is still useful for the researcher/planner even without snapshot pinning. Do NOT return an empty index merely because the snapshot is absent."unknown"
- 从构建,而非实时代码树(当快照固定时)。 当快照固定时,结构索引从固定的
CODE_ROOT构建,确保其反映流水线正在分析的精确字节。CODE_ROOT - 匹配时复用。 若已包含当前
manifest.json(且SNAPSHOT_ID不为SNAPSHOT_ID——MODE-OFF模式下始终重建),则复用该索引——不重建。当"unknown"不同或为SNAPSHOT_ID时重建。单个单元仍可从基于内容寻址的缓存中复用。这可限制重试/崩溃恢复场景下的成本。"unknown" - 清单原子性。 清单最后通过从原子重命名写入。中断的构建会留下未引用的临时对象,但不会损坏最后发布的索引。恢复时,检查
tmp/的状态和manifest.json。snapshot_id - HALT模式下的STALE标记。 当为false(HALT)时,索引可能从未固定的
snapshot_pinned构建,但会标记为可能过期。消费者将结构提示视为建议性内容。CODE_ROOT - MODE-OFF:构建,不跳过。 当缺失(MODE-OFF)时,针对当前目录(cwd)构建,溯源信息中的
active_snapshot设为snapshot_id。这是独立高效的使用场景——即使没有快照固定,索引对研究者/规划器仍然有用。请勿仅因快照缺失而返回空索引。"unknown"
Consumption Contract
消费约定
These are runtime instructions for callers — they define how consumers use the
structural index. The structural index is a HINT-only enhancement; skills that
do not use it behave exactly as they do today.
这些是针对调用者的运行时指令——定义了消费者如何使用结构索引。结构索引仅作为提示性增强;不使用它的技能行为与当前完全一致。
Query Interface
查询接口
The query helper IS the contract. All consumers use
(or a compatible remote endpoint). The JSONL file is
NOT the query contract — it is a compatibility export only.
query_structural_index.pyOperations:
-
→
resolve_symbol(name, language?, file?, namespace?){results, total, ambiguous, coverage}- Returns ALL matches — no silent selection among ambiguous symbols.
- Caller MUST disambiguate before calling /
find_callers.find_callees
-
→
find_callers(symbol_id, limit=100, offset=0){results, total, has_more, coverage}- Bounded, paginated caller lookup.
- Each result carries ,
precision,backend.edge_kind - Empty results carry .
coverage.partition_status
-
→
find_callees(symbol_id, limit=100, offset=0){results, total, has_more, coverage}- Same shape as .
find_callers
- Same shape as
-
→
get_function_boundary(file, line){symbol_id, start_line, end_line, signature, precision, backend} -
→
get_coverage(file?){total_files, indexed_files, failed, deferred, partition_status, backends_used}
Key properties:
- Bounded results and pagination (prevents loading entire index into memory).
- Explicit name resolution before graph traversal (no silent selection among ambiguous symbols).
- Precision and backend attached to every result.
- Coverage attached to empty results.
- Identical operations for local and remote providers (remote via
env var or manifest
MANTIS_STRUCTURAL_INDEX_URL).provider.kind = "remote"
Coverage on empty results:
| Meaning | Consumer action |
|---|---|---|
| All files indexed | "No indexed callers" (still run grep per HINT-only rule) |
| Some files deferred or failed | "Not fully indexed" — MUST run grep fallback |
| No backend available | "Not indexed" — MUST run grep fallback |
| Backend attempted but failed | "Index failed" — MUST run grep fallback |
查询助手是约定的核心。 所有消费者使用(或兼容的远程端点)。JSONL文件不是查询约定——它仅作为兼容导出。
query_structural_index.py操作:
-
→
resolve_symbol(name, language?, file?, namespace?){results, total, ambiguous, coverage}- 返回所有匹配结果——绝不静默选择歧义符号。
- 调用者在调用/
find_callers之前必须消除歧义。find_callees
-
→
find_callers(symbol_id, limit=100, offset=0){results, total, has_more, coverage}- 受限的分页调用者查找。
- 每个结果包含、
precision、backend。edge_kind - 空结果包含。
coverage.partition_status
-
→
find_callees(symbol_id, limit=100, offset=0){results, total, has_more, coverage}- 与格式相同。
find_callers
- 与
-
→
get_function_boundary(file, line){symbol_id, start_line, end_line, signature, precision, backend} -
→
get_coverage(file?){total_files, indexed_files, failed, deferred, partition_status, backends_used}
关键特性:
- 受限结果和分页(防止将整个索引加载到内存中)。
- 图遍历前明确的名称解析(绝不静默选择歧义符号)。
- 每个结果附带精度和后端信息。
- 空结果附带覆盖情况。
- 本地和远程提供者的操作完全一致(远程通过环境变量或清单
MANTIS_STRUCTURAL_INDEX_URL实现)。provider.kind = "remote"
空结果的覆盖情况:
| 含义 | 消费者操作 |
|---|---|---|
| 所有文件已索引 | “无索引调用者”(仍需按仅提示规则运行grep) |
| 部分文件延迟或失败 | “未完全索引”——必须运行grep回退 |
| 无可用后端 | “未索引”——必须运行grep回退 |
| 后端尝试但失败 | “索引失败”——必须运行grep回退 |
mantis-plan
mantis-plan
- Use the structural index query helper for function-level dependency fan-out.
When planning investigations, call then
resolve_symbol()to identify all functions that call into a target — this broadens the audit set beyond single-file analysis.find_callers() - The structural index decides ORDER of investigations (which functions to audit first based on call-graph centrality), never MEMBERSHIP. It may broaden the audit set (safe over-reporting), but must never REMOVE or drop a file; the planner's existing logic remains the membership floor.
- 使用结构索引查询助手进行函数级依赖扩散。规划调查时,调用然后
resolve_symbol()以识别所有调用目标函数的函数——这将审计范围扩展到单文件分析之外。find_callers() - 结构索引决定调查顺序(基于调用图中心性优先审计哪些函数),但绝不决定审计范围。它可能扩大审计范围(安全的过度报告),但绝不能移除或丢弃任何文件;规划器的现有逻辑仍是审计范围的下限。
mantis-researcher
mantis-researcher
- Wave 1 (Rapid Triage): Run a repo-wide grep for the function name to build
the exhaustive set of candidate call-sites — this is the mandatory floor. Then
use the structural index query helper (then
resolve_symbol) to RANK and prioritize which call-sites to audit first (the index distinguishes actual calls from comments/strings/variable names). Audit the union of both result sets — the structural index may miss macro-based calls, function pointers, and dynamic dispatch, so grep remains the floor.find_callers - Wave 2 (Deep Audit): Use to start with the enclosing function, expanding to callers/callees/file as needed for cross-function context — this saves context while preserving coverage.
get_function_boundary(file, line) - If the structural index is absent or empty, fall back to grep-based discovery (today's behavior). The structural index is a coverage HINT only — it improves audit quality but is never required.
- 第一阶段(快速分类): 对函数名称进行仓库级grep以构建完整的候选调用站点集合——这是强制下限。然后使用结构索引查询助手(然后
resolve_symbol)对调用站点进行排序和优先级划分(索引可区分实际调用与注释/字符串/变量名)。审计两个结果集的并集——索引可能遗漏基于宏的调用、函数指针和动态调度,因此grep仍是下限。find_callers - 第二阶段(深度审计): 使用从包含的函数开始,根据需要扩展到调用者/被调用者/文件以获取跨函数上下文——这在保留覆盖范围的同时节省上下文。
get_function_boundary(file, line) - 若结构索引缺失或为空,回退到基于grep的发现(当前行为)。结构索引仅作为覆盖提示——它提升审计质量但绝非必需。
Safety
安全性
Non-negotiable invariants:
- Agnostic. Nothing is ever required; no-tool / parse-fail / not-invoked → empty index → grep fallback = today's behavior byte-for-byte. Optional in the Pass Lifecycle Contract; a non-conformant harness simply skips it.
- HINT-only / union / never MEMBERSHIP. Consumers audit the union with
grep; a symbol the index misses must still be reachable by the exhaustive
sweep. The structural index decides ORDER, never MEMBERSHIP. The query
contract enforces this: returns HINTs, never authoritative membership — consumers MUST union with grep.
find_callers - No verdicts, touches no findings. Cannot violate INV-1 and cannot itself drop a finding.
- Narrow scope: source cross-reference only. Binary/build-derived reachability ("is it compiled into production") is explicitly out of scope — a dev customization, not part of this skill, because absence-from-a-build can hide a real finding (INV-2). Do not fold build-derived reachability into this skill.
A reference blueprint is available at
mantis-pipeline-adapter/references/mantis-structural-index.md.
It is a stub that points to this SKILL.md as the single source of truth — do not
duplicate spec content there.
不可协商的不变量:
- 无强制要求。 绝不强制要求任何内容;无工具/解析失败/未调用→空索引→grep回退=与当前行为完全一致。在Pass Lifecycle Contract中是可选的;不符合要求的harness可直接跳过它。
- 仅提示/并集/绝不决定范围。 消费者审计与grep结果的并集;索引遗漏的符号仍必须可通过 exhaustive sweep 访问。结构索引决定顺序,绝不决定范围。查询约定强制执行此规则:返回提示,绝非权威范围——消费者必须与grep结果取并集。
find_callers - 无结论,不触及发现结果。 不能违反INV-1,也不能自行丢弃任何发现结果。
- 范围狭窄:仅源代码交叉引用。 二进制/构建派生的可达性(“是否编译到生产环境中”)明确超出范围——这是开发者自定义内容,不属于该技能,因为构建中缺失可能隐藏真实发现结果(INV-2)。请勿将构建派生的可达性纳入该技能。
参考蓝图可在mantis-pipeline-adapter/references/mantis-structural-index.md获取。它是一个存根,指向本SKILL.md作为唯一的事实来源——请勿在那里复制规范内容。