braintrust-tracing

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Braintrust Tracing for Claude Code

面向Claude Code的Braintrust追踪

Comprehensive guide to tracing Claude Code sessions in Braintrust, including sub-agent correlation.

本文是一份在Braintrust中追踪Claude Code会话的综合指南，包含子Agent关联相关内容。

Architecture Overview

架构概述

                         PARENT SESSION
                    +---------------------+
                    |  SessionStart       |
                    |  (creates root)     |
                    +----------+----------+
                               |
                    +----------v----------+
                    |  UserPromptSubmit   |
                    |  (creates Turn)     |
                    +----------+----------+
                               |
          +--------------------+--------------------+
          |                    |                    |
+---------v--------+  +--------v--------+  +--------v--------+
| PostToolUse      |  | PostToolUse     |  | PreToolUse      |
| (Read span)      |  | (Edit span)     |  | (Task - inject) |
+------------------+  +-----------------+  +--------+--------+
                                                    |
                                         +----------v----------+
                                         |   SUB-AGENT         |
                                         |   SessionStart      |
                                         |   (NEW root_span_id)|
                                         +----------+----------+
                                                    |
                                         +----------v----------+
                                         |   SubagentStop      |
                                         |   (has session_id)  |
                                         +---------------------+

                         PARENT SESSION
                    +---------------------+
                    |  SessionStart       |
                    |  (creates root)     |
                    +----------+----------+
                               |
                    +----------v----------+
                    |  UserPromptSubmit   |
                    |  (creates Turn)     |
                    +----------+----------+
                               |
          +--------------------+--------------------+
          |                    |                    |
+---------v--------+  +--------v--------+  +--------v--------+
| PostToolUse      |  | PostToolUse     |  | PreToolUse      |
| (Read span)      |  | (Edit span)     |  | (Task - inject) |
+------------------+  +-----------------+  +--------+--------+
                                                    |
                                         +----------v----------+
                                         |   SUB-AGENT         |
                                         |   SessionStart      |
                                         |   (NEW root_span_id)|
                                         +----------+----------+
                                                    |
                                         +----------v----------+
                                         |   SubagentStop      |
                                         |   (has session_id)  |
                                         +---------------------+

Hook Event Flow

钩子事件流

Hook	Trigger	Creates	Key Fields
SessionStart	Session begins	Root span	`session_id` , `root_span_id`
UserPromptSubmit	User sends prompt	Turn span	`prompt` , `turn_number`
PreToolUse	Before tool runs	(modifies Task prompts)	`tool_input.prompt`
PostToolUse	After tool runs	Tool span	`tool_name` , `input` , `output`
Stop	Turn completes	LLM spans	`model` , `tokens` , `tool_calls`
SubagentStop	Sub-agent finishes	(no span)	`session_id` of sub-agent
SessionEnd	Session ends	(finalizes root)	`turn_count` , `tool_count`

钩子	触发条件	创建内容	关键字段
SessionStart	会话开始	根Span	`session_id` , `root_span_id`
UserPromptSubmit	用户发送提示词	Turn Span	`prompt` , `turn_number`
PreToolUse	工具运行前	（修改Task提示词）	`tool_input.prompt`
PostToolUse	工具运行后	工具Span	`tool_name` , `input` , `output`
Stop	Turn完成	LLM Span	`model` , `tokens` , `tool_calls`
SubagentStop	子Agent结束	（无Span）	子Agent的 `session_id`
SessionEnd	会话结束	（最终化根Span）	`turn_count` , `tool_count`

Trace Hierarchy

追踪层级

Session (task span) - root_span_id = session_id
|
+-- Turn 1 (task span)
|   |
|   +-- claude-sonnet (llm span) - model call with tool_use
|   +-- Read (tool span)
|   +-- Edit (tool span)
|   +-- claude-sonnet (llm span) - response after tools
|
+-- Turn 2 (task span)
|   |
|   +-- claude-sonnet (llm span)
|   +-- Task (tool span) -----> [Sub-agent session - SEPARATE trace]
|   +-- claude-sonnet (llm span)
|
+-- Turn 3 ...

Session (task span) - root_span_id = session_id
|
+-- Turn 1 (task span)
|   |
|   +-- claude-sonnet (llm span) - 调用工具的模型请求
|   +-- Read (tool span)
|   +-- Edit (tool span)
|   +-- claude-sonnet (llm span) - 工具调用后的响应
|
+-- Turn 2 (task span)
|   |
|   +-- claude-sonnet (llm span)
|   +-- Task (tool span) -----> [子Agent会话 - 独立追踪]
|   +-- claude-sonnet (llm span)
|
+-- Turn 3 ...

Sub-Agent Tracing: What Works and What Doesn't

子Agent追踪：可行与不可行方案

What Doesn't Work

不可行方案

SessionStart doesn't receive the Task prompt.

We tried injecting trace context into Task prompts via PreToolUse:

bash

undefined

SessionStart无法接收Task提示词

我们尝试通过PreToolUse钩子将追踪上下文注入到Task提示词中：

bash

undefined

PreToolUse hook injects:

PreToolUse钩子注入内容：

[BRAINTRUST_TRACE_CONTEXT] {"root_span_id": "abc", "parent_span_id": "xyz", "project_id": "123"} [/BRAINTRUST_TRACE_CONTEXT]


But SessionStart only receives session metadata, not the modified prompt. The injected context is lost.

[BRAINTRUST_TRACE_CONTEXT] {"root_span_id": "abc", "parent_span_id": "xyz", "project_id": "123"} [/BRAINTRUST_TRACE_CONTEXT]


但SessionStart仅接收会话元数据，无法获取修改后的提示词，注入的上下文会丢失。

What DOES Work

可行方案

Task spans in parent session contain everything:

```
agentId
```
- identifier for the sub-agent run
```
totalTokens
```
,
```
totalToolUseCount
```
- metrics
```
content
```
- full agent response/summary
```
tool_input.prompt
```
- original task prompt
```
tool_input.subagent_type
```
- agent type (e.g., "oracle")

SubagentStop hook receives the sub-agent's
session_id
:

This equals the sub-agent's orphaned trace
```
root_span_id
```
Allows correlation between parent Task span and child trace

父会话中的Task Span包含所有必要信息：

```
agentId
```
- 子Agent运行的标识符
```
totalTokens
```
,
```
totalToolUseCount
```
- 指标数据
```
content
```
- 完整的Agent响应/摘要
```
tool_input.prompt
```
- 原始任务提示词
```
tool_input.subagent_type
```
- Agent类型（例如"oracle"）

SubagentStop钩子接收子Agent的
session_id
：

该ID等于子Agent独立追踪的
```
root_span_id
```
可用于关联父Task Span与子追踪

The Correlation Pattern

关联模式

Current state: Sub-agents create orphaned traces (new

root_span_id

Correlation method:

Query parent session's Task spans for agent metadata
Match
```
agentId
```
or timing with orphaned traces
Sub-agent's
```
session_id
```
= its trace's
```
root_span_id
```

Future solution (not yet implemented):

SubagentStop fires -> writes session_id to temp file
PostToolUse (Task) -> reads temp file -> adds child_session_id to Task span metadata

This would link:

Task.agentId

Task.child_session_id

-> orphaned trace

root_span_id

当前状态： 子Agent会创建独立的追踪（新的

root_span_id

）。

关联方法：

查询父会话的Task Span获取Agent元数据
通过
```
agentId
```
或时间匹配独立追踪
子Agent的
```
session_id
```
= 其追踪的
```
root_span_id
```

未来解决方案（尚未实现）：

SubagentStop触发 -> 将session_id写入临时文件
PostToolUse (Task) -> 读取临时文件 -> 将child_session_id添加到Task Span元数据

这将建立关联：

Task.agentId

Task.child_session_id

-> 独立追踪的

root_span_id

State Management

状态管理

Per-Session State Files

会话级状态文件

~/.claude/state/braintrust_sessions/
  {session_id}.json       # Per-session state

Each session file contains:

json

{
  "root_span_id": "abc-123",
  "project_id": "proj-456",
  "turn_count": 5,
  "tool_count": 23,
  "current_turn_span_id": "turn-789",
  "current_turn_start": 1703456789,
  "started": "2025-12-24T10:00:00.000Z",
  "is_subagent": false
}

~/.claude/state/braintrust_sessions/
  {session_id}.json       # 会话级状态

每个会话文件包含：

json

{
  "root_span_id": "abc-123",
  "project_id": "proj-456",
  "turn_count": 5,
  "tool_count": 23,
  "current_turn_span_id": "turn-789",
  "current_turn_start": 1703456789,
  "started": "2025-12-24T10:00:00.000Z",
  "is_subagent": false
}

Global State

全局状态

~/.claude/state/braintrust_global.json   # Cached project_id
~/.claude/state/braintrust_hook.log      # Debug log

~/.claude/state/braintrust_global.json   # 缓存的project_id
~/.claude/state/braintrust_hook.log      # 调试日志

Debugging Commands

调试命令

Check if Tracing is Active

检查追踪是否激活

bash

undefined

bash

undefined

View hook logs in real-time

实时查看钩子日志

tail -f ~/.claude/state/braintrust_hook.log

Check if session has state

检查会话是否有状态

cat ~/.claude/state/braintrust_sessions/*.json | jq -s '.'

Verify environment

验证环境变量

echo "TRACE_TO_BRAINTRUST=$TRACE_TO_BRAINTRUST" echo "BRAINTRUST_API_KEY=${BRAINTRUST_API_KEY:+set}"

undefined

echo "TRACE_TO_BRAINTRUST=$TRACE_TO_BRAINTRUST" echo "BRAINTRUST_API_KEY=${BRAINTRUST_API_KEY:+set}"

undefined

Query Braintrust Directly

直接查询Braintrust

bash

undefined

bash

undefined

List recent sessions

列出最近的会话

uv run python -m runtime.harness scripts/braintrust_analyze.py --sessions 5

Analyze last session

分析最后一个会话

uv run python -m runtime.harness scripts/braintrust_analyze.py --last-session

Replay specific session

重放特定会话

uv run python -m runtime.harness scripts/braintrust_analyze.py --replay <session-id>

Find sub-agent traces (orphaned roots)

查找子Agent追踪（独立根Span）

uv run python -m runtime.harness scripts/braintrust_analyze.py --agent-stats

undefined

uv run python -m runtime.harness scripts/braintrust_analyze.py --agent-stats

undefined

Debug Hook Execution

调试钩子执行

bash

undefined

bash

undefined

Enable verbose logging

启用详细日志

export BRAINTRUST_CC_DEBUG=true

Test hooks manually

手动测试钩子

echo '{"session_id":"test-123","type":"resume"}' |
bash "$CLAUDE_PROJECT_DIR/.claude/plugins/braintrust-tracing/hooks/session_start.sh"

Test PreToolUse (Task injection)

测试PreToolUse（Task注入）

echo '{"session_id":"test-123","tool_name":"Task","tool_input":{"prompt":"test"}}' |
bash "$CLAUDE_PROJECT_DIR/.claude/plugins/braintrust-tracing/hooks/pre_tool_use.sh"

undefined

echo '{"session_id":"test-123","tool_name":"Task","tool_input":{"prompt":"test"}}' |
bash "$CLAUDE_PROJECT_DIR/.claude/plugins/braintrust-tracing/hooks/pre_tool_use.sh"

undefined

Troubleshooting Checklist

故障排查清单

No traces appearing:

Check

TRACE_TO_BRAINTRUST=true

.claude/settings.local.json

Verify API key:
```
echo $BRAINTRUST_API_KEY
```

Check logs:

tail -20 ~/.claude/state/braintrust_hook.log

Sub-agents not linking:
- This is expected - sub-agents create orphaned traces
- Use
```
--agent-stats
```
  to find agent activity
- Correlate via timing or
```
agentId
```
  in parent Task span
Missing spans:
- Check
```
current_turn_span_id
```
  in session state
- Ensure Stop hook runs (turn finalization)
- Look for "Failed to create" errors in log

State corruption:

Remove session state:

rm ~/.claude/state/braintrust_sessions/*.json

Clear global cache:

rm ~/.claude/state/braintrust_global.json

无追踪数据显示：

检查

.claude/settings.local.json

中

TRACE_TO_BRAINTRUST=true

是否设置

验证API密钥：
```
echo $BRAINTRUST_API_KEY
```

查看日志：

tail -20 ~/.claude/state/braintrust_hook.log

子Agent未关联：
- 这是预期行为 - 子Agent会创建独立追踪
- 使用
```
--agent-stats
```
  查找Agent活动
- 通过父Task Span中的时间或
```
agentId
```
  进行关联
缺失Span：
- 检查会话状态中的
```
current_turn_span_id
```
- 确保Stop钩子已运行（Turn最终化）
- 查找日志中的“Failed to create”错误

状态损坏：

删除会话状态：

rm ~/.claude/state/braintrust_sessions/*.json

清除全局缓存：

rm ~/.claude/state/braintrust_global.json

Key Files

关键文件

File	Purpose
`.claude/plugins/braintrust-tracing/hooks/common.sh`	Shared utilities, API, state management
`.claude/plugins/braintrust-tracing/hooks/session_start.sh`	Creates root span, handles sub-agent context
`.claude/plugins/braintrust-tracing/hooks/user_prompt_submit.sh`	Creates Turn spans per user message
`.claude/plugins/braintrust-tracing/hooks/pre_tool_use.sh`	Injects trace context into Task prompts
`.claude/plugins/braintrust-tracing/hooks/post_tool_use.sh`	Creates tool spans, captures agent/skill metadata
`.claude/plugins/braintrust-tracing/hooks/stop_hook.sh`	Creates LLM spans, finalizes Turns
`.claude/plugins/braintrust-tracing/hooks/session_end.sh`	Finalizes session, triggers learning extraction
`scripts/braintrust_analyze.py`	Query and analyze traced sessions
`~/.claude/state/braintrust_sessions/`	Per-session state files
`~/.claude/state/braintrust_hook.log`	Debug log

文件	用途
`.claude/plugins/braintrust-tracing/hooks/common.sh`	共享工具、API、状态管理
`.claude/plugins/braintrust-tracing/hooks/session_start.sh`	创建根Span，处理子Agent上下文
`.claude/plugins/braintrust-tracing/hooks/user_prompt_submit.sh`	为每条用户消息创建Turn Span
`.claude/plugins/braintrust-tracing/hooks/pre_tool_use.sh`	将追踪上下文注入Task提示词
`.claude/plugins/braintrust-tracing/hooks/post_tool_use.sh`	创建工具Span，捕获Agent/skill元数据
`.claude/plugins/braintrust-tracing/hooks/stop_hook.sh`	创建LLM Span，完成Turn最终化
`.claude/plugins/braintrust-tracing/hooks/session_end.sh`	完成会话最终化，触发学习内容提取
`scripts/braintrust_analyze.py`	查询并分析已追踪的会话
`~/.claude/state/braintrust_sessions/`	会话级状态文件目录
`~/.claude/state/braintrust_hook.log`	调试日志

Environment Variables

环境变量

Variable	Required	Default	Description
`TRACE_TO_BRAINTRUST`	Yes	-	Set to `"true"` to enable
`BRAINTRUST_API_KEY`	Yes	-	API key for Braintrust
`BRAINTRUST_CC_PROJECT`	No	`claude-code`	Project name
`BRAINTRUST_CC_DEBUG`	No	`false`	Verbose logging
`BRAINTRUST_API_URL`	No	`https://api.braintrust.dev`	API endpoint

变量	是否必填	默认值	描述
`TRACE_TO_BRAINTRUST`	是	-	设置为 `"true"` 以启用追踪
`BRAINTRUST_API_KEY`	是	-	Braintrust的API密钥
`BRAINTRUST_CC_PROJECT`	否	`claude-code`	项目名称
`BRAINTRUST_CC_DEBUG`	否	`false`	启用详细日志
`BRAINTRUST_API_URL`	否	`https://api.braintrust.dev`	API端点

Session Learnings

会话经验总结

What We Learned About Sub-Agent Tracing (Dec 2025)

子Agent追踪的经验总结（2025年12月）

Attempted: Inject trace context via PreToolUse into Task prompts.

Result: Failed - SessionStart only receives session metadata, not the prompt.

Discovery: Task spans already contain rich sub-agent data:

```
metadata.agent_type
```
- agent type from
```
subagent_type
```
```
metadata.skill_name
```
- skill from Skill tool
```
tool_input
```
- full prompt sent to agent
```
tool_output
```
- agent response

Current correlation path:

Parent session Task span has
```
agentId
```
and timing
Sub-agent creates orphaned trace with
```
root_span_id = session_id
```
SubagentStop provides the sub-agent's
```
session_id
```
Manual correlation: match timing or use
```
session_id
```
link

Future work: Write

child_session_id

to Task span metadata from PostToolUse after SubagentStop.

尝试方案： 通过PreToolUse将追踪上下文注入Task提示词。

结果： 失败 - SessionStart仅接收会话元数据，无法获取提示词。

发现： Task Span已包含丰富的子Agent数据：

```
metadata.agent_type
```
- 来自
```
subagent_type
```
的Agent类型
```
metadata.skill_name
```
- 来自Skill工具的技能名称
```
tool_input
```
- 发送给Agent的完整提示词
```
tool_output
```
- Agent的响应内容

当前关联路径：

父会话Task Span包含
```
agentId
```
和时间信息
子Agent创建独立追踪，其
```
root_span_id = session_id
```
SubagentStop提供子Agent的
```
session_id
```
手动关联：通过时间或
```
session_id
```
链接

未来工作： 在SubagentStop触发后，通过PostToolUse将

child_session_id

写入Task Span元数据。

What We Learned About Sub-Agent Correlation

子Agent关联的经验总结

The Problem

问题

Sub-agents spawned via Task tool create orphaned Braintrust traces
Parent session has Task spans with
```
agentId
```
, sub-agent has separate
```
session_id
```
No built-in link between them

通过Task工具启动的子Agent会在Braintrust中创建独立追踪
父会话的Task Span包含
```
agentId
```
，子Agent有独立的
```
session_id
```
两者之间没有内置关联

What DOESN'T Work

不可行方案

1. Prompt injection via PreToolUse

SessionStart hook only receives session metadata (

session_id

type

cwd

), NOT the prompt. Injected trace context is never seen.

The hook receives:

json

{
  "session_id": "...",
  "type": "start|resume|compact|clear",
  "cwd": "...",
  "env": {...}
}

No prompt field exists - context injection is impossible at SessionStart.

2. SubagentStop → PostToolUse file handoff

Race condition. These are independent async hooks with no timing guarantees:

SubagentStop fires when sub-agent session ends
PostToolUse (Task) fires when Task tool completes
No ordering guarantee between them
Writing to a correlation file creates a race

3. PreToolUse correlation files

SessionStart can't access the

task_span_id

because it has no context about which Task spawned it. PreToolUse modifies prompts but doesn't create a reliably accessible state file that SessionStart can find.

1. 通过PreToolUse注入提示词

SessionStart钩子仅接收会话元数据（

session_id

type

cwd

），不接收提示词。注入的追踪上下文永远不会被读取到。

钩子接收的内容如下：

json

{
  "session_id": "...",
  "type": "start|resume|compact|clear",
  "cwd": "...",
  "env": {...}
}

不存在提示词字段 - 在SessionStart阶段无法进行上下文注入。

2. SubagentStop → PostToolUse文件传递

存在竞态条件。这两个是独立的异步钩子，没有时序保证：

SubagentStop在子Agent会话结束时触发
PostToolUse (Task)在Task工具完成时触发
两者之间没有执行顺序保证
写入关联文件会导致竞态问题

3. PreToolUse关联文件

SessionStart无法访问

task_span_id

，因为它不知道是哪个Task启动了它。PreToolUse会修改提示词，但无法创建SessionStart可以可靠访问的状态文件。

What DOES Work

可行方案

Post-hoc matching for dataset building:

Parent session Task spans contain:

```
agentId
```
- identifier for the sub-agent run
```
totalTokens
```
,
```
totalToolUseCount
```
- aggregated metrics
```
content
```
- full agent response/summary
```
tool_input.prompt
```
- original task prompt
```
tool_input.subagent_type
```
- agent type (e.g., "oracle")
Start/end timestamps

Sub-agent sessions contain:

```
session_id
```
(equals orphaned trace
```
root_span_id
```
)
Start/end timestamps
All internal spans and tool calls

Correlation strategy:

Export parent session traces (query parent
```
root_span_id
```
)
Export sub-agent traces (query all sessions created within parent's time window)
Match by:
- Timing: Task span end ≈ sub-agent session end
- Metadata:
```
subagent_type
```
  from Task prompt
- IDs: SubagentStop hook provides
```
session_id
```
  (can be captured and logged)

用于数据集构建的事后匹配：

父会话的Task Span包含：

```
agentId
```
- 子Agent运行的标识符
```
totalTokens
```
,
```
totalToolUseCount
```
- 聚合指标
```
content
```
- 完整的Agent响应/摘要
```
tool_input.prompt
```
- 原始任务提示词
```
tool_input.subagent_type
```
- Agent类型（例如"oracle"）
开始/结束时间戳

子Agent会话包含：

```
session_id
```
（等于独立追踪的
```
root_span_id
```
）
开始/结束时间戳
所有内部Span和工具调用

关联策略：

导出父会话追踪（查询父
```
root_span_id
```
）
导出子Agent追踪（查询父会话时间窗口内创建的所有会话）
通过以下方式匹配：
- 时间：Task Span结束时间 ≈ 子Agent会话结束时间
- 元数据：Task提示词中的
```
subagent_type
```
- ID：SubagentStop钩子提供的
```
session_id
```
  （可捕获并记录）

Architecture Insight

架构洞察

SessionStart input is intentionally minimal - it contains no prompt or tool context:

typescript

interface SessionStartInput {
  session_id: string;
  type: "start" | "resume" | "compact" | "clear";
  cwd: string;
  env: { [key: string]: string };
  // NO: prompt, tool_context, task_span_id, parent_span_id
}

This design boundary prevents real-time correlation at hook time.

SessionStart的输入被有意设计为极简 - 不包含提示词或工具上下文：

typescript

interface SessionStartInput {
  session_id: string;
  type: "start" | "resume" | "compact" | "clear";
  cwd: string;
  env: { [key: string]: string };
  // 无：prompt, tool_context, task_span_id, parent_span_id
}

这个设计边界导致在钩子运行时无法进行实时关联。

Recommendation

建议

For building agent run datasets with sub-agent correlation:

In-session logging: Capture SubagentStop
```
session_id
```
in logs or state
Post-session export: Query Braintrust API for parent and sub-agent traces
Offline correlation: Match traces by timing and metadata in a script
Don't try real-time linking: Hooks don't have necessary context

Example script pattern:

bash

undefined

对于构建包含子Agent关联的Agent运行数据集：

会话内日志： 在日志或状态中捕获SubagentStop的
```
session_id
```
会话后导出： 查询Braintrust API获取父和子Agent追踪
离线关联： 在脚本中通过时间和元数据匹配追踪
不要尝试实时链接： 钩子没有必要的上下文

示例脚本模式：

bash

undefined

1. Export parent session

1. 导出父会话

braintrust_analyze.py --replay <parent-session-id> > parent_traces.json

2. Query for orphaned sub-agent traces (those created during parent's time window)

2. 查询独立子Agent追踪（父会话时间窗口内创建的追踪）

braintrust_analyze.py --agent-stats > all_agent_traces.json

3. Correlate in Python:

3. 在Python中进行关联：

- Parent Task spans -> agentId, timestamps, subagent_type

- 父Task Span -> agentId、时间戳、subagent_type

- Orphaned traces -> root_span_id, timestamps

- 独立追踪 -> root_span_id、时间戳

- Match by timing and type

- 通过时间和类型匹配


This approach is reliable, testable, and doesn't require hooks to maintain implicit state.


这种方法可靠、可测试，且不需要钩子维护隐式状态。