langsmith-trace

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
<oneliner> Two main topics: **adding tracing** to your application, and **querying traces** for debugging and analysis. Python and Javascript implementations are both supported. </oneliner> <setup> Environment Variables
bash
LANGSMITH_API_KEY=lsv2_pt_your_api_key_here          # Required
LANGSMITH_PROJECT=your-project-name                   # Optional: default project
LANGSMITH_WORKSPACE_ID=your-workspace-id              # Optional: for org-scoped keys
IMPORTANT: Always check the environment variables or
.env
file for
LANGSMITH_PROJECT
before querying or interacting with LangSmith. This tells you which project contains the relevant traces and data. If the LangSmith project is not available, use your best judgement to identify the right one.
CLI Tool
bash
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
</setup>
<trace_langchain_oss> For LangChain/LangGraph apps, tracing is automatic. Just set environment variables:
bash
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key>  # or your LLM provider's key
Optional variables:
  • LANGSMITH_PROJECT
    - specify project name (defaults to "default")
  • LANGCHAIN_CALLBACKS_BACKGROUND=false
    - use for serverless to ensure traces complete before function exit (Python) </trace_langchain_oss>
<trace_other_frameworks> For non-LangChain apps, if the framework has native OpenTelemetry support, use LangSmith's OpenTelemetry integration.
If the app is NOT using a framework, or using one without automatic OTel support, use the traceable decorator/wrapper and wrap your LLM client.
<python> Use @traceable decorator and wrap_openai() for automatic tracing. ```python from langsmith import traceable from langsmith.wrappers import wrap_openai from openai import OpenAI
client = wrap_openai(OpenAI())
@traceable def my_llm_pipeline(question: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": question}], ) return resp.choices[0].message.content
<oneliner> 两大核心主题:为你的应用**添加追踪功能**,以及**查询追踪数据**用于调试和分析。同时支持Python和Javascript实现。 </oneliner> <setup> 环境变量
bash
LANGSMITH_API_KEY=lsv2_pt_your_api_key_here          # Required
LANGSMITH_PROJECT=your-project-name                   # Optional: default project
LANGSMITH_WORKSPACE_ID=your-workspace-id              # Optional: for org-scoped keys
重要提示: 在查询或与LangSmith交互前,务必检查环境变量或
.env
文件中的
LANGSMITH_PROJECT
配置。它会告知你相关追踪数据和信息所在的项目。如果LangSmith项目不可用,请根据判断选择合适的项目。
CLI工具
bash
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
</setup>
<trace_langchain_oss> 对于LangChain/LangGraph应用,追踪功能是自动开启的。只需设置以下环境变量:
bash
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key>  # or your LLM provider's key
可选变量:
  • LANGSMITH_PROJECT
    - 指定项目名称(默认值为"default")
  • LANGCHAIN_CALLBACKS_BACKGROUND=false
    - 用于无服务器环境,确保在函数退出前完成追踪(Python) </trace_langchain_oss>
<trace_other_frameworks> 对于非LangChain应用,如果框架原生支持OpenTelemetry,请使用LangSmith的OpenTelemetry集成。 如果应用未使用框架,或使用的框架不支持自动OTel,请使用traceable装饰器/包装器来包裹你的LLM客户端。
<python> 使用@traceable装饰器和wrap_openai()实现自动追踪。 ```python from langsmith import traceable from langsmith.wrappers import wrap_openai from openai import OpenAI
client = wrap_openai(OpenAI())
@traceable def my_llm_pipeline(question: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": question}], ) return resp.choices[0].message.content

Nested tracing example

Nested tracing example

@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)
@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs
@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)
</python>

<typescript>
Use traceable() wrapper and wrapOpenAI() for automatic tracing.
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";

const client = wrapOpenAI(new OpenAI());

const myLlmPipeline = traceable(async (question: string): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: question }],
  });
  return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });

// Nested tracing example
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
  return docs;
}, { name: "retrieve_docs" });

const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
  });
  return resp.choices[0].message.content || "";
}, { name: "generate_answer" });

const ragPipeline = traceable(async (question: string): Promise<string> => {
  const docs = await retrieveDocs(question);
  return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });
</typescript>
Best Practices:
  • Apply traceable to all nested functions you want visible in LangSmith
  • Wrapped clients auto-trace all calls
    wrap_openai()
    /
    wrapOpenAI()
    records every LLM call
  • Name your traces for easier filtering
  • Add metadata for searchability </trace_other_frameworks>
<traces_vs_runs> Use the
langsmith
CLI to query trace data.
Understanding the difference is critical:
  • Trace = A complete execution tree (root run + all child runs). A trace represents one full agent invocation with all its LLM calls, tool calls, and nested operations.
  • Run = A single node in the tree (one LLM call, one tool call, etc.)
Generally, query traces first — they provide complete context and preserve hierarchy needed for trajectory analysis and dataset generation. </traces_vs_runs>
<command_structure> Two command groups with consistent behavior:
langsmith
├── trace (operations on trace trees - USE THIS FIRST)
│   ├── list    - List traces (filters apply to root run)
│   ├── get     - Get single trace with full hierarchy
│   └── export  - Export traces to JSONL files (one file per trace)
├── run (operations on individual runs - for specific analysis)
│   ├── list    - List runs (flat, filters apply to any run)
│   ├── get     - Get single run
│   └── export  - Export runs to single JSONL file (flat)
├── dataset (dataset operations)
│   ├── list    - List datasets
│   ├── get     - Get dataset details
│   ├── create  - Create empty dataset
│   ├── delete  - Delete dataset
│   ├── export  - Export dataset to file
│   └── upload  - Upload local JSON as dataset
├── example (example operations)
│   ├── list    - List examples in a dataset
│   ├── create  - Add example to a dataset
│   └── delete  - Delete an example
├── evaluator (evaluator operations)
│   ├── list    - List evaluators
│   ├── upload  - Upload evaluator
│   └── delete  - Delete evaluator
├── experiment (experiment operations)
│   ├── list    - List experiments
│   └── get     - Get experiment results
├── thread (thread operations)
│   ├── list    - List conversation threads
│   └── get     - Get thread details
└── project (project operations)
    └── list    - List tracing projects
Key differences:
traces *
runs *
Filters apply toRoot run onlyAny matching run
--run-type
Not availableAvailable
ReturnsFull hierarchyFlat list
Export outputDirectory (one file/trace)Single file
</command_structure>
<querying_traces> Query traces using the
langsmith
CLI. Commands are language-agnostic.
bash
undefined
@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)
@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs
@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)
</python>

<typescript>
使用traceable()包装器和wrapOpenAI()实现自动追踪。
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";

const client = wrapOpenAI(new OpenAI());

const myLlmPipeline = traceable(async (question: string): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: question }],
  });
  return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });

// Nested tracing example
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
  return docs;
}, { name: "retrieve_docs" });

const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
  });
  return resp.choices[0].message.content || "";
}, { name: "generate_answer" });

const ragPipeline = traceable(async (question: string): Promise<string> => {
  const docs = await retrieveDocs(question);
  return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });
</typescript>
最佳实践:
  • 为所有你希望在LangSmith中可见的嵌套函数应用traceable装饰器
  • 包装后的客户端会自动追踪所有调用 ——
    wrap_openai()
    /
    wrapOpenAI()
    会记录每一次LLM调用
  • 为你的追踪命名以便于筛选
  • 添加元数据提升可搜索性 </trace_other_frameworks>
<traces_vs_runs> 使用
langsmith
CLI查询追踪数据。
理解两者的区别至关重要:
  • Trace(追踪) = 完整的执行树(根运行节点 + 所有子运行节点)。一个Trace代表一次完整的Agent调用,包含所有LLM调用、工具调用和嵌套操作。
  • Run(运行) = 树中的单个节点(一次LLM调用、一次工具调用等)
通常优先查询traces —— 它们提供完整的上下文,保留轨迹分析和数据集生成所需的层级结构。 </traces_vs_runs>
<command_structure> 包含两组行为一致的命令:
langsmith
├── trace (operations on trace trees - USE THIS FIRST)
│   ├── list    - List traces (filters apply to root run)
│   ├── get     - Get single trace with full hierarchy
│   └── export  - Export traces to JSONL files (one file per trace)
├── run (operations on individual runs - for specific analysis)
│   ├── list    - List runs (flat, filters apply to any run)
│   ├── get     - Get single run
│   └── export  - Export runs to single JSONL file (flat)
├── dataset (dataset operations)
│   ├── list    - List datasets
│   ├── get     - Get dataset details
│   ├── create  - Create empty dataset
│   ├── delete  - Delete dataset
│   ├── export  - Export dataset to file
│   └── upload  - Upload local JSON as dataset
├── example (example operations)
│   ├── list    - List examples in a dataset
│   ├── create  - Add example to a dataset
│   └── delete  - Delete an example
├── evaluator (evaluator operations)
│   ├── list    - List evaluators
│   ├── upload  - Upload evaluator
│   └── delete  - Delete evaluator
├── experiment (experiment operations)
│   ├── list    - List experiments
│   └── get     - Get experiment results
├── thread (thread operations)
│   ├── list    - List conversation threads
│   └── get     - Get thread details
└── project (project operations)
    └── list    - List tracing projects
主要差异:
traces *
runs *
筛选规则应用于仅根运行节点所有匹配的运行节点
--run-type
选项
不可用可用
返回结果完整层级结构扁平化列表
导出输出目录(每个trace对应一个文件)单个文件
</command_structure>
<querying_traces> 使用
langsmith
CLI查询追踪数据。命令与语言无关。
bash
undefined

List recent traces (most common operation)

列出最近的追踪数据(最常用操作)

langsmith trace list --limit 10 --project my-project
langsmith trace list --limit 10 --project my-project

List traces with metadata (timing, tokens, costs)

列出包含元数据的追踪数据(耗时、token、成本)

langsmith trace list --limit 10 --include-metadata
langsmith trace list --limit 10 --include-metadata

Filter traces by time

按时间筛选追踪数据

langsmith trace list --last-n-minutes 60 langsmith trace list --since 2025-01-20T10:00:00Z
langsmith trace list --last-n-minutes 60 langsmith trace list --since 2025-01-20T10:00:00Z

Get specific trace with full hierarchy

获取特定追踪数据的完整层级结构

langsmith trace get <trace-id>
langsmith trace get <trace-id>

List traces and show hierarchy inline

列出追踪数据并内联显示层级结构

langsmith trace list --limit 5 --show-hierarchy
langsmith trace list --limit 5 --show-hierarchy

Export traces to JSONL (one file per trace, includes all runs)

导出追踪数据到JSONL(每个trace对应一个文件,包含所有运行节点)

langsmith trace export ./traces --limit 20 --full
langsmith trace export ./traces --limit 20 --full

Filter traces by performance

按性能筛选追踪数据

langsmith trace list --min-latency 5.0 --limit 10 # Slow traces (>= 5s) langsmith trace list --error --last-n-minutes 60 # Failed traces
langsmith trace list --min-latency 5.0 --limit 10 # 慢追踪数据(≥5秒) langsmith trace list --error --last-n-minutes 60 # 失败的追踪数据

List specific run types (flat list)

列出特定类型的运行节点(扁平化列表)

langsmith run list --run-type llm --limit 20
</querying_traces>

<filters>
All commands support these filters (all AND together):

**Basic filters:**
- `--trace-ids abc,def` - Filter to specific traces
- `--limit N` - Max results
- `--project NAME` - Project name
- `--last-n-minutes N` - Time filter
- `--since TIMESTAMP` - Time filter (ISO format)
- `--error / --no-error` - Error status
- `--name PATTERN` - Name contains (case-insensitive)

**Performance filters:**
- `--min-latency SECONDS` - Minimum latency (e.g., `5` for >= 5s)
- `--max-latency SECONDS` - Maximum latency
- `--min-tokens N` - Minimum total tokens
- `--tags tag1,tag2` - Has any of these tags

**Advanced filter:**
- `--filter QUERY` - Raw LangSmith filter query for complex cases (feedback, metadata, etc.)

```bash
langsmith run list --run-type llm --limit 20
</querying_traces>

<filters>
所有命令都支持以下筛选条件(多条件为逻辑与关系):

**基础筛选条件:**
- `--trace-ids abc,def` - 筛选特定追踪数据
- `--limit N` - 最大结果数
- `--project NAME` - 项目名称
- `--last-n-minutes N` - 时间筛选
- `--since TIMESTAMP` - 时间筛选(ISO格式)
- `--error / --no-error` - 错误状态
- `--name PATTERN` - 名称包含指定内容(不区分大小写)

**性能筛选条件:**
- `--min-latency SECONDS` - 最小耗时(例如`5`表示≥5秒)
- `--max-latency SECONDS` - 最大耗时
- `--min-tokens N` - 最小总token数
- `--tags tag1,tag2` - 包含指定标签中的任意一个

**高级筛选条件:**
- `--filter QUERY` - 用于复杂场景的原生LangSmith筛选查询(反馈、元数据等)

```bash

Filter traces by feedback score using raw LangSmith query

使用原生LangSmith查询按反馈分数筛选追踪数据

langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))'
</filters>

<export_format>
Export creates `.jsonl` files (one run per line) with these fields:
```json
{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}
Use
--include-io
or
--full
to include inputs/outputs (required for dataset generation). </export_format>
<tips> - **Start with traces** — they provide complete context needed for trajectory and dataset generation - Use `traces export --full` for bulk data destined for datasets - Always specify `--project` to avoid mixing data from different projects - Use `/tmp` for temporary exports - Include `--include-metadata` for performance/cost analysis - Stitch files: `cat ./traces/*.jsonl > all.jsonl` </tips>
langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))'
</filters>

<export_format>
导出操作会生成`.jsonl`文件(每行一个运行节点),包含以下字段:
```json
{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}
使用
--include-io
--full
参数来包含输入/输出数据(数据集生成时需要)。 </export_format>
<tips> - **优先从traces开始** —— 它们提供轨迹分析和数据集生成所需的完整上下文 - 使用`traces export --full`导出用于数据集的批量数据 - 始终指定`--project`参数,避免混合不同项目的数据 - 使用`/tmp`目录存储临时导出文件 - 加上`--include-metadata`参数用于性能/成本分析 - 合并文件:`cat ./traces/*.jsonl > all.jsonl` </tips>