opik

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Opik — Observability for LLM Agents

Opik — LLM Agent可观测性工具

Integrating with Opik always means adding all three components unless the user explicitly asks for only one:
  1. Tracing — instrument LLM calls with the appropriate integration or
    @opik.track
  2. Entrypoint — mark the top-level function with
    entrypoint=True
    for Local Runner and UI integration
  3. Agent Configuration — externalize all tunable parameters into
    AgentConfig
    : model names, temperatures, top_p, max_tokens, all prompts and prompt templates, and any other runtime parameters the user may want to compare or optimize
集成Opik时默认需要添加全部三个组件,除非用户明确要求仅使用其中一个:
  1. 链路追踪 — 通过适配的集成方式或
    @opik.track
    装饰器埋点LLM调用
  2. 入口点标记 — 给顶层函数添加
    entrypoint=True
    标记,以适配本地运行器和UI集成
  3. Agent配置 — 将所有可调整参数外部化到
    AgentConfig
    中:包括模型名称、温度系数、top_p、max_tokens、所有提示词和提示词模板,以及其他用户可能需要对比或优化的运行时参数

Setup

配置

Environment Config Decision Tree

环境配置决策树

Before adding Opik config, inspect the project's existing config approach. Follow this decision tree exactly:
  1. Check for existing
    .env
    /
    .env.local
    files and
    dotenv
    usage in code.
    • If the project loads a
      .env
      file (via
      python-dotenv
      ,
      dotenv
      , or framework auto-loading): append
      OPIK_API_KEY
      and
      OPIK_WORKSPACE
      to that same file. Do NOT create a separate config file.
    • If there is a
      .env.example
      or
      .env.sample
      : also update it with the new Opik vars (using placeholder values) so future developers know which vars are needed.
  2. If no
    .env
    file exists:
    • Python: create or update
      ~/.opik.config
      (INI format). This is the SDK's native config file.
    • TypeScript/JavaScript: create
      .env
      (or
      .env.local
      if the project uses Next.js or similar).
  3. Never introduce a second config mechanism. If the project already uses
    .env
    for API keys, do NOT also create
    ~/.opik.config
    . If it uses
    ~/.opik.config
    , do NOT add Opik vars to
    .env
    .
  4. Never overwrite existing values. If
    OPIK_API_KEY
    is already set in
    .env
    , leave it. Only add vars that are missing.
  5. Prefer setting
    project_name
    in code
    , not in env files — one machine may log to many projects.
  6. If the user provides an API key and workspace in the prompt, use those values directly. If they provide only an API key, ask for the workspace or default to
    "default"
    for local OSS.
添加Opik配置前,请先检查项目现有的配置方案,严格遵循以下决策树执行:
  1. 检查项目是否存在
    .env
    /
    .env.local
    文件,以及代码中是否使用
    dotenv
    加载配置
    • 如果项目通过
      python-dotenv
      dotenv
      或框架自动加载机制读取
      .env
      文件:
      OPIK_API_KEY
      OPIK_WORKSPACE
      追加到该文件中
      ,不要创建独立的配置文件
    • 如果项目存在
      .env.example
      .env.sample
      文件:同时更新该文件,添加Opik相关变量(使用占位符值),方便后续开发者知晓需要配置的变量
  2. 如果项目不存在
    .env
    文件:
    • Python项目:创建或更新
      ~/.opik.config
      (INI格式),这是SDK原生支持的配置文件
    • TypeScript/JavaScript项目:创建
      .env
      文件(如果是Next.js等框架项目则创建
      .env.local
  3. 不要引入第二种配置机制:如果项目已经使用
    .env
    存储API密钥,就不要再创建
    ~/.opik.config
    ;如果项目已经使用
    ~/.opik.config
    ,就不要把Opik变量添加到
    .env
  4. 不要覆盖现有配置值:如果
    OPIK_API_KEY
    已经在
    .env
    中配置,保留原有值,仅添加缺失的变量
  5. 优先在代码中设置
    project_name
    ,而非在环境变量中配置——同一台机器可能会向多个项目上报数据
  6. 如果用户在提示中提供了API密钥和工作空间,直接使用这些值;如果仅提供了API密钥,询问工作空间信息,本地开源版本默认使用
    "default"
    作为工作空间

Config Formats

配置格式

Python
~/.opik.config
(INI):
ini
[opik]
api_key=your-api-key
url_override=https://www.comet.com/opik/api
workspace=your-workspace
Environment variables (append to existing
.env
):
bash
undefined
Python
~/.opik.config
(INI格式):
ini
[opik]
api_key=your-api-key
url_override=https://www.comet.com/opik/api
workspace=your-workspace
环境变量(追加到现有
.env
文件中):
bash
undefined

Opik

Opik

OPIK_API_KEY=your-api-key OPIK_URL_OVERRIDE=https://www.comet.com/opik/api OPIK_WORKSPACE=your-workspace

TypeScript uses `OPIK_WORKSPACE` as the env var and `workspaceName` in `new Opik({...})`.
OPIK_API_KEY=your-api-key OPIK_URL_OVERRIDE=https://www.comet.com/opik/api OPIK_WORKSPACE=your-workspace

TypeScript中使用`OPIK_WORKSPACE`作为环境变量名,初始化`new Opik({...})`时传入`workspaceName`参数

Standard Deployments

标准部署方式

  • Cloud:
    https://www.comet.com/opik/api
    — requires
    api_key
    +
    workspace
  • Local OSS:
    http://localhost:5173/api
    — usually workspace
    default
  • Self-hosted: use the deployment's custom URL, following the project's existing config style
  • 云版本:
    https://www.comet.com/opik/api
    — 需要配置
    api_key
    +
    workspace
  • 本地开源版本:
    http://localhost:5173/api
    — 通常工作空间为
    default
  • 自托管版本:使用部署的自定义URL,遵循项目现有配置风格

Interactive Config (optional)

交互式配置(可选)

bash
opik configure
opik configure --use_local
npx opik-ts configure
npx opik-ts configure --use-local
Set the project name in code:
python
@opik.track(project_name="my-project")
def run():
    ...
typescript
const client = new Opik({ projectName: "my-project" });
bash
opik configure
opik configure --use_local
npx opik-ts configure
npx opik-ts configure --use-local
在代码中设置项目名称:
python
@opik.track(project_name="my-project")
def run():
    ...
typescript
const client = new Opik({ projectName: "my-project" });

Python Instrumentation

Python 埋点教程

python
import opik

@opik.track(entrypoint=True, name="my-agent")
def agent(query: str) -> str:
    context = retrieve(query)
    return generate(query, context)

@opik.track(type="tool")
def retrieve(query: str) -> list:
    return search_db(query)

@opik.track(type="llm")
def generate(query: str, context: list) -> str:
    return llm_call(query, context)

result = agent("What is ML?")
opik.flush_tracker()  # required in scripts
Valid span types for manual instrumentation:
general
,
llm
,
tool
,
guardrail
.
Framework integrations — these capture tokens, model, and cost automatically:
python
from opik.integrations.openai import track_openai        # OpenAI
from opik.integrations.anthropic import track_anthropic   # Anthropic
from opik.integrations.langchain import OpikTracer        # LangChain
from opik.integrations.crewai import track_crewai         # CrewAI
from opik.integrations.dspy import OpikCallback           # DSPy
from opik.integrations.adk import track_adk_agent_recursive  # Google ADK
CRITICAL — LiteLLM
OpikLogger
inside
@opik.track
:
If the codebase uses
litellm
AND you are adding
@opik.track
decorators, you MUST pass
current_span_data
via the metadata parameter on every
litellm.completion()
/
litellm.acompletion()
call. This tells the
OpikLogger
callback to nest under the active trace. Without it,
OpikLogger
creates orphaned top-level traces that are separate from your
@opik.track
hierarchy.
python
from opik import track
from opik.opik_context import get_current_span_data
from litellm.integrations.opik.opik import OpikLogger
import litellm

litellm.callbacks = [OpikLogger()]

@track
def call_llm(messages, model="gpt-4o"):
    return litellm.completion(
        model=model,
        messages=messages,
        metadata={
            "opik": {
                "current_span_data": get_current_span_data(),
                "tags": ["litellm"],
            },
        },
    )

@track(entrypoint=True)
def agent(query: str) -> str:
    return call_llm([{"role": "user", "content": query}])
This pattern applies whenever you see
litellm.completion
or
litellm.acompletion
in existing code that you are instrumenting with
@opik.track
.
python
import opik

@opik.track(entrypoint=True, name="my-agent")
def agent(query: str) -> str:
    context = retrieve(query)
    return generate(query, context)

@opik.track(type="tool")
def retrieve(query: str) -> list:
    return search_db(query)

@opik.track(type="llm")
def generate(query: str, context: list) -> str:
    return llm_call(query, context)

result = agent("What is ML?")
opik.flush_tracker()  # 脚本运行时必须调用
手动埋点支持的span类型:
general
llm
tool
guardrail
框架集成 — 可自动采集Token、模型、成本数据:
python
from opik.integrations.openai import track_openai        # OpenAI
from opik.integrations.anthropic import track_anthropic   # Anthropic
from opik.integrations.langchain import OpikTracer        # LangChain
from opik.integrations.crewai import track_crewai         # CrewAI
from opik.integrations.dspy import OpikCallback           # DSPy
from opik.integrations.adk import track_adk_agent_recursive  # Google ADK
重要注意事项 — 在
@opik.track
中使用LiteLLM的
OpikLogger
如果代码库使用
litellm
,且你正在添加
@opik.track
装饰器,必须在每次调用
litellm.completion()
/
litellm.acompletion()
时,通过metadata参数传入
current_span_data
。这会让
OpikLogger
回调将数据嵌套到当前活跃链路下,否则
OpikLogger
会生成孤立的顶层链路,和你的
@opik.track
层级相互独立
python
from opik import track
from opik.opik_context import get_current_span_data
from litellm.integrations.opik.opik import OpikLogger
import litellm

litellm.callbacks = [OpikLogger()]

@track
def call_llm(messages, model="gpt-4o"):
    return litellm.completion(
        model=model,
        messages=messages,
        metadata={
            "opik": {
                "current_span_data": get_current_span_data(),
                "tags": ["litellm"],
            },
        },
    )

@track(entrypoint=True)
def agent(query: str) -> str:
    return call_llm([{"role": "user", "content": query}])
只要你在使用
@opik.track
埋点的现有代码中看到
litellm.completion
litellm.acompletion
,都需要遵循这个模式

TypeScript Instrumentation

TypeScript 埋点教程

typescript
import { Opik } from "opik";

const client = new Opik({ projectName: "my-project" });

const trace = client.trace({
  name: "my-agent",
  input: { query: "What is ML?" },
});

const toolSpan = trace.span({
  name: "retrieve-context",
  type: "tool",
  input: { query: "What is ML?" },
});

// retrieval logic
toolSpan.end({ output: { documents: [] } });

const llmSpan = trace.span({
  name: "generate-response",
  type: "llm",
  input: { prompt: "What is ML?" },
});

// model call
llmSpan.end({ output: { response: "Machine learning is..." } });

trace.end({ output: { response: "Machine learning is..." } });
await client.flush();
Prefer the client-based path in TypeScript. Use
projectName
in code rather than machine-wide config when possible.
For framework-specific integrations such as Vercel AI SDK or LangChain.js, see
references/tracing-typescript.md
.
Always
await client.flush()
before exit.
Valid span types for manual instrumentation:
general
,
llm
,
tool
,
guardrail
.
typescript
import { Opik } from "opik";

const client = new Opik({ projectName: "my-project" });

const trace = client.trace({
  name: "my-agent",
  input: { query: "What is ML?" },
});

const toolSpan = trace.span({
  name: "retrieve-context",
  type: "tool",
  input: { query: "What is ML?" },
});

// 检索逻辑
toolSpan.end({ output: { documents: [] } });

const llmSpan = trace.span({
  name: "generate-response",
  type: "llm",
  input: { prompt: "What is ML?" },
});

// 模型调用
llmSpan.end({ output: { response: "Machine learning is..." } });

trace.end({ output: { response: "Machine learning is..." } });
await client.flush();
TypeScript中优先使用客户端调用方式,尽可能在代码中配置
projectName
,而非使用机器全局配置
如需Vercel AI SDK、LangChain.js等框架集成教程,请查看
references/tracing-typescript.md
程序退出前必须调用
await client.flush()
手动埋点支持的span类型:
general
llm
tool
guardrail

Threads (Conversations)

对话线程(Threads)

Group conversation turns via
thread_id
. Each turn = one trace; shared
thread_id
= one thread.
python
@opik.track(entrypoint=True)
def handle_message(session_id: str, message: str) -> str:
    opik.update_current_trace(thread_id=session_id)
    return generate_response(session_id, message)
Thread metrics:
python
from opik.evaluation import evaluate_threads
from opik.evaluation.metrics.conversation import (
    SessionCompletenessMetric, UserFrustrationMetric, ConversationalCoherenceMetric,
)

results = evaluate_threads(project_name="chat-agent", metrics=[
    SessionCompletenessMetric(), UserFrustrationMetric(), ConversationalCoherenceMetric(),
])
Use for chat agents, support bots, multi-step assistants. Skip for single-shot agents or batch processing.
Pitfalls: Missing
thread_id
→ turns appear as unrelated traces. Shared
thread_id
across users → conversations get mixed.
通过
thread_id
对对话轮次进行分组,每一轮对话对应一条链路,共享
thread_id
的链路属于同一个对话线程
python
@opik.track(entrypoint=True)
def handle_message(session_id: str, message: str) -> str:
    opik.update_current_trace(thread_id=session_id)
    return generate_response(session_id, message)
对话线程指标评估:
python
from opik.evaluation import evaluate_threads
from opik.evaluation.metrics.conversation import (
    SessionCompletenessMetric, UserFrustrationMetric, ConversationalCoherenceMetric,
)

results = evaluate_threads(project_name="chat-agent", metrics=[
    SessionCompletenessMetric(), UserFrustrationMetric(), ConversationalCoherenceMetric(),
])
适用于聊天Agent、客服机器人、多轮助手场景,单轮Agent或批处理场景无需使用
常见问题: 缺失
thread_id
→ 对话轮次会显示为无关的独立链路;不同用户共享
thread_id
→ 不同用户的对话会混合在一起

Agent Configuration

Agent配置

Externalize the parts of your agent you expect to tune over time into versioned, immutable config snapshots. This includes prompts, models, temperatures, token limits, and other runtime parameters you may want to compare, optimize, or roll out gradually.
CRITICAL — Search for existing config classes first. Before creating a new
AgentConfig
, search the codebase for existing classes that hold tunable parameters (model names, temperatures, prompts, token limits, etc.). Look for names like
AgentConfig
,
Config
,
Settings
,
AgentSettings
,
ModelConfig
, or any
@dataclass
/Pydantic model with fields like
model
,
temperature
,
system_prompt
,
max_tokens
. An existing config class is a migration target, not a reason to skip this step. If found, convert it to inherit from
opik.AgentConfig
:
  1. Replace the existing base (
    @dataclass
    ,
    BaseModel
    , plain class) with
    opik.AgentConfig
  2. Add
    Annotated
    type hints with descriptions to each field
  3. Convert plain
    str
    prompt fields to
    opik.Prompt
  4. Wire up
    client.create_agent_config_version()
    at startup and
    client.get_agent_config()
    inside the entrypoint
  5. Update all call sites that reference the old config to use the new Opik-managed config
python
from typing import Annotated
import opik

class AgentConfig(opik.AgentConfig):
    model: Annotated[str, "LLM model"]             # NO defaults
    temperature: Annotated[float, "Sampling temperature"]
    system_prompt: Annotated[opik.Prompt, "Managed system prompt"]

DEFAULT_AGENT_CONFIG = AgentConfig(
    model="gpt-4o",
    temperature=0.7,
    system_prompt=opik.Prompt(
        name="agent-system-prompt",
        prompt="You are a helpful assistant for {{product}}.",
    ),
)

client = opik.Opik()
client.create_agent_config_version(
    AgentConfig(
        model="gpt-4o",
        temperature=0.7,
        system_prompt=opik.Prompt(
            name="agent-system-prompt",
            prompt="You are a helpful assistant for {{product}}.",
        ),
    ),
    project_name="my-agent",
)
将Agent中需要长期调整的部分外部化为版本化、不可变的配置快照,包括提示词、模型、温度系数、Token限制,以及其他你可能需要对比、优化或灰度发布的运行时参数
重要注意事项 — 先查找现有配置类:创建新的
AgentConfig
前,先在代码库中搜索存储可调整参数(模型名称、温度系数、提示词、Token限制等)的现有类,查找命名为
AgentConfig
Config
Settings
AgentSettings
ModelConfig
的类,或者包含
model
temperature
system_prompt
max_tokens
等字段的
@dataclass
/Pydantic模型。现有配置类是迁移目标,不是跳过本步骤的理由,如果找到现有配置类,将其修改为继承
opik.AgentConfig
  1. 将现有的基类(
    @dataclass
    BaseModel
    、普通类)替换为
    opik.AgentConfig
  2. 给每个字段添加带描述的
    Annotated
    类型注解
  3. 将普通
    str
    类型的提示词字段转换为
    opik.Prompt
    类型
  4. 在程序启动时调用
    client.create_agent_config_version()
    ,在入口点内调用
    client.get_agent_config()
  5. 更新所有引用旧配置的调用点,使用Opik托管的新配置
python
from typing import Annotated
import opik

class AgentConfig(opik.AgentConfig):
    model: Annotated[str, "LLM model"]             # 不要设置默认值
    temperature: Annotated[float, "Sampling temperature"]
    system_prompt: Annotated[opik.Prompt, "Managed system prompt"]

DEFAULT_AGENT_CONFIG = AgentConfig(
    model="gpt-4o",
    temperature=0.7,
    system_prompt=opik.Prompt(
        name="agent-system-prompt",
        prompt="You are a helpful assistant for {{product}}.",
    ),
)

client = opik.Opik()
client.create_agent_config_version(
    AgentConfig(
        model="gpt-4o",
        temperature=0.7,
        system_prompt=opik.Prompt(
            name="agent-system-prompt",
            prompt="You are a helpful assistant for {{product}}.",
        ),
    ),
    project_name="my-agent",
)

Identical values → same version (dedup). Different values → new version.

配置值相同 → 复用同一版本(自动去重),配置值不同 → 生成新版本

@opik.track(entrypoint=True, project_name="my-agent") def run_agent(question: str) -> str: cfg = client.get_agent_config( fallback=DEFAULT_AGENT_CONFIG, project_name="my-agent", # optional: latest=True | env="staging" | version="v1" (default: prod) ) return llm_call( model=cfg.model, temperature=cfg.temperature, system_prompt=cfg.system_prompt.format(product="Opik"), question=question, )

- `get_agent_config()` **must** be inside `@opik.track` — raises error otherwise
- Deploy: `cfg.deploy_to("prod")` — tags a version with an environment
- Prompt fields: use `Prompt` (from `opik.api_objects.prompt.text.prompt`) / `ChatPrompt` (from `opik.api_objects.prompt.chat.chat_prompt`) typed config fields for managed prompts
- **Extract:** model, temperature, top_p, max_tokens, system prompt, tunable params
- **Don't extract:** API keys, structural logic, true constants
@opik.track(entrypoint=True, project_name="my-agent") def run_agent(question: str) -> str: cfg = client.get_agent_config( fallback=DEFAULT_AGENT_CONFIG, project_name="my-agent", # 可选参数: latest=True | env="staging" | version="v1" (默认取生产环境版本) ) return llm_call( model=cfg.model, temperature=cfg.temperature, system_prompt=cfg.system_prompt.format(product="Opik"), question=question, )

- `get_agent_config()` **必须**放在`@opik.track`装饰的函数内部,否则会抛出错误
- 发布配置:调用`cfg.deploy_to("prod")` 可以给指定版本打上环境标签
- 提示词字段:使用`opik.api_objects.prompt.text.prompt`下的`Prompt`类型、或`opik.api_objects.prompt.chat.chat_prompt`下的`ChatPrompt`类型作为配置字段类型,实现提示词托管
- **需要提取的配置项:** 模型、温度系数、top_p、max_tokens、系统提示词、可调整参数
- **不需要提取的配置项:** API密钥、结构性逻辑、固定常量

Local Runner (opik connect)

本地运行器(opik connect)

Pair your local agent with the Opik browser UI. Get a pairing code from the UI, then:
bash
opik connect --pair <CODE> python3 app.py        # Python
opik connect --pair <CODE> npx tsx app.ts         # TypeScript
Replace
python3 app.py
or
npx tsx app.ts
with the normal command you use to start your app locally.
Python:
@track(entrypoint=True)
+ type-hinted parameters for schema discovery. TypeScript:
track({ entrypoint: true, params: [{name, type}] }, fn)
.
After pairing: entrypoint registered as agent, UI shows input form, jobs from UI or Optimizer trigger runs.
IssueFix
No entrypoint foundAdd
entrypoint=True
(Python) or
entrypoint: true
(TS)
Invalid pair codeCodes expire — get a new one
Connection refusedCheck Opik server (OSS) or API key (Cloud)
将本地Agent和Opik浏览器UI绑定,从UI获取配对码后执行以下命令:
bash
opik connect --pair <CODE> python3 app.py        # Python
opik connect --pair <CODE> npx tsx app.ts         # TypeScript
python3 app.py
npx tsx app.ts
替换为你本地启动应用的常规命令即可
Python侧要求:
@track(entrypoint=True)
+ 参数带类型注解,用于自动发现Schema TypeScript侧要求:
track({ entrypoint: true, params: [{name, type}] }, fn)
配对完成后:入口点会被注册为Agent,UI会显示输入表单,来自UI或优化器的任务会触发本地Agent运行
问题修复方案
未找到入口点给对应函数添加
entrypoint=True
(Python)或
entrypoint: true
(TS)
配对码无效配对码已过期,请获取新的配对码
连接被拒绝检查Opik服务器状态(开源版)或API密钥配置(云版本)

Anti-Patterns

反模式

Anti-PatternFix
Existing config class left unconverted (e.g.,
@dataclass
with model/temperature/prompt fields)
Convert to
opik.AgentConfig
subclass — an existing config is a migration target, not a skip signal
Hardcoded configUse
AgentConfig
+
get_agent_config()
Missing entrypointAdd
entrypoint=True
for Local Runner
No thread_id on conversational agentWire
thread_id
from session ID
get_agent_config()
outside
@track
Must be inside decorated function
TS missing
params
Add explicit
params
array
Missing
flush_tracker()
in scripts
Call before exit
反模式修复方案
未迁移现有配置类(例如包含model/temperature/prompt字段的
@dataclass
转换为
opik.AgentConfig
子类,现有配置是迁移目标,不是跳过的理由
硬编码配置使用
AgentConfig
+
get_agent_config()
缺失入口点标记给入口函数添加
entrypoint=True
适配本地运行器
对话类Agent未设置
thread_id
从会话ID中提取
thread_id
并绑定到链路
get_agent_config()
放在
@track
装饰的函数外部
必须放在装饰器内部
TypeScript侧缺失
params
配置
显式添加
params
数组
脚本中未调用
flush_tracker()
在程序退出前调用

References

参考文档

TopicFile
Python SDK (decorators, async, distributed, config, entrypoint)
references/tracing-python.md
TypeScript SDK (client, decorators, entrypoint, params)
references/tracing-typescript.md
REST API
references/tracing-rest-api.md
All integrations
references/integrations.md
Core concepts (traces, spans, threads, metadata)
references/observability.md
Evaluation (suites, 41 built-in metrics, trajectory)
references/evaluation.md
主题文件路径
Python SDK(装饰器、异步、分布式、配置、入口点)
references/tracing-python.md
TypeScript SDK(客户端、装饰器、入口点、参数)
references/tracing-typescript.md
REST API
references/tracing-rest-api.md
所有集成
references/integrations.md
核心概念(链路、Span、线程、元数据)
references/observability.md
评估(套件、41个内置指标、轨迹评估)
references/evaluation.md