opik
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseOpik — Observability for LLM Agents
Opik — LLM Agent可观测性工具
Integrating with Opik always means adding all three components unless the user explicitly asks for only one:
- Tracing — instrument LLM calls with the appropriate integration or
@opik.track - Entrypoint — mark the top-level function with for Local Runner and UI integration
entrypoint=True - Agent Configuration — externalize all tunable parameters into : model names, temperatures, top_p, max_tokens, all prompts and prompt templates, and any other runtime parameters the user may want to compare or optimize
AgentConfig
集成Opik时默认需要添加全部三个组件,除非用户明确要求仅使用其中一个:
- 链路追踪 — 通过适配的集成方式或装饰器埋点LLM调用
@opik.track - 入口点标记 — 给顶层函数添加标记,以适配本地运行器和UI集成
entrypoint=True - Agent配置 — 将所有可调整参数外部化到中:包括模型名称、温度系数、top_p、max_tokens、所有提示词和提示词模板,以及其他用户可能需要对比或优化的运行时参数
AgentConfig
Setup
配置
Environment Config Decision Tree
环境配置决策树
Before adding Opik config, inspect the project's existing config approach. Follow this decision tree exactly:
-
Check for existing/
.envfiles and.env.localusage in code.dotenv- If the project loads a file (via
.env,python-dotenv, or framework auto-loading): appenddotenvandOPIK_API_KEYto that same file. Do NOT create a separate config file.OPIK_WORKSPACE - If there is a or
.env.example: also update it with the new Opik vars (using placeholder values) so future developers know which vars are needed..env.sample
- If the project loads a
-
If nofile exists:
.env- Python: create or update (INI format). This is the SDK's native config file.
~/.opik.config - TypeScript/JavaScript: create (or
.envif the project uses Next.js or similar)..env.local
- Python: create or update
-
Never introduce a second config mechanism. If the project already usesfor API keys, do NOT also create
.env. If it uses~/.opik.config, do NOT add Opik vars to~/.opik.config..env -
Never overwrite existing values. Ifis already set in
OPIK_API_KEY, leave it. Only add vars that are missing..env -
Prefer settingin code, not in env files — one machine may log to many projects.
project_name -
If the user provides an API key and workspace in the prompt, use those values directly. If they provide only an API key, ask for the workspace or default tofor local OSS.
"default"
添加Opik配置前,请先检查项目现有的配置方案,严格遵循以下决策树执行:
-
检查项目是否存在/
.env文件,以及代码中是否使用.env.local加载配置dotenv- 如果项目通过、
python-dotenv或框架自动加载机制读取dotenv文件:将.env和OPIK_API_KEY追加到该文件中,不要创建独立的配置文件OPIK_WORKSPACE - 如果项目存在或
.env.example文件:同时更新该文件,添加Opik相关变量(使用占位符值),方便后续开发者知晓需要配置的变量.env.sample
- 如果项目通过
-
如果项目不存在文件:
.env- Python项目:创建或更新(INI格式),这是SDK原生支持的配置文件
~/.opik.config - TypeScript/JavaScript项目:创建文件(如果是Next.js等框架项目则创建
.env).env.local
- Python项目:创建或更新
-
不要引入第二种配置机制:如果项目已经使用存储API密钥,就不要再创建
.env;如果项目已经使用~/.opik.config,就不要把Opik变量添加到~/.opik.config中.env -
不要覆盖现有配置值:如果已经在
OPIK_API_KEY中配置,保留原有值,仅添加缺失的变量.env -
优先在代码中设置,而非在环境变量中配置——同一台机器可能会向多个项目上报数据
project_name -
如果用户在提示中提供了API密钥和工作空间,直接使用这些值;如果仅提供了API密钥,询问工作空间信息,本地开源版本默认使用作为工作空间
"default"
Config Formats
配置格式
Python (INI):
~/.opik.configini
[opik]
api_key=your-api-key
url_override=https://www.comet.com/opik/api
workspace=your-workspaceEnvironment variables (append to existing ):
.envbash
undefinedPython (INI格式):
~/.opik.configini
[opik]
api_key=your-api-key
url_override=https://www.comet.com/opik/api
workspace=your-workspace环境变量(追加到现有文件中):
.envbash
undefinedOpik
Opik
OPIK_API_KEY=your-api-key
OPIK_URL_OVERRIDE=https://www.comet.com/opik/api
OPIK_WORKSPACE=your-workspace
TypeScript uses `OPIK_WORKSPACE` as the env var and `workspaceName` in `new Opik({...})`.OPIK_API_KEY=your-api-key
OPIK_URL_OVERRIDE=https://www.comet.com/opik/api
OPIK_WORKSPACE=your-workspace
TypeScript中使用`OPIK_WORKSPACE`作为环境变量名,初始化`new Opik({...})`时传入`workspaceName`参数Standard Deployments
标准部署方式
- Cloud: — requires
https://www.comet.com/opik/api+api_keyworkspace - Local OSS: — usually workspace
http://localhost:5173/apidefault - Self-hosted: use the deployment's custom URL, following the project's existing config style
- 云版本:— 需要配置
https://www.comet.com/opik/api+api_keyworkspace - 本地开源版本:— 通常工作空间为
http://localhost:5173/apidefault - 自托管版本:使用部署的自定义URL,遵循项目现有配置风格
Interactive Config (optional)
交互式配置(可选)
bash
opik configure
opik configure --use_local
npx opik-ts configure
npx opik-ts configure --use-localSet the project name in code:
python
@opik.track(project_name="my-project")
def run():
...typescript
const client = new Opik({ projectName: "my-project" });bash
opik configure
opik configure --use_local
npx opik-ts configure
npx opik-ts configure --use-local在代码中设置项目名称:
python
@opik.track(project_name="my-project")
def run():
...typescript
const client = new Opik({ projectName: "my-project" });Python Instrumentation
Python 埋点教程
python
import opik
@opik.track(entrypoint=True, name="my-agent")
def agent(query: str) -> str:
context = retrieve(query)
return generate(query, context)
@opik.track(type="tool")
def retrieve(query: str) -> list:
return search_db(query)
@opik.track(type="llm")
def generate(query: str, context: list) -> str:
return llm_call(query, context)
result = agent("What is ML?")
opik.flush_tracker() # required in scriptsValid span types for manual instrumentation: , , , .
generalllmtoolguardrailFramework integrations — these capture tokens, model, and cost automatically:
python
from opik.integrations.openai import track_openai # OpenAI
from opik.integrations.anthropic import track_anthropic # Anthropic
from opik.integrations.langchain import OpikTracer # LangChain
from opik.integrations.crewai import track_crewai # CrewAI
from opik.integrations.dspy import OpikCallback # DSPy
from opik.integrations.adk import track_adk_agent_recursive # Google ADKCRITICAL — LiteLLM inside :
OpikLogger@opik.trackIf the codebase uses AND you are adding decorators, you MUST pass via the metadata parameter on every / call. This tells the callback to nest under the active trace. Without it, creates orphaned top-level traces that are separate from your hierarchy.
litellm@opik.trackcurrent_span_datalitellm.completion()litellm.acompletion()OpikLoggerOpikLogger@opik.trackpython
from opik import track
from opik.opik_context import get_current_span_data
from litellm.integrations.opik.opik import OpikLogger
import litellm
litellm.callbacks = [OpikLogger()]
@track
def call_llm(messages, model="gpt-4o"):
return litellm.completion(
model=model,
messages=messages,
metadata={
"opik": {
"current_span_data": get_current_span_data(),
"tags": ["litellm"],
},
},
)
@track(entrypoint=True)
def agent(query: str) -> str:
return call_llm([{"role": "user", "content": query}])This pattern applies whenever you see or in existing code that you are instrumenting with .
litellm.completionlitellm.acompletion@opik.trackpython
import opik
@opik.track(entrypoint=True, name="my-agent")
def agent(query: str) -> str:
context = retrieve(query)
return generate(query, context)
@opik.track(type="tool")
def retrieve(query: str) -> list:
return search_db(query)
@opik.track(type="llm")
def generate(query: str, context: list) -> str:
return llm_call(query, context)
result = agent("What is ML?")
opik.flush_tracker() # 脚本运行时必须调用手动埋点支持的span类型:、、、
generalllmtoolguardrail框架集成 — 可自动采集Token、模型、成本数据:
python
from opik.integrations.openai import track_openai # OpenAI
from opik.integrations.anthropic import track_anthropic # Anthropic
from opik.integrations.langchain import OpikTracer # LangChain
from opik.integrations.crewai import track_crewai # CrewAI
from opik.integrations.dspy import OpikCallback # DSPy
from opik.integrations.adk import track_adk_agent_recursive # Google ADK重要注意事项 — 在中使用LiteLLM的:
@opik.trackOpikLogger如果代码库使用,且你正在添加装饰器,必须在每次调用/时,通过metadata参数传入。这会让回调将数据嵌套到当前活跃链路下,否则会生成孤立的顶层链路,和你的层级相互独立
litellm@opik.tracklitellm.completion()litellm.acompletion()current_span_dataOpikLoggerOpikLogger@opik.trackpython
from opik import track
from opik.opik_context import get_current_span_data
from litellm.integrations.opik.opik import OpikLogger
import litellm
litellm.callbacks = [OpikLogger()]
@track
def call_llm(messages, model="gpt-4o"):
return litellm.completion(
model=model,
messages=messages,
metadata={
"opik": {
"current_span_data": get_current_span_data(),
"tags": ["litellm"],
},
},
)
@track(entrypoint=True)
def agent(query: str) -> str:
return call_llm([{"role": "user", "content": query}])只要你在使用埋点的现有代码中看到或,都需要遵循这个模式
@opik.tracklitellm.completionlitellm.acompletionTypeScript Instrumentation
TypeScript 埋点教程
typescript
import { Opik } from "opik";
const client = new Opik({ projectName: "my-project" });
const trace = client.trace({
name: "my-agent",
input: { query: "What is ML?" },
});
const toolSpan = trace.span({
name: "retrieve-context",
type: "tool",
input: { query: "What is ML?" },
});
// retrieval logic
toolSpan.end({ output: { documents: [] } });
const llmSpan = trace.span({
name: "generate-response",
type: "llm",
input: { prompt: "What is ML?" },
});
// model call
llmSpan.end({ output: { response: "Machine learning is..." } });
trace.end({ output: { response: "Machine learning is..." } });
await client.flush();Prefer the client-based path in TypeScript. Use in code rather than machine-wide config when possible.
projectNameFor framework-specific integrations such as Vercel AI SDK or LangChain.js, see .
references/tracing-typescript.mdAlways before exit.
await client.flush()Valid span types for manual instrumentation: , , , .
generalllmtoolguardrailtypescript
import { Opik } from "opik";
const client = new Opik({ projectName: "my-project" });
const trace = client.trace({
name: "my-agent",
input: { query: "What is ML?" },
});
const toolSpan = trace.span({
name: "retrieve-context",
type: "tool",
input: { query: "What is ML?" },
});
// 检索逻辑
toolSpan.end({ output: { documents: [] } });
const llmSpan = trace.span({
name: "generate-response",
type: "llm",
input: { prompt: "What is ML?" },
});
// 模型调用
llmSpan.end({ output: { response: "Machine learning is..." } });
trace.end({ output: { response: "Machine learning is..." } });
await client.flush();TypeScript中优先使用客户端调用方式,尽可能在代码中配置,而非使用机器全局配置
projectName如需Vercel AI SDK、LangChain.js等框架集成教程,请查看
references/tracing-typescript.md程序退出前必须调用
await client.flush()手动埋点支持的span类型:、、、
generalllmtoolguardrailThreads (Conversations)
对话线程(Threads)
Group conversation turns via . Each turn = one trace; shared = one thread.
thread_idthread_idpython
@opik.track(entrypoint=True)
def handle_message(session_id: str, message: str) -> str:
opik.update_current_trace(thread_id=session_id)
return generate_response(session_id, message)Thread metrics:
python
from opik.evaluation import evaluate_threads
from opik.evaluation.metrics.conversation import (
SessionCompletenessMetric, UserFrustrationMetric, ConversationalCoherenceMetric,
)
results = evaluate_threads(project_name="chat-agent", metrics=[
SessionCompletenessMetric(), UserFrustrationMetric(), ConversationalCoherenceMetric(),
])Use for chat agents, support bots, multi-step assistants. Skip for single-shot agents or batch processing.
Pitfalls: Missing → turns appear as unrelated traces. Shared across users → conversations get mixed.
thread_idthread_id通过对对话轮次进行分组,每一轮对话对应一条链路,共享的链路属于同一个对话线程
thread_idthread_idpython
@opik.track(entrypoint=True)
def handle_message(session_id: str, message: str) -> str:
opik.update_current_trace(thread_id=session_id)
return generate_response(session_id, message)对话线程指标评估:
python
from opik.evaluation import evaluate_threads
from opik.evaluation.metrics.conversation import (
SessionCompletenessMetric, UserFrustrationMetric, ConversationalCoherenceMetric,
)
results = evaluate_threads(project_name="chat-agent", metrics=[
SessionCompletenessMetric(), UserFrustrationMetric(), ConversationalCoherenceMetric(),
])适用于聊天Agent、客服机器人、多轮助手场景,单轮Agent或批处理场景无需使用
常见问题: 缺失 → 对话轮次会显示为无关的独立链路;不同用户共享 → 不同用户的对话会混合在一起
thread_idthread_idAgent Configuration
Agent配置
Externalize the parts of your agent you expect to tune over time into versioned, immutable config snapshots. This includes prompts, models, temperatures, token limits, and other runtime parameters you may want to compare, optimize, or roll out gradually.
CRITICAL — Search for existing config classes first. Before creating a new , search the codebase for existing classes that hold tunable parameters (model names, temperatures, prompts, token limits, etc.). Look for names like , , , , , or any /Pydantic model with fields like , , , . An existing config class is a migration target, not a reason to skip this step. If found, convert it to inherit from :
AgentConfigAgentConfigConfigSettingsAgentSettingsModelConfig@dataclassmodeltemperaturesystem_promptmax_tokensopik.AgentConfig- Replace the existing base (,
@dataclass, plain class) withBaseModelopik.AgentConfig - Add type hints with descriptions to each field
Annotated - Convert plain prompt fields to
stropik.Prompt - Wire up at startup and
client.create_agent_config_version()inside the entrypointclient.get_agent_config() - Update all call sites that reference the old config to use the new Opik-managed config
python
from typing import Annotated
import opik
class AgentConfig(opik.AgentConfig):
model: Annotated[str, "LLM model"] # NO defaults
temperature: Annotated[float, "Sampling temperature"]
system_prompt: Annotated[opik.Prompt, "Managed system prompt"]
DEFAULT_AGENT_CONFIG = AgentConfig(
model="gpt-4o",
temperature=0.7,
system_prompt=opik.Prompt(
name="agent-system-prompt",
prompt="You are a helpful assistant for {{product}}.",
),
)
client = opik.Opik()
client.create_agent_config_version(
AgentConfig(
model="gpt-4o",
temperature=0.7,
system_prompt=opik.Prompt(
name="agent-system-prompt",
prompt="You are a helpful assistant for {{product}}.",
),
),
project_name="my-agent",
)将Agent中需要长期调整的部分外部化为版本化、不可变的配置快照,包括提示词、模型、温度系数、Token限制,以及其他你可能需要对比、优化或灰度发布的运行时参数
重要注意事项 — 先查找现有配置类:创建新的前,先在代码库中搜索存储可调整参数(模型名称、温度系数、提示词、Token限制等)的现有类,查找命名为、、、、的类,或者包含、、、等字段的/Pydantic模型。现有配置类是迁移目标,不是跳过本步骤的理由,如果找到现有配置类,将其修改为继承:
AgentConfigAgentConfigConfigSettingsAgentSettingsModelConfigmodeltemperaturesystem_promptmax_tokens@dataclassopik.AgentConfig- 将现有的基类(、
@dataclass、普通类)替换为BaseModelopik.AgentConfig - 给每个字段添加带描述的类型注解
Annotated - 将普通类型的提示词字段转换为
str类型opik.Prompt - 在程序启动时调用,在入口点内调用
client.create_agent_config_version()client.get_agent_config() - 更新所有引用旧配置的调用点,使用Opik托管的新配置
python
from typing import Annotated
import opik
class AgentConfig(opik.AgentConfig):
model: Annotated[str, "LLM model"] # 不要设置默认值
temperature: Annotated[float, "Sampling temperature"]
system_prompt: Annotated[opik.Prompt, "Managed system prompt"]
DEFAULT_AGENT_CONFIG = AgentConfig(
model="gpt-4o",
temperature=0.7,
system_prompt=opik.Prompt(
name="agent-system-prompt",
prompt="You are a helpful assistant for {{product}}.",
),
)
client = opik.Opik()
client.create_agent_config_version(
AgentConfig(
model="gpt-4o",
temperature=0.7,
system_prompt=opik.Prompt(
name="agent-system-prompt",
prompt="You are a helpful assistant for {{product}}.",
),
),
project_name="my-agent",
)Identical values → same version (dedup). Different values → new version.
配置值相同 → 复用同一版本(自动去重),配置值不同 → 生成新版本
@opik.track(entrypoint=True, project_name="my-agent")
def run_agent(question: str) -> str:
cfg = client.get_agent_config(
fallback=DEFAULT_AGENT_CONFIG,
project_name="my-agent",
# optional: latest=True | env="staging" | version="v1" (default: prod)
)
return llm_call(
model=cfg.model,
temperature=cfg.temperature,
system_prompt=cfg.system_prompt.format(product="Opik"),
question=question,
)
- `get_agent_config()` **must** be inside `@opik.track` — raises error otherwise
- Deploy: `cfg.deploy_to("prod")` — tags a version with an environment
- Prompt fields: use `Prompt` (from `opik.api_objects.prompt.text.prompt`) / `ChatPrompt` (from `opik.api_objects.prompt.chat.chat_prompt`) typed config fields for managed prompts
- **Extract:** model, temperature, top_p, max_tokens, system prompt, tunable params
- **Don't extract:** API keys, structural logic, true constants@opik.track(entrypoint=True, project_name="my-agent")
def run_agent(question: str) -> str:
cfg = client.get_agent_config(
fallback=DEFAULT_AGENT_CONFIG,
project_name="my-agent",
# 可选参数: latest=True | env="staging" | version="v1" (默认取生产环境版本)
)
return llm_call(
model=cfg.model,
temperature=cfg.temperature,
system_prompt=cfg.system_prompt.format(product="Opik"),
question=question,
)
- `get_agent_config()` **必须**放在`@opik.track`装饰的函数内部,否则会抛出错误
- 发布配置:调用`cfg.deploy_to("prod")` 可以给指定版本打上环境标签
- 提示词字段:使用`opik.api_objects.prompt.text.prompt`下的`Prompt`类型、或`opik.api_objects.prompt.chat.chat_prompt`下的`ChatPrompt`类型作为配置字段类型,实现提示词托管
- **需要提取的配置项:** 模型、温度系数、top_p、max_tokens、系统提示词、可调整参数
- **不需要提取的配置项:** API密钥、结构性逻辑、固定常量Local Runner (opik connect)
本地运行器(opik connect)
Pair your local agent with the Opik browser UI. Get a pairing code from the UI, then:
bash
opik connect --pair <CODE> python3 app.py # Python
opik connect --pair <CODE> npx tsx app.ts # TypeScriptReplace or with the normal command you use to start your app locally.
python3 app.pynpx tsx app.tsPython: + type-hinted parameters for schema discovery.
TypeScript: .
@track(entrypoint=True)track({ entrypoint: true, params: [{name, type}] }, fn)After pairing: entrypoint registered as agent, UI shows input form, jobs from UI or Optimizer trigger runs.
| Issue | Fix |
|---|---|
| No entrypoint found | Add |
| Invalid pair code | Codes expire — get a new one |
| Connection refused | Check Opik server (OSS) or API key (Cloud) |
将本地Agent和Opik浏览器UI绑定,从UI获取配对码后执行以下命令:
bash
opik connect --pair <CODE> python3 app.py # Python
opik connect --pair <CODE> npx tsx app.ts # TypeScript将或替换为你本地启动应用的常规命令即可
python3 app.pynpx tsx app.tsPython侧要求: + 参数带类型注解,用于自动发现Schema
TypeScript侧要求:
@track(entrypoint=True)track({ entrypoint: true, params: [{name, type}] }, fn)配对完成后:入口点会被注册为Agent,UI会显示输入表单,来自UI或优化器的任务会触发本地Agent运行
| 问题 | 修复方案 |
|---|---|
| 未找到入口点 | 给对应函数添加 |
| 配对码无效 | 配对码已过期,请获取新的配对码 |
| 连接被拒绝 | 检查Opik服务器状态(开源版)或API密钥配置(云版本) |
Anti-Patterns
反模式
| Anti-Pattern | Fix |
|---|---|
Existing config class left unconverted (e.g., | Convert to |
| Hardcoded config | Use |
| Missing entrypoint | Add |
| No thread_id on conversational agent | Wire |
| Must be inside decorated function |
TS missing | Add explicit |
Missing | Call before exit |
| 反模式 | 修复方案 |
|---|---|
未迁移现有配置类(例如包含model/temperature/prompt字段的 | 转换为 |
| 硬编码配置 | 使用 |
| 缺失入口点标记 | 给入口函数添加 |
对话类Agent未设置 | 从会话ID中提取 |
| 必须放在装饰器内部 |
TypeScript侧缺失 | 显式添加 |
脚本中未调用 | 在程序退出前调用 |
References
参考文档
| Topic | File |
|---|---|
| Python SDK (decorators, async, distributed, config, entrypoint) | |
| TypeScript SDK (client, decorators, entrypoint, params) | |
| REST API | |
| All integrations | |
| Core concepts (traces, spans, threads, metadata) | |
| Evaluation (suites, 41 built-in metrics, trajectory) | |
| 主题 | 文件路径 |
|---|---|
| Python SDK(装饰器、异步、分布式、配置、入口点) | |
| TypeScript SDK(客户端、装饰器、入口点、参数) | |
| REST API | |
| 所有集成 | |
| 核心概念(链路、Span、线程、元数据) | |
| 评估(套件、41个内置指标、轨迹评估) | |