exploring-llm-costs

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Exploring LLM costs

探索 LLM 成本

PostHog attaches per-call cost metadata to every

$ai_generation

and

$ai_embedding

event at ingestion time. Every cost question reduces to an aggregation over those two event types — the interesting variation is only in how you group, filter, and compare.

This skill covers the common cost investigations: total spend, breakdowns (model, provider, user, trace, custom property), token and cache-hit analysis, regression debugging, and materializing results as insights, dashboards, or alerts.

PostHog 会在数据摄取时为每个

$ai_generation

和

$ai_embedding

Event 附加单次调用成本元数据。所有成本问题都可归结为对这两类 Event 的聚合——真正的差异只在于如何分组、筛选和比较。

本 Skill 涵盖常见的成本调查：总支出、各项拆分（Model、Provider、用户、Trace、自定义属性）、Token 与 cache-hit 分析、回归调试，以及将结果物化为 Insight、Dashboard 或 Alert。

Tools

工具

Tool	Purpose
`posthog:query-trends`	Native cost trends with supported sums, breakdowns, and comparisons
`posthog:execute-sql`	Ad-hoc HogQL for any cost aggregation — the workhorse of this skill
`posthog:query-llm-traces-list`	List traces with rolled-up cost, token, and error metrics
`posthog:query-llm-trace`	Cost breakdown of a single trace across all its events
`posthog:read-data-schema`	Discover which custom properties exist for breakdowns
`posthog:insight-create`	Materialize a cost chart as a saved insight
`posthog:dashboard-create`	Bundle cost insights into a dashboard
`posthog:alert-create`	Alert when cost crosses a threshold
`posthog:generate-app-url`	Build region- and project-qualified links back to the UI

Tool	Purpose
`posthog:query-trends`	原生成本趋势分析，支持求和、拆分和比较
`posthog:execute-sql`	用于任意成本聚合的临时 HogQL——本 Skill 的主力工具
`posthog:query-llm-traces-list`	列出包含汇总成本、Token 和错误指标的 Trace
`posthog:query-llm-trace`	跨单个 Trace 的所有 Event 展示成本拆分
`posthog:read-data-schema`	查找可用于拆分的自定义属性
`posthog:insight-create`	将成本图表物化为已保存的 Insight
`posthog:dashboard-create`	将成本 Insight 汇总到 Dashboard
`posthog:alert-create`	当成本超过阈值时发出 Alert
`posthog:generate-app-url`	构建同时限定 Region 和 Project、可返回 UI 的链接

Core rules

核心规则

Three rules cover most of what goes wrong:

Sum
$ai_total_cost_usd
for rollups, never the components. Components drop request and web-search fees. The UI's cost cells sum
```
$ai_total_cost_usd
```
over
```
event IN ('$ai_generation', '$ai_embedding')
```
; mirror that. Full schema and rationale in cost properties.
Always include both
$ai_generation
and
$ai_embedding
in cost queries unless the project demonstrably does not use embeddings — missing them silently under-counts.
```
$ai_trace
```
and
```
$ai_span
```
carry no rollup cost; some SDK wrappers duplicate
```
$ai_total_cost_usd
```
onto
```
$ai_trace
```
so don't include it in rollups or you'll double-count.
Always set a time range. Cost queries without one scan the full events table.

$ai_total_cost_usd

is set at ingestion via one of three paths (passthrough, custom pricing, automatic lookup). When a cost looks wrong, read

$ai_cost_model_source

first — see cost sources for the precedence rules and a diagnostic query.

Cache-hit math depends on whether the provider reports cache tokens inclusively or exclusively of

$ai_input_tokens

. Always branch on the per-event

$ai_cache_reporting_exclusive

flag, never on provider name — see cache accounting for the exclusive-vs-inclusive formula.

distinct_id

is the canonical user dimension. Customers often attach custom properties (

feature

tenant_id

workflow_name

) — discover them with

posthog:read-data-schema

before grouping. Don't guess names.

以下三条规则可以解决大多数常见问题：

汇总时应对
$ai_total_cost_usd
求和，绝不要对各组成项求和。组成项会漏掉请求和 Web Search 费用。UI 的成本单元格会对
```
event IN ('$ai_generation', '$ai_embedding')
```
上的
```
$ai_total_cost_usd
```
求和；请保持一致。完整的 Schema 和原理说明见成本属性。
成本查询中始终同时包含
$ai_generation
和
$ai_embedding
，除非能明确证明项目不使用 Embedding——遗漏它们会在无提示的情况下低估成本。
```
$ai_trace
```
和
```
$ai_span
```
不承载汇总成本；某些 SDK 封装会将
```
$ai_total_cost_usd
```
重复写入
```
$ai_trace
```
，因此不要在汇总中包含它，否则会重复计算。
始终设置时间范围。 未设置时间范围的成本查询会扫描整个 events 表。

$ai_total_cost_usd

会在摄取时通过三种路径之一设置：passthrough、custom pricing 或 automatic lookup。当成本看起来有误时，首先检查

$ai_cost_model_source

——优先级规则和诊断查询见成本来源。

cache-hit 的计算方式取决于 Provider 报告 Cache Token 时，是将其包含在

$ai_input_tokens

中，还是与其分开报告。始终根据每个 Event 的

$ai_cache_reporting_exclusive

标志分支处理，绝不要根据 Provider 名称判断——exclusive 与 inclusive 的计算公式见缓存核算。

distinct_id

是标准的用户维度。客户通常会附加自定义属性（

feature

、

tenant_id

、

workflow_name

）——分组前请使用

posthog:read-data-schema

查找这些属性。不要猜测属性名。

Workflow: total spend in a window

工作流：时间窗口内的总支出

sql

posthog:execute-sql
SELECT round(sum(toFloat(properties.$ai_total_cost_usd)), 4) AS total_cost_usd
FROM events
WHERE event IN ('$ai_generation', '$ai_embedding')
    AND timestamp >= now() - INTERVAL 30 DAY

sql

posthog:execute-sql
SELECT round(sum(toFloat(properties.$ai_total_cost_usd)), 4) AS total_cost_usd
FROM events
WHERE event IN ('$ai_generation', '$ai_embedding')
    AND timestamp >= now() - INTERVAL 30 DAY

Workflow: cost breakdowns

工作流：成本拆分

Every cost question is a variation of the same template — group by a dimension, aggregate

$ai_total_cost_usd

. See breakdown patterns for ready-to-run recipes:

Cost over time (daily)
Cost by model
Cost by user (top spenders)
Cost by trace (top expensive traces)
Cost by custom dimension
Cost-per-call distribution
Input vs output vs cache economics

每个成本问题都是同一模板的变体——按某个维度分组，并聚合

$ai_total_cost_usd

。可直接运行的配方见拆分模式：

随时间变化的成本（按日）
按 Model 划分的成本
按用户划分的成本（支出最高用户）
按 Trace 划分的成本（成本最高的 Trace）
按自定义维度划分的成本
每次调用成本分布
输入、输出与 Cache 成本对比

Workflow: inspect a single trace's cost

工作流：检查单个 Trace 的成本

When the user pastes a trace URL and asks about its cost, fetch the trace and surface the per-event breakdown:

json

posthog:query-llm-trace
{ "traceId": "<trace_id>", "dateRange": {"date_from": "-30d"} }

Sum

$ai_total_cost_usd

across the returned events, grouped by span name or model, to show which step(s) drove the cost. The trace response already includes

totalCost

as a convenience.

当用户粘贴 Trace URL 并询问其成本时，获取该 Trace 并展示逐 Event 拆分：

json

posthog:query-llm-trace
{ "traceId": "<trace_id>", "dateRange": {"date_from": "-30d"} }

对返回 Event 中的

$ai_total_cost_usd

求和，并按 Span 名称或 Model 分组，以显示哪些步骤构成了成本。Trace 响应中已经包含便捷字段

totalCost

。

Workflow: debug a cost regression

工作流：调试成本回归

"Our LLM bill jumped — why?" is almost always one of: more calls, bigger prompts, a new model, or a change in cache-hit rate. Work through them in order — see regression debugging for the 5-step playbook.

“我们的 LLM 账单涨了——为什么？”几乎总是以下原因之一：调用次数增加、Prompt 更大、切换到新 Model，或 Cache 命中率发生变化。请按顺序逐一排查——成本激增的 5 步排查手册见回归调试。

Workflow: materialize as an insight, dashboard, or alert

工作流：物化为 Insight、Dashboard 或 Alert

After ad-hoc queries answer the question, persist them as insights, bundle into a dashboard, or wire up alerts. See materializing for ready-to-run JSON for

posthog:insight-create

posthog:dashboard-create

, and

posthog:alert-create

完成临时查询并回答问题后，将其持久化为 Insight、汇总到 Dashboard，或接入 Alert。可直接运行的

posthog:insight-create

、

posthog:dashboard-create

和

posthog:alert-create

JSON 见物化。

Constructing UI links

构建 UI 链接

Never hand-write

https://app.posthog.com/...

links. That host drops the region and the project prefix, so the user is redirected to login instead of the page you meant.

Prefer the canonical URL the tool returns.
```
query-llm-traces-list
```
and
```
query-llm-trace
```
return
```
_posthogUrl
```
— surface that value. For a single trace, append
```
?timestamp=<url_encoded_iso>
```
(the trace's earliest event time) to that URL; the returned link carries no timestamp, and without one the trace page scans from a fixed early date instead of the ten-minute window around the trace.

Otherwise build the link with
generate-app-url
. It resolves the correct region host and

/project/<id>/

prefix (e.g.

https://us.posthog.com/project/2/ai-observability/traces

). Pass concrete ids via

params

, never inline them into the path.

Dashboard:

generate-app-url {url: "/ai-observability/dashboard"}

Traces list (sort by cost):

generate-app-url {url: "/ai-observability/traces"}

Generations list:

generate-app-url {url: "/ai-observability/generations"}

Users list (per-user cost):

generate-app-url {url: "/ai-observability/users"}

Single trace:

generate-app-url {url: "/ai-observability/traces/{id}", params: {id: "<trace_id>"}}

generate-app-url

cannot express query params, so append the

?timestamp=<url_encoded_iso>

described above to a single-trace link yourself.

Always surface a UI link so the user can verify visually.

绝不手动编写

https://app.posthog.com/...

链接。该 Host 会丢失 Region 和 Project 前缀，导致用户被重定向到登录页，而不是目标页面。

优先使用工具返回的标准 URL。
```
query-llm-traces-list
```
和
```
query-llm-trace
```
会返回
```
_posthogUrl
```
——请直接提供该值。对于单个 Trace，在该 URL 后追加
```
?timestamp=<url_encoded_iso>
```
（Trace 中最早 Event 的时间）；返回的链接不包含时间戳，如果不追加，Trace 页面会从固定的早期日期开始扫描，而不是扫描 Trace 周围的十分钟窗口。

否则，使用
generate-app-url
构建链接。它会解析正确的 Region Host 和

/project/<id>/

前缀（例如

https://us.posthog.com/project/2/ai-observability/traces

）。通过

params

传递具体 ID，绝不要直接内联到路径中。

Dashboard：

generate-app-url {url: "/ai-observability/dashboard"}

Trace 列表（按成本排序）：

generate-app-url {url: "/ai-observability/traces"}

Generation 列表：

generate-app-url {url: "/ai-observability/generations"}

用户列表（单用户成本）：

generate-app-url {url: "/ai-observability/users"}

单个 Trace：

generate-app-url {url: "/ai-observability/traces/{id}", params: {id: "<trace_id>"}}

generate-app-url

无法表达查询参数，因此请自行按上述方式为单个 Trace 链接追加

?timestamp=<url_encoded_iso>

。

始终提供 UI 链接，以便用户通过可视化界面进行核验。

Keeping this skill current

保持本 Skill 与时俱进

Provider reporting behavior (which tokens are inclusive vs exclusive, which costs show up where) shifts over time and can differ between SDK versions for the same provider. To avoid rot:

Branch on event-level flags (
```
$ai_cache_reporting_exclusive
```
,
```
$ai_cost_model_source
```
) rather than hardcoded provider or model names. Those flags are ingestion's resolved answer for the specific event and are the right source of truth.
```
$ai_total_cost_usd
```
is always authoritative for rollups — prefer it over summing components, which can drift as new cost categories are added.
For anything not covered here (new cost categories, changes to pricing lookup, provider additions), run
```
posthog:docs-search
```
for "calculating costs" or "AI observability" first rather than trusting a hardcoded rule in this file.
If you find this skill contradicting the UI, trust the UI and flag the skill for an update.

Provider 的上报行为（哪些 Token 属于 inclusive、哪些属于 exclusive，以及成本会显示在哪里）会随时间变化，同一 Provider 在不同 SDK 版本中的行为也可能不同。为避免内容过时：

根据 Event 级标志（
```
$ai_cache_reporting_exclusive
```
、
```
$ai_cost_model_source
```
）分支处理，不要硬编码 Provider 或 Model 名称。这些标志是摄取时针对具体 Event 得出的结论，也是正确的事实来源。
对汇总而言，
```
$ai_total_cost_usd
```
始终是权威依据——优先使用它，而不是对各组成项求和，因为随着新的成本类别加入，组成项之和可能产生偏差。
对于此处未涵盖的任何内容（新成本类别、定价查询方式变更、新增 Provider 等），先运行
```
posthog:docs-search
```
搜索“calculating costs”或“AI observability”，不要直接依赖此文件中的硬编码规则。
如果发现本 Skill 与 UI 矛盾，请以 UI 为准，并标记本 Skill 以便更新。

Tips

提示

Always set a time range — cost queries without one scan the full events table
Token, cost, model, and
```
$ai_trace_id
```
properties are on
```
events
```
— but message content (
```
$ai_input
```
/
```
$ai_output_choices
```
) lives only on the
```
posthog.ai_events
```
table; see the traces skill's event reference if you need content alongside cost
Always include
```
$ai_embedding
```
alongside
```
$ai_generation
```
when summing cost; embeddings are cheap per-call but add up at scale
Costs are written at ingestion (see Calculating LLM costs) — if
```
$ai_total_cost_usd
```
is missing or zero, read
```
$ai_cost_model_source
```
first:
```
passthrough
```
means the SDK supplied costs;
```
custom
```
means custom token prices;
```
openrouter
```
/
```
manual
```
mean automatic lookup; missing means the model wasn't matched (unusual custom model, fine-tune). Grep:
```
countIf(properties.$ai_total_cost_usd IS NULL)
```
per
```
(model, source)
```
Custom pricing uses per-token prices, not per-million — if a custom-priced model looks ~1M× too expensive or too cheap, that's almost always the bug
Exclude errored calls from cost totals only when explicitly asked — providers still charge for many error modes, and including them gives the truthful bill
For per-user totals, exclude rows where
```
distinct_id = properties.$ai_trace_id
```
— some SDKs default distinct_id to the trace ID when no user is set
Cost is additive across
```
$ai_generation
```
+
```
$ai_embedding
```
events within a trace; summing on
```
$ai_span
```
gives zero.
```
$ai_trace
```
may carry
```
$ai_total_cost_usd
```
from some SDK wrappers — don't include it in rollups or you'll double-count.
```
$ai_evaluation
```
events also carry cost but are not part of the stock UI rollups; include them only when the user explicitly wants evaluation spend in the total
Cache-hit rate depends on
```
$ai_cache_reporting_exclusive
```
— branch on the event-level flag rather than on provider or model name. Provider behavior and SDK versions drift; the flag is ingestion's resolved answer for that specific event
When answering "why is X expensive?", show the cost and the token split — the user almost always wants to know whether to shrink prompts, shrink outputs, or switch models
Before building a custom dashboard, check whether the stock
```
/ai-observability/dashboard
```
tiles already answer the question — re-creating them is churn
For large tenants, materialize common cost queries as insights and reuse via
```
insight-query
```
; ad-hoc SQL is fine for one-offs but re-running it on every dashboard load is expensive

始终设置时间范围——未设置时间范围的成本查询会扫描整个 events 表
Token、Cost、Model 和
```
$ai_trace_id
```
属性位于
```
events
```
——但消息 content（
```
$ai_input
```
／
```
$ai_output_choices
```
）仅位于
```
posthog.ai_events
```
表；如果需要在成本旁查看内容，请参阅 Trace Skill 的事件参考
汇总成本时，务必在
```
$ai_generation
```
之外同时包含
```
$ai_embedding
```
；Embedding 单次调用成本低，但规模扩大后累计起来也很可观
成本在摄取时写入（见计算 LLM 成本）——如果
```
$ai_total_cost_usd
```
缺失或为零，首先读取
```
$ai_cost_model_source
```
：
```
passthrough
```
表示 SDK 提供了成本；
```
custom
```
表示使用自定义 Token 价格；
```
openrouter
```
／
```
manual
```
表示自动查询；缺失表示未匹配到 Model（通常是不常见的自定义 Model 或微调模型）。按
```
(model, source)
```
分组使用
```
countIf(properties.$ai_total_cost_usd IS NULL)
```
自定义定价采用每 Token价格，而不是每百万 Token 价格——如果自定义定价的 Model 看起来贵了或便宜了约
```
1M×
```
，几乎可以肯定是这个错误
仅当用户明确要求时，才从成本总额中排除错误调用——Provider 对许多错误模式仍会收费，纳入它们才能得到真实账单
计算单用户总额时，排除
```
distinct_id = properties.$ai_trace_id
```
的记录——某些 SDK 在未设置用户时默认将
```
distinct_id
```
设为 Trace ID
在同一个 Trace 内，
```
$ai_generation
```
+
```
$ai_embedding
```
Event 的成本可相加；对
```
$ai_span
```
求和会得到零。
```
$ai_trace
```
可能由某些 SDK 封装写入
```
$ai_total_cost_usd
```
——不要在汇总中包含它，否则会重复计算。
```
$ai_evaluation
```
Event 也承载成本，但不属于 UI 默认汇总范围；只有在用户明确要求将评估支出计入总额时才包含它们
Cache 命中率取决于
```
$ai_cache_reporting_exclusive
```
——应根据 Event 级标志分支处理，而不是根据 Provider 或 Model 名称。Provider 行为和 SDK 版本会发生变化；该标志是摄取时针对具体 Event 得出的结论
回答“为什么 X 很贵？”时，同时展示 Cost 和 Token 拆分——用户通常还想知道应该缩短 Prompt、缩短输出，还是切换 Model
构建自定义 Dashboard 前，先检查默认的
```
/ai-observability/dashboard
```
卡片是否已经能回答问题——重复创建只会制造维护负担
对于大型租户，将常见成本查询物化为 Insight，并通过
```
insight-query
```
复用；临时 SQL 适合一次性使用，但在每次 Dashboard 加载时重新运行成本很高

References

参考文档

cost properties — full property schema, total-cost rationale, event-set rules
cost sources — how costs get set at ingestion plus a diagnostic query
cache accounting — exclusive vs inclusive providers, cache-hit-rate formula
breakdown patterns — SQL recipes for every common breakdown
regression debugging — 5-step playbook for cost spikes
materializing — insight, dashboard, and alert JSON

成本属性——完整的属性 Schema、总成本依据、Event 集合规则
成本来源——成本在摄取时如何设置，以及诊断查询
缓存核算——exclusive 与 inclusive Provider、Cache 命中率公式
拆分模式——常见拆分的 SQL 配方
回归调试——成本激增的 5 步排查手册
物化——Insight、Dashboard 和 Alert 的 JSON