authoring-log-alerts

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Authoring log alerts

创建日志告警

Authoring an alert is a measurement problem, not a guessing problem. You are not trying to be exhaustive — you are trying to land thresholds that fire 0–3 times per week on real production patterns, on services that matter.

创建告警是一个量化问题，而非猜测问题。无需追求面面俱到，只需针对重要服务设定符合真实生产模式的阈值，确保每周触发0-3次告警即可。

When to use this skill

何时使用此技能

The user asks to "set up alerts" / "suggest alerts" for their project.
The user wants to evaluate whether a service is producing alertable signal.
The user has just enabled log alerting and wants a starter set.

用户要求为其项目“设置告警”/“建议告警”时。
用户希望评估某项服务是否产生可告警的信号时。
用户刚启用日志告警功能，需要一套初始配置时。

When not to use this skill

何时不使用此技能

Tuning an alert that already exists — that's a different job (use
```
posthog:logs-alerts-events-list
```
to inspect fire/resolve cadence and
```
posthog:logs-alerts-partial-update
```
to adjust).
Investigating an active incident — pull rows with
```
posthog:query-logs
```
, don't author an alert mid-incident.

调整已存在的告警——这是另一项任务（使用
```
posthog:logs-alerts-events-list
```
查看告警触发/恢复频率，使用
```
posthog:logs-alerts-partial-update
```
进行调整）。
调查活跃事件——使用
```
posthog:query-logs
```
提取日志行，不要在事件处理过程中创建告警。

Tools

工具

Tool	Job	Where it fits
`posthog:logs-services`	Top-25 services in window with log_count, error_count, error_rate, sparkline.	Step 1 — triage.
`posthog:logs-attributes-list` / `posthog:logs-attribute-values-list`	Discover keys/values for narrower filters.	Step 2, optional.
`posthog:logs-count-ranges`	Adaptive time-bucketed counts for a filter.	Step 3 — baseline.
`posthog:logs-alerts-simulate-create`	Replay a draft config against `-7d` history with full state machine.	Step 4 — validate.
`posthog:logs-alerts-create`	Persist the alert.	Step 5 — ship.
`posthog:logs-alerts-destinations-create`	Wire the alert to Slack or webhook.	Step 5 — ship.

Do not call

posthog:query-logs

during authoring. You need distributions, not rows. Reserve

posthog:query-logs

for the very end if the user asks "show me a sample of what would have fired" —

limit: 10

is plenty.

工具	用途	适用步骤
`posthog:logs-services`	返回指定时间窗口内的前25个服务，包含log_count、error_count、error_rate和趋势图。	步骤1 — 分类排查。
`posthog:logs-attributes-list` / `posthog:logs-attribute-values-list`	发现用于缩小过滤范围的键/值。	步骤2（可选）。
`posthog:logs-count-ranges`	针对过滤条件返回自适应时间桶的计数数据。	步骤3 — 基线分析。
`posthog:logs-alerts-simulate-create`	使用完整状态机，基于过去7天的历史数据回放草稿配置。	步骤4 — 验证。
`posthog:logs-alerts-create`	保存告警配置。	步骤5 — 上线。
`posthog:logs-alerts-destinations-create`	将告警关联至Slack或webhook。	步骤5 — 上线。

创建告警过程中不要调用

posthog:query-logs

。你需要的是分布数据，而非日志行。仅当用户询问“展示一些本应触发告警的示例”时，才在最后调用该工具——设置

limit: 10

足够。

Workflow

工作流程

1. Triage — pick candidate services

1. 分类排查——挑选候选服务

Call

posthog:logs-services

for the last 24h with no filters. The response is capped at 25 services and includes a sparkline, so it is small and bounded.

A service is a candidate when both are true:

```
log_count
```
is non-trivial (≥ ~1k in 24h — quieter services produce too little signal to alert on).
```
error_rate
```
is non-zero, or the user has named the service explicitly.

Skip services with high volume but

error_rate == 0

unless the user wants a volume-shape alert (e.g. "warn me if api-gateway suddenly stops producing logs"). Volume-floor alerts use

threshold_operator: below

and need different reasoning — see references/volume-floor-alerts.md.

If the user names a service, treat it as a candidate even without error signal.

调用

posthog:logs-services

获取过去24小时无过滤条件的数据。响应结果最多包含25个服务，且带有趋势图，数据量小且可控。

满足以下两个条件的服务可作为候选：

```
log_count
```
达到一定规模（24小时内≥约1000条——流量过低的服务信号不足，不适合告警）。
```
error_rate
```
不为0，或用户明确指定该服务。

除非用户需要流量形态告警（例如“如果api-gateway突然停止生成日志，请提醒我”），否则跳过流量高但

error_rate == 0

的服务。流量下限告警需使用

threshold_operator: below

，且逻辑不同——详见references/volume-floor-alerts.md。

如果用户指定了某服务，即使没有错误信号也将其视为候选。

2. (Optional) Narrow the filter

2.（可选）缩小过滤范围

If a service has many error sub-types, an alert on "all errors" is usually too broad. Use

posthog:logs-attributes-list

(try

attribute_type: log

) and

posthog:logs-attribute-values-list

to find a discriminator — common ones are

http.status_code

error.type

k8s.container.name

. Add the narrowing filter to your draft.

Keep it simple: one severity filter + one or two attribute filters is plenty. Multi-clause filters are harder to reason about and rarely improve precision.

如果某服务存在多种错误子类型，针对“所有错误”的告警通常范围过宽。使用

posthog:logs-attributes-list

（尝试设置

attribute_type: log

）和

posthog:logs-attribute-values-list

找到区分条件——常见的有

http.status_code

、

error.type

、

k8s.container.name

。将缩小后的过滤条件添加至草稿。

保持简洁：一个严重性过滤条件+一到两个属性过滤条件足够。多条件过滤更难理解，且很少能提升精度。

3. Baseline — characterise the candidate over 7 days

3. 基线分析——分析候选服务过去7天的特征

Call

posthog:logs-count-ranges

with the candidate's filters,

dateRange: { date_from: "-7d" }

, and

targetBuckets: 24

(one bucket ≈ 7h). The response gives you bucket counts.

Do not eyeball the percentiles or scale the threshold to the alert window manually. Pipe the count-ranges response into the helper script:

bash

echo '<count-ranges JSON>' | python3 scripts/baseline_stats.py --window-minutes 5

The script returns:

json

{
  "n_buckets": 12,
  "bucket_minutes": 420.0,
  "alert_window_minutes": 5,
  "stats": { "p50": 12.0, "p95": 71.25, "p99": 126.25, "max": 140 },
  "suggested_threshold_count": 5,
  "rationale": "max(p99=126.25, median*3=36.0, floor=5) scaled from 420m bucket to 5m window",
  "health": []
}

Use

suggested_threshold_count

as your starting threshold. Read

health

`health` flag	What it means	What to do
`sparse:N_of_M_buckets`	Too few non-empty buckets for a 7d baseline.	Widen filter, extend to `-30d` , or skip.
`empty`	All buckets are zero.	Skip — no signal.
`spiky`	`max` is 10×+ `p95` .	Count-threshold alerts work well. Proceed.
`flat`	`p95` ≈ `p50` .	Be cautious — either no incidents in lookback, or the metric is too smooth. Try a longer lookback or skip.
`[]` (empty)	Healthy distribution.	Proceed.

调用

posthog:logs-count-ranges

，传入候选服务的过滤条件、

dateRange: { date_from: "-7d" }

和

targetBuckets: 24

（每个时间桶≈7小时）。响应结果会给出各时间桶的计数。

**不要手动估算百分位数或按告警窗口缩放阈值。**将count-ranges的响应传入辅助脚本：

bash

echo '<count-ranges JSON>' | python3 scripts/baseline_stats.py --window-minutes 5

脚本返回结果示例：

json

{
  "n_buckets": 12,
  "bucket_minutes": 420.0,
  "alert_window_minutes": 5,
  "stats": { "p50": 12.0, "p95": 71.25, "p99": 126.25, "max": 140 },
  "suggested_threshold_count": 5,
  "rationale": "max(p99=126.25, median*3=36.0, floor=5) scaled from 420m bucket to 5m window",
  "health": []
}

将

suggested_threshold_count

作为初始阈值。查看

health

字段：

`health` 标识	含义	处理方式
`sparse:N_of_M_buckets`	7天基线中非空时间桶过少。	扩大过滤范围、延长至 `-30d` 或跳过该服务。
`empty`	所有时间桶计数均为0。	跳过——无有效信号。
`spiky`	`max` 是 `p95` 的10倍以上。	基于计数的阈值告警效果良好，继续执行。
`flat`	`p95` ≈ `p50` 。	需谨慎——要么回溯期内无事件，要么指标过于平稳。尝试延长回溯期或跳过。
`[]` （空）	数据分布健康。	继续执行。

4. Draft and simulate

4. 草拟并模拟

Pick a starter draft from these defaults — see references/threshold-defaults.md for the reasoning:

Setting	Default	Notes
`threshold_count`	`suggested_threshold_count` from the script	Already scaled to the alert window.
`threshold_operator`	`above`	Use `below` only for volume-floor alerts.
`window_minutes`	`5`	Allowed: 5, 10, 15, 30, 60. Must match what you passed to the script.
`evaluation_periods`	`3`	M in N-of-M.
`datapoints_to_alarm`	`2`	N in N-of-M. 2-of-3 reduces flap from a single noisy bucket.
`cooldown_minutes`	`30`	Minimum time between repeat fires.

Call

posthog:logs-alerts-simulate-create

with these settings and

date_from: "-7d"

. The response gives you

fire_count

and

resolve_count

从以下默认配置中选择初始草稿——详见references/threshold-defaults.md中的设计思路：

设置项	默认值	说明
`threshold_count`	脚本返回的 `suggested_threshold_count`	已按告警窗口缩放。
`threshold_operator`	`above`	仅流量下限告警使用 `below` 。
`window_minutes`	`5`	允许值：5、10、15、30、60。必须与传入脚本的参数一致。
`evaluation_periods`	`3`	N-of-M中的M值。
`datapoints_to_alarm`	`2`	N-of-M中的N值。2-of-3可减少单个噪声时间桶导致的告警波动。
`cooldown_minutes`	`30`	重复触发告警的最小间隔时间。

调用

posthog:logs-alerts-simulate-create

，传入上述配置和

date_from: "-7d"

。响应结果会给出

fire_count

和

resolve_count

。

5. Iterate — three rounds, then ship or skip

5. 迭代——最多三轮，然后上线或跳过

Target:

fire_count

between 0 and ~3 over

-7d

. If outside the band:

Outcome	Adjustment
`fire_count` = 0 over 7d and the baseline was spiky	Lower `threshold_count` toward `stats.p95` from the script, or drop to 1-of-2.
`fire_count` = 0 and the baseline was flat	The service has no alertable signal. Skip it; log why.
`fire_count` > 5	Raise `threshold_count` toward `stats.max` from the script, or move to 3-of-5 for a smoother window.
`fire_count` is fine but resolve_count never matches fire_count	Cooldown is too long, or the underlying state is genuinely sticky. Acceptable for now.

When adjusting the threshold, read values from the script's
stats
block — never recompute percentiles by hand.

Cap iteration at 3 simulate calls per candidate. If you can't land in the band in 3 rounds, the metric is wrong — either the filter is too broad, the window is wrong, or the service genuinely doesn't have a threshold-shape signal. Note it and move on.

目标：过去7天内

fire_count

在0到约3之间。若超出范围：

结果	调整方案
7天内 `fire_count` = 0 且基线呈尖峰状	将 `threshold_count` 调低至脚本返回的 `stats.p95` 附近，或改为1-of-2规则。
`fire_count` = 0 且基线平稳	该服务无有效告警信号，跳过并记录原因。
`fire_count` > 5	将 `threshold_count` 调高至脚本返回的 `stats.max` 附近，或改为3-of-5规则以平滑窗口。
`fire_count` 正常但 `resolve_count` 与 `fire_count` 不匹配	冷却时间过长，或底层状态持续异常。暂时可接受。

调整阈值时，务必从脚本的
stats
块中读取数值——切勿手动重新计算百分位数。

每个候选服务最多进行3次simulate调用。若3轮后仍无法达到目标范围，说明指标存在问题——要么过滤范围过宽，要么窗口设置错误，要么该服务确实没有符合阈值形态的信号。记录问题并继续处理其他服务。

6. Ship — create + attach destination

6. 上线——创建告警+关联通知目标

Once a draft simulates cleanly:

Call
```
posthog:logs-alerts-create
```
with the validated config. Use a name like
```
<service> error rate (auto)
```
so the user can see at a glance which alerts came from this skill.
Call
```
posthog:logs-alerts-destinations-create
```
to wire it to a notification target. An alert with no destination is silent. Always confirm the channel name or webhook URL with the user before attaching — never wire an auto-generated alert to a production channel without explicit confirmation. If the user is unsure, suggest a low-traffic testing channel for the first few alerts.

If the user wants alerts created in

enabled: false

state for review-then-flip, pass

enabled: false

-create

and tell them how many drafts you produced.

草稿模拟验证通过后：

调用
```
posthog:logs-alerts-create
```
传入验证后的配置。命名格式建议为
```
<service> error rate (auto)
```
，方便用户一眼识别哪些告警是通过此技能创建的。
调用
```
posthog:logs-alerts-destinations-create
```
关联通知目标。**无通知目标的告警是静默的。**关联前务必与用户确认渠道名称或webhook URL——切勿未经明确确认就将自动生成的告警关联至生产渠道。若用户不确定，建议先关联低流量的测试渠道。

如果用户希望创建的告警处于

enabled: false

状态，以便先审核再启用，可在

-create

接口中传入

enabled: false

，并告知用户已生成的草稿数量。

Filter shape — required

过滤条件格式——必填

The

filters

field on

posthog:logs-alerts-create

takes a subset of

LogsViewerFilters

and must contain at least one of:

severityLevels

— list of

["trace","debug","info","warn","error","fatal"]

```
serviceNames
```
— list of service name strings
```
filterGroup
```
— property filter group

The same shape goes into

posthog:logs-alerts-simulate-create

filters

field. Match the simulate filters to the alert filters exactly — otherwise the simulation is testing a different alert than the one you ship.

Example minimum:

json

{
  "severityLevels": ["error", "fatal"],
  "serviceNames": ["api-gateway"]
}

posthog:logs-alerts-create

的

filters

字段接受

LogsViewerFilters

的子集，且必须包含至少一项：

severityLevels

—

["trace","debug","info","warn","error","fatal"]

列表

```
serviceNames
```
— 服务名称字符串列表
```
filterGroup
```
— 属性过滤组

posthog:logs-alerts-simulate-create

的

filters

字段需使用相同格式。模拟的过滤条件必须与告警的过滤条件完全一致——否则模拟测试的是与最终上线不同的告警。

最简示例：

json

{
  "severityLevels": ["error", "fatal"],
  "serviceNames": ["api-gateway"]
}

Token-economy rules

调用次数规则

One
```
posthog:logs-services
```
call at the start, not per-candidate.
One
```
posthog:logs-count-ranges
```
call per candidate at
```
targetBuckets: 24
```
. Don't go above 30 during authoring.
≤ 3
```
posthog:logs-alerts-simulate-create
```
calls per candidate.
Zero
```
posthog:query-logs
```
calls during the authoring loop.
Prefer reporting a small set of well-validated alerts over a long list of unvalidated drafts.

仅在开始时调用一次
```
posthog:logs-services
```
，而非每个候选服务都调用。
每个候选服务调用一次
```
posthog:logs-count-ranges
```
，设置
```
targetBuckets: 24
```
。创建告警过程中不要超过30。
每个候选服务最多调用3次
```
posthog:logs-alerts-simulate-create
```
。
创建告警流程中不调用
```
posthog:query-logs
```
。
优先报告少量经过充分验证的告警，而非大量未验证的草稿。

Output

输出内容

Report what you did, in this shape:

For each shipped alert: name, filters, threshold, simulated fire_count over 7d, destination.
For each skipped candidate: service name + why (flat baseline, can't land threshold, low volume).
Total simulate calls made, total alerts created.

The user should be able to read this and decide whether to disable any drafts before they go live.

按以下格式报告操作结果：

每个上线告警：名称、过滤条件、阈值、过去7天模拟触发次数、关联的通知目标。
每个跳过的候选服务：服务名称+原因（基线平稳、无法设定合适阈值、流量过低）。
总模拟调用次数、总创建告警数量。

用户应能通过此报告决定是否在告警生效前禁用某些草稿。