outlier-detection-analysis

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Outlier Detection Analysis Skill

异常值检测分析Skill

Find which field values are statistically correlated with "bad" behavior in any time-series dataset or metric. Bad behavior is defined by a threshold (e.g.,

duration > 500ms

status_code >= 500

memory > 8Gi

error_rate > 0.05

); the skill computes the phi coefficient for every candidate field value and ranks the strongest correlations.

The algorithm is generic over data shape. It applies equally to:

metrics (Prometheus, OTEL, custom) — correlate metric values against tag dimensions
structured logs / events — correlate log-level / status / message-class against attributes
span / trace data — correlate error or duration against span attributes
any dataset with a numeric or categorical "performance" field and one or more candidate dimension fields

This skill orchestrates three OPAL queries:

(Optional) Compute a percentile-based threshold value when the user has not provided one.
Run the phi-coefficient correlation pipeline over the chosen dataset / metric.
Interpret the resulting ranked table for the user.

找出任何时间序列数据集或指标中，哪些字段值与“不良”行为存在统计相关性。不良行为由阈值定义（例如：

duration > 500ms

、

status_code >= 500

、

memory > 8Gi

、

error_rate > 0.05

）；该Skill会计算每个候选字段值的phi系数，并对相关性最强的结果进行排序。

该算法适用于任意数据结构，同样适用于：

指标（Prometheus、OTEL、自定义指标）——将指标值与标签维度进行关联分析
结构化日志/事件——将日志级别/状态/消息类别与属性进行关联分析
链路/追踪数据——将错误或持续时间与链路属性进行关联分析
任何包含数值或分类“性能”字段以及一个或多个候选维度字段的数据集

此Skill编排三个OPAL查询：

（可选）当用户未提供阈值时，计算基于百分位数的阈值。
在选定的数据集/指标上运行phi系数相关性分析流程。
为用户解读生成的排序结果表。

How this skill is used

如何使用此Skill

There are two valid invocation modes; the workflow below works for both.

Standalone. The user directly asks "what attributes correlate with slow / failed / unusual rows", "why is X bad", "what's different about the bad cohort", or similar. The agent loads this skill and runs through Step 1 — Step 5 from scratch.
As a companion to
alert-investigation
. When the broader alert-investigation methodology reaches the point of asking "which attributes / services / hosts / regions correlate with the bad behavior?", the alert-investigation skill should defer the phi-coefficient sub-problem to this skill rather than reinventing it. In that mode, some Step 1 inputs (dataset, time range, scoping filter, named service) are typically already resolved by the outer investigation and you can carry them forward — but you still need to confirm
```
thresholdField
```
,
```
thresholdValue
```
/
```
thresholdOperator
```
, and
```
fieldsToAnalyze
```
, and you still need to run the narrow span-perspective caveat (Step 1) before composing any trace filter.

The phi algorithm itself is the same in both modes. This skill is the canonical place that owns it; do not duplicate the pipeline elsewhere.

有两种有效的调用模式，以下流程适用于两种模式。

独立调用：用户直接询问“哪些属性与缓慢/失败/异常行相关”、“为什么X表现不佳”、“不良群组有什么不同”等类似问题。Agent会加载此Skill并从头执行步骤1至步骤5。
作为
alert-investigation
的配套工具：当更广泛的告警调查方法进入“哪些属性/服务/主机/区域与不良行为相关？”的阶段时，alert-investigation Skill应将phi系数相关的子问题委托给此Skill，而非自行实现。在此模式下，步骤1的部分输入（数据集、时间范围、范围过滤器、指定服务）通常已由外部调查确定，可以直接沿用——但仍需确认
```
thresholdField
```
、
```
thresholdValue
```
/
```
thresholdOperator
```
和
```
fieldsToAnalyze
```
，并且在编写任何追踪过滤器之前，仍需执行步骤1中的窄链路视角注意事项。

两种模式下的phi算法是相同的。此Skill是该算法的标准实现位置，请勿在其他地方重复实现该流程。

Prerequisites — load these skills first

前置条件——先加载这些Skill

You MUST use

generate-opal

skill and relevant references first before writing OPAL. The pipeline templates in this skill rely on

flatten_leaves

statsby

align

case

window

, and

coalesce

semantics that are documented there.

在编写OPAL之前，必须先使用

generate-opal

Skill及相关参考文档。此Skill中的流程模板依赖于

flatten_leaves

、

statsby

、

align

、

case

、

window

和

coalesce

的语义，这些语义在

generate-opal

中有文档说明。

Workflow

工作流程

Step 1 — Resolve required inputs (ask the user when missing)

步骤1——解析必要输入（缺失时询问用户）

You MUST ask the user for any required input you cannot confidently determine from the user message, page context, selection context, or available knowledge-graph / dataset-discovery capabilities. NEVER guess values that affect the cohort definition or scope of the analysis. If the host environment exposes a structured user-question capability, prefer it; otherwise ask in plain text and wait for a reply before continuing.

Input	How to resolve	Ask the user when
Dataset (and optional metric)	Use whatever dataset/metric discovery capability the host agent provides; prefer schemas that match the user's intent	More than one plausible match, or no clear match
Time range ( `timeStart` , `timeEnd` )	Use the time range stated by the user or implied by page / selection context, exactly as given — do not silently widen it. If the user did not specify a range, default to the last 15 minutes for the first run. The phi pipeline is expensive on high-volume datasets, so always start with the smallest range that satisfies the user's question and only widen later (Step 5) if the first run produced no meaningful correlations and the user agrees.	The user implied "over the past X" but the value is ambiguous
`thresholdField` (the "performance" field)	Match against common names — `duration` , `latency` , `response_time` , `request_duration` , `elapsed_time` , `processing_time` , `execution_time` , `bytes` , `response_size` , `request_size` , `status_code` , `status` — among numeric/duration fields in the schema	No numeric field matches and the user did not name one
`thresholdValue` and `thresholdOperator`	Use the user's stated threshold; otherwise run the threshold pipeline (Step 3) at P95 with operator `>`	The percentile or operator is ambiguous (e.g., user says "slow" but field is `status_code` )
`fieldFilter` (optional scoping)	Only include if the user explicitly scoped the analysis ("for the cart service")	Never invent a filter; ask only when the user implied scoping but did not name the field/value
`fieldsToAnalyze` (optional)	Default to schema-driven selection — see `outlier-detection-field-selection` reference	The user requested a specific subset that is unclear

Do not proceed to Step 2 until every required input above is resolved.

对于无法从用户消息、页面上下文、选择上下文或可用的知识图谱/数据集发现能力中确定的必要输入，必须询问用户。切勿猜测会影响群组定义或分析范围的值。如果宿主环境提供结构化的用户提问能力，请优先使用；否则用自然文本询问，等待用户回复后再继续。

输入项	解析方式	需要询问用户的场景
数据集（及可选指标）	使用宿主Agent提供的任何数据集/指标发现能力；优先选择符合用户意图的Schema	存在多个合理匹配项，或无明确匹配项
时间范围（ `timeStart` 、 `timeEnd` ）	使用用户指定的时间范围，或页面/选择上下文隐含的时间范围——严格按照给定值，请勿擅自扩大范围。如果用户未指定范围，首次运行默认使用过去15分钟。phi流程在高容量数据集上运行成本较高，因此始终从满足用户问题的最小范围开始，只有当首次运行未产生有意义的相关性且用户同意时，才在步骤5中扩大范围。	用户隐含“在过去X时间内”但X的值不明确
`thresholdField` （“性能”字段）	匹配常见名称—— `duration` 、 `latency` 、 `response_time` 、 `request_duration` 、 `elapsed_time` 、 `processing_time` 、 `execution_time` 、 `bytes` 、 `response_size` 、 `request_size` 、 `status_code` 、 `status` ——从Schema的数值/持续时间字段中选择	无匹配的数值字段且用户未指定具体字段
`thresholdValue` 和 `thresholdOperator`	使用用户指定的阈值；否则以P95和运算符 `>` 运行阈值流程（步骤3）	百分位数或运算符不明确（例如用户说“缓慢”但字段是 `status_code` ）
`fieldFilter` （可选范围限定）	仅当用户明确限定分析范围时才包含（例如“针对购物车服务”）	切勿自行创建过滤器；仅当用户隐含范围限定但未指定字段/值时询问
`fieldsToAnalyze` （可选）	默认基于Schema选择——参考 `outlier-detection-field-selection` 文档	用户请求的特定子集不明确

在解析完上述所有必要输入之前，请勿进入步骤2。

Narrow caveat — span perspective on trace-error questions

窄范围注意事项——追踪错误问题中的链路视角

This caveat is narrow. It applies when all three of the following are true; if any one is missing, skip it and proceed normally:

The user named a specific service (e.g.
```
redis-result-cache
```
,
```
apiserver
```
) — not a generic noun like "pods", "the cluster", or "my service".
The question is about errors / failures / faults / exceptions / broken behavior on that service. Pure slowness questions do NOT count.
The investigation will run against distributed-trace data (
```
Tracing/Span
```
, OTEL spans). Metric, log, pod-health, and any other non-span investigations do NOT count.

When all three hold, the user's question is genuinely ambiguous: a request through a distributed system produces span records on both the receiver and the caller, so "errors on

" can mean (A) errors X experienced (server-side spans where

service_name = X

and

error = true

), (B) errors X caused for callers (client-side spans from OTHER services whose target was X), or (C) the full-trace view (all spans in traces that contained an error involving X). These three cohorts produce very different phi-correlation results, and silently picking (A) — which is what

service_name = "X"

filtering does — hides upstream root causes.

When all three preconditions hold, you MUST stop and ask the user to pick A / B / C before doing any other tool call: no query execution, no dataset/metric/schema lookups, no exploratory queries. Only after they answer may you compose the filter and continue. If they have already specified the perspective in their question, no clarification needed. When the user picks, also include

span_kind

(or

span_type

) in

scalarFields[]

so the inbound/outbound distinction is preserved in the phi output.

For metric, log, or non-trace investigations: this caveat does not apply — proceed normally.

此注意事项适用范围较窄。仅当以下三个条件同时满足时适用；若缺少任意一个，则跳过此注意事项并正常执行：

用户指定了特定服务（例如
```
redis-result-cache
```
、
```
apiserver
```
）——而非通用名词如“Pod”、“集群”或“我的服务”。
问题是关于该服务的错误/故障/失效/异常/异常行为。单纯的性能缓慢问题不适用。
调查将针对分布式追踪数据（
```
Tracing/Span
```
、OTEL链路）。指标、日志、Pod健康状态及其他非链路调查不适用。

当三个条件都满足时，用户的问题存在歧义：分布式系统中的请求会在接收方和调用方都生成链路记录，因此“服务X上的错误”可能指(A) X自身遇到的错误（

service_name = X

且

error = true

的服务端链路），(B) X给调用方造成的错误（来自其他服务、目标为X的客户端链路），或(C)全追踪视图（所有包含涉及X的错误的追踪中的链路）。这三个群组的phi相关性结果差异很大，若默认选择(A)（即使用

service_name = "X"

过滤）会隐藏上游根因。

当三个前提条件都满足时，必须停止操作并询问用户选择A/B/C，之后才能进行任何其他工具调用：不得执行查询、不得查找数据集/指标/Schema、不得执行探索性查询。只有在用户回复后，才能编写过滤器并继续。如果用户已在问题中指定视角，则无需澄清。当用户做出选择后，需将

span_kind

（或

span_type

）加入

scalarFields[]

，以便在phi输出中保留入站/出站的区分。

对于指标、日志或非追踪调查：此注意事项不适用——正常执行即可。

Step 2 — Pick the candidate fields to analyze

步骤2——选择要分析的候选字段

Load the outlier-detection-field-selection reference and apply its rules to the dataset schema. Categorize each selected field as scalar (string/number/boolean) or complex (Object/Array). You will need both lists in Step 4.

加载outlier-detection-field-selection参考文档，并将其规则应用于数据集Schema。将每个选中的字段分类为标量（字符串/数字/布尔值）或复杂类型（对象/数组）。步骤4中需要这两个列表。

Step 3 — Compute the threshold (only when not provided)

步骤3——计算阈值（仅当用户未提供时）

If the user did not give an explicit

thresholdValue

, load the outlier-detection-threshold reference and run that OPAL pipeline first. Use the requested percentile (default P95) as the recommended threshold value, then ask the user to confirm before running the correlation analysis.

如果用户未提供明确的

thresholdValue

，加载outlier-detection-threshold参考文档并先运行该OPAL流程。使用请求的百分位数（默认P95）作为推荐阈值，然后在运行相关性分析前请用户确认。

Step 4 — Build and run the correlation pipeline

步骤4——构建并运行相关性分析流程

Load the outlier-detection-pipeline reference and assemble the pipeline using the templates verbatim, substituting only the placeholders. Execute it with whatever OPAL execution capability the host agent provides.

The pipeline is non-trivial; do NOT improvise. The phi formula

((a*d) - (b*c)) / sqrt((a+b)*(c+d)*(a+c)*(b+d))

and the cohort/window construction must match the templates verbatim.

Two rules the template enforces — read outlier-detection-pipeline carefully before composing the OPAL:

Field categorization. Use the rules in outlier-detection-field-selection to split the chosen fields into
```
scalarFields[]
```
(string/number/bool) and
```
complexFields[]
```
(Object/Array). Never put a complex field into the scalar bundle — the
```
make_object("name":string(field), …)
```
step would coerce the entire object to a single string. Scalars are packed into
```
_scalar_fields_obj
```
and flattened once; complex fields each get their own empty-object guard and their own
```
flatten_leaves
```
.
Always include a scoping
filter
when the user's question implies one (e.g.
```
filter service_name = "checkout"
```
,
```
filter error = true
```
). Never invent a filter when the user did not imply one — ask instead.

加载outlier-detection-pipeline参考文档，使用模板逐字组装流程，仅替换占位符。使用宿主Agent提供的OPAL执行能力运行该流程。

该流程较为复杂；请勿自行修改。phi公式

((a*d) - (b*c)) / sqrt((a+b)*(c+d)*(a+c)*(b+d))

以及群组/窗口的构建必须严格匹配模板。

模板强制执行两条规则——在编写OPAL前请仔细阅读outlier-detection-pipeline：

字段分类：使用outlier-detection-field-selection中的规则将所选字段分为
```
scalarFields[]
```
（字符串/数字/布尔值）和
```
complexFields[]
```
（对象/数组）。切勿将复杂字段放入标量集合——
```
make_object("name":string(field), …)
```
步骤会将整个对象强制转换为单个字符串。标量字段会被打包到
```
_scalar_fields_obj
```
并扁平化一次；每个复杂字段都有自己的空对象保护和
```
flatten_leaves
```
处理。
始终包含范围限定
filter
：当用户的问题隐含范围时（例如
```
filter service_name = "checkout"
```
、
```
filter error = true
```
）。当用户未隐含范围时，切勿自行创建过滤器——应询问用户。

Step 5 — Interpret the results

步骤5——解读结果

The pipeline returns one row per

(field, value)

candidate, sorted by

phi

descending and limited to 50 rows. Columns:

```
attribute
```
— the field name / JSON path (e.g.,
```
service.name
```
,
```
db.statement
```
)
```
value
```
— the specific value (e.g.,
```
checkout-service
```
,
```
TimeoutException
```
)
```
bad_count
```
,
```
good_count
```
— counts of rows in each cohort with this value
```
total_bad
```
,
```
total_good
```
— totals across the entire query

frequency_in_bad_cohort

frequency_in_good_cohort

— fractions (0..1)

```
phi
```
— phi coefficient in [-1, 1]

Load the outlier-detection-interpretation reference for phi/bad% bands, root-cause framing, and the recommended response structure.

流程返回每行对应一个

(field, value)

候选，按

phi

降序排序，最多返回50行。列包括：

```
attribute
```
——字段名称/JSON路径（例如
```
service.name
```
、
```
db.statement
```
）
```
value
```
——具体值（例如
```
checkout-service
```
、
```
TimeoutException
```
）
```
bad_count
```
、
```
good_count
```
——该值在每个群组中的行数
```
total_bad
```
、
```
total_good
```
——整个查询的总行数

frequency_in_bad_cohort

、

frequency_in_good_cohort

——占比（0..1）

```
phi
```
——phi系数，范围[-1, 1]

加载outlier-detection-interpretation参考文档，了解phi/不良占比区间、根因框架以及推荐的回复结构。

When the result set is empty or weak

结果集为空或相关性较弱时

If the pipeline returned no rows, or every

phi

is below ~0.1 (no meaningful correlation), do not silently re-run with a wider window. Ask the user whether they want to:

Widen the time range (e.g. from 15 min to 1 hour) to get more samples.
Adjust the threshold (e.g. P95 → P90) to enlarge the bad cohort.
Add or remove fields from the analysis set.
Add a scoping filter (e.g. limit to a single service or environment).

Summarize what was tried (range, threshold, fields, filter) and let the user pick before re-running. Never widen the range silently; the original range was the user's stated intent.

如果流程未返回任何行，或所有

phi

值都低于约0.1（无有意义的相关性），请勿擅自扩大范围重新运行。询问用户是否希望：

扩大时间范围（例如从15分钟改为1小时）以获取更多样本。
调整阈值（例如从P95改为P90）以扩大不良群组。
添加或移除分析字段。
添加范围限定过滤器（例如限定为单个服务或环境）。

总结已尝试的操作（范围、阈值、字段、过滤器），让用户选择后再重新运行。切勿擅自扩大范围；原始范围是用户明确的需求。

Things to NOT do

禁止操作

Do not invent dataset IDs, field names, percentile values, threshold operators, or filter values. Ask the user.
Do not regenerate the phi formula or rebuild the cohort/window logic from scratch — copy the template.
Do not post-process or rename the query result columns. The pipeline already produces the final shape; read the rows directly.
Do not include
```
validFromField
```
/
```
validToField
```
columns in the analysis field list — they are temporal columns, not categorical attributes.
Do not include hidden, const, metric-aggregation, or
```
_
```
-prefixed fields.
Do not analyze more than 10 fields at once — high cardinality kills query performance.
Do not skip the empty-object / empty-array guard
```
make_col
```
step before
```
flatten_leaves
```
for complex fields, otherwise null/empty rows drop out and bias the cohort counts.
Do not run on a raw spans / logs dataset without a scoping
```
filter
```
when the user's question implies one. If no scope is implied, ask the user.
When (and only when) the narrow trace-perspective caveat in Step 1 applies, do not equate "errors on service X" with
```
filter service_name = "X" and error = true
```
and do not omit
```
span_kind
```
/
```
span_type
```
from the analyzed fields — both are easy ways to surface the wrong side of a distributed call. This caveat does not apply to metric, log, or non-trace investigations.

请勿自行生成数据集ID、字段名称、百分位数值、阈值运算符或过滤器值。请询问用户。
请勿重新生成phi公式或从头构建群组/窗口逻辑——直接复制模板。
请勿对查询结果列进行后处理或重命名。流程已生成最终格式；直接读取行数据即可。
请勿在分析字段列表中包含
```
validFromField
```
/
```
validToField
```
列——这些是时间列，不是分类属性。
请勿包含隐藏字段、常量字段、指标聚合字段或下划线前缀的字段。
请勿同时分析超过10个字段——高基数会严重影响查询性能。
对于复杂字段，在
```
flatten_leaves
```
之前请勿省略空对象/空数组保护的
```
make_col
```
步骤，否则空行或null行会被过滤掉，导致群组计数出现偏差。
当用户的问题隐含范围时，请勿在未添加范围限定
```
filter
```
的情况下直接在原始链路/日志数据集上运行。如果未隐含范围，请询问用户。
当（且仅当）步骤1中的窄链路视角注意事项适用时，请勿将“服务X上的错误”等同于
```
filter service_name = "X" and error = true
```
，也请勿将
```
span_kind
```
/
```
span_type
```
从分析字段中排除——这两种做法都会轻易掩盖分布式调用的错误方。此注意事项不适用于指标、日志或非追踪调查。

Output format

输出格式

Present the results as a ranked markdown table followed by a short root-cause narrative. See

outlier-detection-interpretation

for the exact format and example.

将结果以排序后的Markdown表格形式呈现，后面附上简短的根因分析说明。请参考

outlier-detection-interpretation

获取准确格式和示例。