elasticsearch-anomaly-detection

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Elasticsearch Anomaly Detection

Elasticsearch 异常检测

Create, open, and start ML anomaly detection jobs on time-series data. Choose the right count-family detector direction, configure bucket span and time field, wire the datafeed to the correct index, and confirm running state from stats — not from assumptions.

针对时序数据创建、开启并启动ML异常检测任务。选择合适的计数类检测器方向，配置bucket span和时间字段，将数据馈送关联至正确的索引，并通过统计信息确认运行状态——而非主观假设。

Environment Configuration

环境配置

This skill executes Elasticsearch operations through the

elastic

CLI. If the

elastic

CLI is not installed, tell the user what it is needed for. Do not guess credentials, call the HTTP API directly, or attempt other workarounds.

This skill references operations in HTTP-shorthand form (e.g.,

GET /

GET /_cat/indices

GET /{index}/_mapping

GET /{index}/_settings/index.mode

POST /_query

). The Operations table at the end of this document maps each shorthand to the equivalent

elastic

CLI command — always use the CLI rather than calling the HTTP API directly.

Prerequisite: ML anomaly detection requires a Platinum-equivalent license on self-managed clusters. Serverless projects include ML. The caller needs
manage_ml
to create and manage jobs.
Related skill: For interpreting anomaly scores, influencers, and model behavior after a job is running, use
elasticsearch-anomaly-detection-explainer
— not this skill.

本技能通过

elastic

CLI执行Elasticsearch操作。如果未安装

elastic

CLI，请告知用户其用途。请勿猜测凭证、直接调用HTTP API或尝试其他变通方法。

本技能采用HTTP简写形式引用操作（例如：

GET /

、

GET /_cat/indices

、

GET /{index}/_mapping

、

GET /{index}/_settings/index.mode

、

POST /_query

）。本文档末尾的操作表格将每个简写映射为对应的

elastic

CLI命令——请始终使用CLI而非直接调用HTTP API。

前提条件： 自托管集群上的ML异常检测需要白金级等效许可证。Serverless项目包含ML功能。调用者需要拥有
manage_ml
权限才能创建和管理任务。
相关技能： 若需在任务运行后解读异常分数、影响因素和模型行为，请使用
elasticsearch-anomaly-detection-explainer
——而非本技能。

Process

操作流程

Discover the target index and time field. List candidate indices with
```
GET /_cat/indices
```
(pass a pattern when the user names one). Fetch field types for the chosen index with
```
GET /{index}/_mapping
```
. The decision: confirm the index exists, identify the time field (often
```
@timestamp
```
), and verify document volume is sufficient for baseline learning. Never guess index or field names — they vary across deployments.
Choose detector function and direction. Match the user's intent to a count-family detector in
```
analysis_config.detectors
```
:
- Spike, surge, unusual increase in event volume →
```
high_count
```
  (or
```
count
```
  , which flags both directions but is acceptable when the user cares about spikes). Do not use
```
low_count
```
  — it will miss spikes.
- Drop, outage, absence of events, traffic stops →
```
low_count
```
  . Do not use
```
high_count
```
  — it will miss drops and silence.
- Metric deviation (CPU, latency, a numeric field) → mean-family functions (
```
mean
```
  ,
```
high_mean
```
  ,
```
low_mean
```
  ) with
```
field_name
```
  set — only when the user asks about a numeric metric, not raw event volume.
The decision: pick one primary detector whose direction matches the anomaly type. For volume spike/drop questions on document counts, stay in the count family — mean detectors are unsuited to "how many events" questions.

Set immutable job shape before creation. These fields cannot change after

PUT /_ml/anomaly_detectors/{job_id}

```
analysis_config.bucket_span
```
— use the interval the user specifies (e.g.
```
15m
```
for 15-minute buckets). Match the granularity of anomalies they care about; too short is noisy, too long is slow to detect.
```
data_description.time_field
```
— the time field from the mapping (commonly
```
@timestamp
```
).
```
analysis_config.detectors
```
— the function and direction from step 2.

Example job body for a volume-spike detector:

json

{
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [{ "function": "high_count" }]
  },
  "data_description": { "time_field": "@timestamp" }
}

Example for an outage / drop detector:

json

{
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [{ "function": "low_count" }]
  },
  "data_description": { "time_field": "@timestamp" }
}

Create the job. Call
```
PUT /_ml/anomaly_detectors/{job_id}
```
with the job id the user requested (or a descriptive id you propose). The job starts in
```
closed
```
state — creating it does not start analysis.
Create the datafeed. Call
```
PUT /_ml/datafeeds/datafeed-{job_id}
```
immediately after job creation. Set
```
job_id
```
to the same id,
```
indices
```
to the target index (exact name or pattern from step 1), and a query that selects the relevant documents (typically
```
match_all
```
). The datafeed id convention is
```
datafeed-{job_id}
```
.
json
```
{
  "job_id": "{job_id}",
  "indices": ["{index}"],
  "query": { "match_all": {} }
}
```
Open the job, then start the datafeed — in that order. This sequence is mandatory; do not skip or reorder:
1. ```
POST /_ml/anomaly_detectors/{job_id}/_open
```
  — transitions the job to
```
opened
```
  .
2. ```
POST /_ml/datafeeds/datafeed-{job_id}/_start
```
  — transitions the datafeed to
```
started
```
  .
Opening before the datafeed exists fails. Starting the datafeed before opening the job fails. Do not report success after only creating resources — the job is not running until both are active.
Confirm running state from stats. Verify the outcome with:
- ```
GET /_ml/anomaly_detectors/{job_id}/_stats
```
  — expect
```
state: "opened"
```
  .
- ```
GET /_ml/datafeeds/datafeed-{job_id}/_stats
```
  — expect
```
state: "started"
```
  .
Optionally call
```
GET /_ml/anomaly_detectors/{job_id}
```
to confirm configuration (detectors,
```
bucket_span
```
,
```
time_field
```
, datafeed indices). Report both stats states explicitly — "created" is not the same as "opened" and "started".

确定目标索引和时间字段。使用
```
GET /_cat/indices
```
列出候选索引（当用户指定模式时传入对应模式）。使用
```
GET /{index}/_mapping
```
获取所选索引的字段类型。决策要点：确认索引存在，识别时间字段（通常为
```
@timestamp
```
），并验证文档量足以支撑基线学习。切勿猜测索引或字段名称——它们在不同部署中会有所差异。
选择检测器函数和方向。将用户需求与
```
analysis_config.detectors
```
中的计数类检测器匹配：
- 峰值、激增、事件量异常增长 →
```
high_count
```
  （或
```
count
```
  ，该函数会标记双向异常，但在用户关注峰值时也适用）。请勿使用
```
low_count
```
  ——它无法检测到峰值。
- 下降、中断、无事件、流量停止 →
```
low_count
```
  。请勿使用
```
high_count
```
  ——它无法检测到下降和静默情况。
- 指标偏差（CPU、延迟、数值字段）→ 均值类函数（
```
mean
```
  、
```
high_mean
```
  、
```
low_mean
```
  ）并设置
```
field_name
```
  ——仅当用户询问数值指标时使用，不适用于原始事件量相关问题。
决策要点：选择一个与异常类型匹配的主检测器。针对文档计数的峰值/下降问题，始终使用计数类检测器——均值检测器不适用于“事件数量”相关问题。

创建任务前设置不可变的任务结构。以下字段在执行

PUT /_ml/anomaly_detectors/{job_id}

后无法修改：

```
analysis_config.bucket_span
```
— 使用用户指定的时间间隔（例如
```
15m
```
代表15分钟桶）。匹配用户关注的异常粒度；间隔过短会产生大量噪声，过长则检测速度变慢。
```
data_description.time_field
```
— 来自映射的时间字段（通常为
```
@timestamp
```
）。
```
analysis_config.detectors
```
— 步骤2中确定的函数和方向。

流量峰值检测器的任务体示例：

json

{
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [{ "function": "high_count" }]
  },
  "data_description": { "time_field": "@timestamp" }
}

中断/下降检测器的示例：

json

{
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [{ "function": "low_count" }]
  },
  "data_description": { "time_field": "@timestamp" }
}

创建任务。使用用户请求的任务ID（或你建议的描述性ID）调用
```
PUT /_ml/anomaly_detectors/{job_id}
```
。任务初始状态为
```
closed
```
——创建任务并不启动分析。
创建数据馈送。任务创建完成后立即调用
```
PUT /_ml/datafeeds/datafeed-{job_id}
```
。将
```
job_id
```
设置为相同的ID，
```
indices
```
设置为步骤1中的目标索引（精确名称或模式），并设置一个选择相关文档的查询（通常为
```
match_all
```
）。数据馈送ID的惯例是
```
datafeed-{job_id}
```
。
json
```
{
  "job_id": "{job_id}",
  "indices": ["{index}"],
  "query": { "match_all": {} }
}
```
先打开任务，再启动数据馈送——顺序不可颠倒。此序列是强制性的；请勿跳过或重新排序：
1. ```
POST /_ml/anomaly_detectors/{job_id}/_open
```
  — 将任务转换为
```
opened
```
  状态。
2. ```
POST /_ml/datafeeds/datafeed-{job_id}/_start
```
  — 将数据馈送转换为
```
started
```
  状态。
在数据馈送创建前打开任务会失败。在打开任务前启动数据馈送也会失败。仅创建资源后请勿报告成功——只有两者都处于活跃状态时，任务才真正运行。
通过统计信息确认运行状态。使用以下命令验证结果：
- ```
GET /_ml/anomaly_detectors/{job_id}/_stats
```
  — 预期结果为
```
state: "opened"
```
  。
- ```
GET /_ml/datafeeds/datafeed-{job_id}/_stats
```
  — 预期结果为
```
state: "started"
```
  。
可选择调用
```
GET /_ml/anomaly_detectors/{job_id}
```
确认配置（检测器、
```
bucket_span
```
、
```
time_field
```
、数据馈送索引）。请明确报告这两种统计状态——“已创建”与“已打开”和“已启动”并非同一概念。

Teardown

销毁流程

When stopping or deleting a job, reverse the startup order:

POST /_ml/datafeeds/datafeed-{job_id}/_stop

— stop the datafeed first.

POST /_ml/anomaly_detectors/{job_id}/_close

— then close the job.

Stop the datafeed before closing the job. Close the job before resetting or deleting it.

停止或删除任务时，请颠倒启动顺序：

POST /_ml/datafeeds/datafeed-{job_id}/_stop

— 先停止数据馈送。

POST /_ml/anomaly_detectors/{job_id}/_close

— 再关闭任务。

关闭任务前需先停止数据馈送。重置或删除任务前需先关闭任务。

Guidelines

指导原则

Required lifecycle order (create): job → datafeed → open job → start datafeed. Every new job follows this sequence.
Detector direction is the highest-impact decision for volume anomalies. Re-read the user's wording: "spike", "surge", and "unusual increase" → high direction; "drop", "outage", "stops", "absence" → low direction.
Immutable fields (
```
bucket_span
```
, detectors,
```
time_field
```
) require delete-and-recreate if wrong — validate mapping and intent before the first
```
PUT
```
.
Datafeed index must match the user's target. Point
```
indices
```
at the exact index or pattern they named — not a nearby guess.
Entity-level analysis (
```
by_field_name
```
,
```
over_field_name
```
,
```
partition_field_name
```
) and advanced tuning live in references/anomaly-detection-reference.md.

创建时的必需生命周期顺序： 任务 → 数据馈送 → 打开任务 → 启动数据馈送。每个新任务都需遵循此序列。
检测器方向是影响流量异常检测的关键决策。重新阅读用户的表述：“峰值”、“激增”、“异常增长”对应高方向；“下降”、“中断”、“停止”、“缺失”对应低方向。
不可变字段（
```
bucket_span
```
、检测器、
```
time_field
```
）若设置错误，需删除后重新创建——在首次执行
```
PUT
```
前请验证映射和用户需求。
数据馈送索引必须与用户的目标匹配。将
```
indices
```
指向用户指定的精确索引或模式——请勿随意猜测。
实体级分析（
```
by_field_name
```
、
```
over_field_name
```
、
```
partition_field_name
```
）和高级调优请参考references/anomaly-detection-reference.md。

Full Reference

完整参考

For API paths, request/response fields, score semantics, and field interactions, read references/anomaly-detection-reference.md.

如需了解API路径、请求/响应字段、分数语义和字段交互，请阅读references/anomaly-detection-reference.md。

Operations

操作

HTTP API (shorthand)	`elastic` CLI command
`GET /_cat/indices`	`elastic es cat indices --index '<pattern>'`
`GET /{index}/_mapping`	`elastic es indices get-mapping --index '<index>'`
`PUT /_ml/anomaly_detectors/{job_id}`	`elastic es ml put-job --job-id '<job_id>' --analysis-config '<json>' --data-description '<json>'`
`PUT /_ml/datafeeds/datafeed-{job_id}`	`elastic es ml put-datafeed --datafeed-id 'datafeed-<job_id>' --job-id '<job_id>' --indices '<index>' --query '<json>'`
`POST /_ml/anomaly_detectors/{job_id}/_open`	`elastic es ml open-job --job-id '<job_id>'`
`POST /_ml/datafeeds/datafeed-{job_id}/_start`	`elastic es ml start-datafeed --datafeed-id 'datafeed-<job_id>'`
`GET /_ml/anomaly_detectors/{job_id}`	`elastic es ml get-jobs --job-id '<job_id>'`
`GET /_ml/anomaly_detectors/{job_id}/_stats`	`elastic es ml get-job-stats --job-id '<job_id>'`
`GET /_ml/datafeeds/datafeed-{job_id}/_stats`	`elastic es ml get-datafeed-stats --datafeed-id 'datafeed-<job_id>'`
`POST /_ml/datafeeds/datafeed-{job_id}/_stop`	`elastic es ml stop-datafeed --datafeed-id 'datafeed-<job_id>'`
`POST /_ml/anomaly_detectors/{job_id}/_close`	`elastic es ml close-job --job-id '<job_id>'`

HTTP API (简写形式)	`elastic` CLI命令
`GET /_cat/indices`	`elastic es cat indices --index '<pattern>'`
`GET /{index}/_mapping`	`elastic es indices get-mapping --index '<index>'`
`PUT /_ml/anomaly_detectors/{job_id}`	`elastic es ml put-job --job-id '<job_id>' --analysis-config '<json>' --data-description '<json>'`
`PUT /_ml/datafeeds/datafeed-{job_id}`	`elastic es ml put-datafeed --datafeed-id 'datafeed-<job_id>' --job-id '<job_id>' --indices '<index>' --query '<json>'`
`POST /_ml/anomaly_detectors/{job_id}/_open`	`elastic es ml open-job --job-id '<job_id>'`
`POST /_ml/datafeeds/datafeed-{job_id}/_start`	`elastic es ml start-datafeed --datafeed-id 'datafeed-<job_id>'`
`GET /_ml/anomaly_detectors/{job_id}`	`elastic es ml get-jobs --job-id '<job_id>'`
`GET /_ml/anomaly_detectors/{job_id}/_stats`	`elastic es ml get-job-stats --job-id '<job_id>'`
`GET /_ml/datafeeds/datafeed-{job_id}/_stats`	`elastic es ml get-datafeed-stats --datafeed-id 'datafeed-<job_id>'`
`POST /_ml/datafeeds/datafeed-{job_id}/_stop`	`elastic es ml stop-datafeed --datafeed-id 'datafeed-<job_id>'`
`POST /_ml/anomaly_detectors/{job_id}/_close`	`elastic es ml close-job --job-id '<job_id>'`