elasticsearch-index-design

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Elasticsearch Index Design

Elasticsearch 索引设计

Design explicit index mappings from access patterns, review existing mappings for type and storage mistakes, and apply corrections through a new index plus reindex when field types must change.

根据访问模式设计明确的索引映射，审核现有映射中的类型和存储错误，并在必须更改字段类型时，通过创建新索引并重新索引的方式应用修正。

Environment Configuration

环境配置

This skill executes Elasticsearch operations through the

elastic

CLI. If the

elastic

CLI is not installed, tell the user what it is needed for. Do not guess credentials, call the HTTP API directly, or attempt other workarounds.

This skill references operations in HTTP-shorthand form (e.g.,

GET /

GET /_cat/indices

GET /{index}/_mapping

GET /{index}/_settings/index.mode

POST /_query

). The Operations table at the end of this document maps each shorthand to the equivalent

elastic

CLI command — always use the CLI rather than calling the HTTP API directly.

本技能通过

elastic

CLI执行Elasticsearch操作。若未安装

elastic

CLI，请告知用户其用途。请勿猜测凭据、直接调用HTTP API或尝试其他变通方法。

本技能以HTTP简写形式引用操作（例如

GET /

、

GET /_cat/indices

、

GET /{index}/_mapping

、

GET /{index}/_settings/index.mode

、

POST /_query

）。本文档末尾的操作表格将每个简写映射为等效的

elastic

CLI命令——请始终使用CLI而非直接调用HTTP API。

Process

流程

Gather access patterns per field. Before choosing types, list how each field is used. For every field capture:
- Search — full-text match, phrase, relevance scoring?
- Filter — exact term, terms set, prefix?
- Aggregate — terms, cardinality, histogram, stats?
- Sort — ascending/d descending in result sets?
- Retrieve only — returned in
```
_source
```
  but never queried?
The decision: classify each field into one primary access pattern (search, exact, numeric metric, date, boolean, structured object, or retrieve-only). Missing access-pattern data is a blocker — ask the user rather than guessing. Call
```
GET /
```
to confirm connectivity; when reviewing an existing index, call
```
GET /{index}/_mapping
```
to ground the discussion in the current mapping.

Choose field types from access patterns. Map each field to the minimal type set that satisfies its pattern. Read Field Type Decisions and Multi-Field Patterns before proposing mappings.

Key judgments:

Pattern	Mapping
Full-text search only	`text` (no keyword sub-field)
Filter / agg / sort only	`keyword` (not `text` )
Full-text search and sort or aggregation	`text` with `fields.keyword` multi-field
Decimal price or metric	`double` , `float` , or `scaled_float` — not `text` or integer
Timestamp	`date`
True/false flag	`boolean`
Free-form key/value map with many distinct keys	`flattened` — not dynamic `object`

Multi-field rule: When a field must be searchable and sortable/aggregatable (e.g. product

name

), map it as

text

with a

keyword

sub-field — search on

name

, sort and aggregate on

name.keyword

. Mapping as only

text

or only

keyword

is wrong for that combined pattern.

Explicit mapping rule: For new indices, always define mappings explicitly with

PUT /{index}

. Do not rely on dynamic mapping for production indices — the first document can lock in wrong types (strings as

text

, ambiguous numbers as

keyword

Index settings: Set deliberate

number_of_shards

and

number_of_replicas

in the same

PUT /{index}

request when the deployment allows it (Self-Managed / Elastic Cloud Hosted). On Serverless, omit shard and replica counts (Elastic manages them); still supply explicit mappings. State chosen values or document that defaults apply.

Example —

products

index optimized for search plus sort/agg on name:

json

{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "fields": {
          "keyword": { "type": "keyword", "ignore_above": 256 }
        }
      },
      "price": { "type": "double" },
      "created": { "type": "date" },
      "in_stock": { "type": "boolean" }
    }
  }
}

Create with

PUT /products

passing the

settings

and

mappings

blocks. Verify with

GET /products/_mapping

Guard against mapping explosion and storage bloat. On high-volume indices, type mistakes multiply cost. Read Mapping Explosion and Storage Bloat and apply these review checks:
- Analyzed-but-not-searched fields — Fields used only for filter and aggregation (
```
url
```
  , HTTP
```
status_code
```
  ,
```
tags
```
  , IDs) must be
```
keyword
```
  , not
```
text
```
  .
```
text
```
  wastes space; aggregations on
```
text
```
  require fielddata or a
```
.keyword
```
  sub-field that should not exist if the field is not searched.
- message.keyword
  without
  ignore_above
  — A keyword sub-field on a large full-text body indexes the entire raw string as one term. Flag this anti-pattern; remove the sub-field when only full-text search is needed, or add
```
ignore_above
```
  when a bounded exact-match sub-field is truly required.
- Dynamic free-form objects —
```
object
```
  with
```
"dynamic": true
```
  on user-supplied key/value data with thousands of distinct keys causes mapping explosion. Recommend
```
flattened
```
  (or strict dynamic / allowlist strategy).
- doc_values: false
  — On fields retrieved in hits but never sorted, aggregated, or filtered (e.g. display-only
```
session_id
```
  ), set
```
"doc_values": false
```
  on
```
keyword
```
  to save disk at scale.
- scaled_float
  — For metrics with bounded precision (e.g.
```
response_time_ms
```
  ), prefer
```
scaled_float
```
  with an appropriate
```
scaling_factor
```
  over plain
```
float
```
  /
```
double
```
  when storage dominates.
Prefer
```
"dynamic": "strict"
```
on the root mapping unless unknown fields are an explicit requirement.
Apply design: create new index and reindex when types change. Elasticsearch cannot change an existing field's type in place. When review finds wrong types (text→keyword, object→flattened, float→scaled_float, doc_values changes on existing fields), state clearly that fixes require a new index and reindex — not a mapping update on the live index.
Workflow for correcting an existing high-volume index such as
```
events
```
:
1. Design the corrected mapping on a new index name (e.g.
```
events-v2
```
  ) incorporating all fixes from steps 2–3.
2. Create the destination with
```
PUT /events-v2
```
  and the full corrected
```
mappings
```
  (and
```
settings
```
  where applicable).
3. Copy documents with
```
POST /_reindex
```
  — for large indices use
```
wait_for_completion=false
```
  and track the task. Source:
```
{ "index": "events" }
```
  , destination:
```
{ "index": "events-v2" }
```
  .
4. Verify with
```
GET /events-v2/_count
```
  (compare to source count) and
```
GET /events-v2/_mapping
```
  (confirm types).
5. Cut over reads and writes (index alias swap or application config) after validation.
Example corrected excerpt for the
```
events
```
review pattern:
json
```
{
  "mappings": {
    "properties": {
      "@timestamp": { "type": "date" },
      "event_id": { "type": "keyword" },
      "session_id": { "type": "keyword", "doc_values": false },
      "url": { "type": "keyword" },
      "status_code": { "type": "keyword" },
      "response_time_ms": { "type": "scaled_float", "scaling_factor": 100 },
      "tags": { "type": "keyword" },
      "message": { "type": "text" },
      "labels": { "type": "flattened" }
    }
  }
}
```
Do not attempt in-place mapping fixes for these type changes — they are rejected or leave data inconsistent. For greenfield indices, a single
```
PUT /{index}
```
before first ingest avoids reindex entirely.

收集每个字段的访问模式。在选择类型之前，列出每个字段的使用方式。为每个字段记录：
- 搜索——是否为全文匹配、短语匹配、相关性评分？
- 过滤——是否为精确术语、术语集、前缀匹配？
- 聚合——是否为术语、基数、直方图、统计？
- 排序——结果集中是否需要升序/降序排列？
- 仅检索——仅在
```
_source
```
  中返回但从不查询？
决策：将每个字段归类为一种主要访问模式（搜索、精确匹配、数值指标、日期、布尔值、结构化对象或仅检索）。缺少访问模式数据会阻碍进程——请询问用户而非猜测。调用
```
GET /
```
确认连通性；审核现有索引时，调用
```
GET /{index}/_mapping
```
以基于当前映射展开讨论。

根据访问模式选择字段类型。将每个字段映射到满足其模式的最小类型集合。在提出映射方案之前，请阅读字段类型决策和多字段模式。

关键判断：

模式	映射方案
仅全文搜索	`text` （无keyword子字段）
仅过滤/聚合/排序	`keyword` （而非 `text` ）
全文搜索且排序或聚合	`text` 搭配 `fields.keyword` 多字段
十进制价格或指标	`double` 、 `float` 或 `scaled_float` ——而非 `text` 或整数
时间戳	`date`
布尔标记	`boolean`
包含大量不同键的自由形式键值对	`flattened` ——而非动态 `object`

多字段规则：当一个字段必须同时支持搜索和排序/聚合时（例如产品

name

），将其映射为带

keyword

子字段的

text

——在

name

上执行搜索，在

name.keyword

上执行排序和聚合。仅映射为

text

或仅映射为

keyword

对于这种组合模式来说都是错误的。

显式映射规则：对于新索引，始终通过

PUT /{index}

显式定义映射。生产环境索引请勿依赖动态映射——首个文档可能会锁定错误类型（字符串被识别为

text

、模糊数字被识别为

keyword

）。

索引设置：在部署允许的情况下（自托管/Elastic Cloud托管），在同一个

PUT /{index}

请求中设置明确的

number_of_shards

和

number_of_replicas

。在Serverless环境中，省略分片和副本计数（由Elastic管理）；仍需提供显式映射。说明所选值或记录使用默认值。

示例——针对搜索及名称排序/聚合优化的

products

索引：

json

{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "fields": {
          "keyword": { "type": "keyword", "ignore_above": 256 }
        }
      },
      "price": { "type": "double" },
      "created": { "type": "date" },
      "in_stock": { "type": "boolean" }
    }
  }
}

通过

PUT /products

请求传入

settings

和

mappings

块来创建索引。使用

GET /products/_mapping

验证。

防止映射爆炸和存储膨胀。在高容量索引上，类型错误会成倍增加成本。阅读映射爆炸与存储膨胀并应用以下审核检查：
- 仅用于过滤聚合的已分析字段——仅用于过滤和聚合的字段（
```
url
```
  、HTTP
```
status_code
```
  、
```
tags
```
  、ID）必须设为
```
keyword
```
  ，而非
```
text
```
  。
```
text
```
  会浪费空间；在
```
text
```
  上执行聚合需要fielddata或
```
.keyword
```
  子字段，若字段无需搜索则不应存在该子字段。
- 无
  ignore_above
  的
  message.keyword
  ——大型全文内容上的keyword子字段会将整个原始字符串作为单个词项索引。标记这种反模式；仅需全文搜索时移除该子字段，或在确实需要有界精确匹配子字段时添加
```
ignore_above
```
  。
- 动态自由形式对象——对包含数千个不同键的用户提供键值数据使用
```
"dynamic": true
```
  的
```
object
```
  会导致映射爆炸。建议使用
```
flattened
```
  （或严格动态/白名单策略）。
- doc_values: false
  ——对于仅在结果中返回但从不排序、聚合或过滤的字段（例如仅用于展示的
```
session_id
```
  ），在
```
keyword
```
  上设置
```
"doc_values": false
```
  以节省大规模场景下的磁盘空间。
- scaled_float
  ——对于精度有限的指标（例如
```
response_time_ms
```
  ），当存储为主要考量因素时，优先选择带合适
```
scaling_factor
```
  的
```
scaled_float
```
  而非普通
```
float
```
  /
```
double
```
  。
除非明确需要支持未知字段，否则在根映射上优先使用
```
"dynamic": "strict"
```
。
应用设计：类型变更时创建新索引并重新索引。Elasticsearch无法在原地修改现有字段的类型。当审核发现错误类型（text→keyword、object→flattened、float→scaled_float、现有字段的doc_values变更）时，明确说明修复需要新索引和重新索引——而非对实时索引进行映射更新。
修正现有高容量索引（如
```
events
```
）的工作流程：
1. 设计修正后的映射，使用新索引名称（例如
```
events-v2
```
  ），整合步骤2-3中的所有修复内容。
2. 创建目标索引，通过
```
PUT /events-v2
```
  传入完整的修正后
```
mappings
```
  （以及适用的
```
settings
```
  ）。
3. 复制文档，使用
```
POST /_reindex
```
  ——对于大型索引，使用
```
wait_for_completion=false
```
  并跟踪任务。源：
```
{ "index": "events" }
```
  ，目标：
```
{ "index": "events-v2" }
```
  。
4. 验证，使用
```
GET /events-v2/_count
```
  （与源索引计数对比）和
```
GET /events-v2/_mapping
```
  （确认类型）。
5. 切换流量，验证完成后切换读写操作（索引别名交换或应用配置修改）。
```
events
```
索引审核模式的修正示例片段：
json
```
{
  "mappings": {
    "properties": {
      "@timestamp": { "type": "date" },
      "event_id": { "type": "keyword" },
      "session_id": { "type": "keyword", "doc_values": false },
      "url": { "type": "keyword" },
      "status_code": { "type": "keyword" },
      "response_time_ms": { "type": "scaled_float", "scaling_factor": 100 },
      "tags": { "type": "keyword" },
      "message": { "type": "text" },
      "labels": { "type": "flattened" }
    }
  }
}
```
请勿尝试对这些类型变更进行原地映射修复——此类操作会被拒绝或导致数据不一致。对于全新索引，在首次写入前执行一次
```
PUT /{index}
```
即可完全避免重新索引操作。

Review checklist

审核清单

When the user supplies a mapping JSON and usage notes, walk this checklist in order:

Match each field's type to its stated access pattern (see step 2).
Flag
```
text
```
on filter/agg-only fields; flag missing multi-fields where search and sort/agg share one logical field.
Flag
```
message.keyword
```
(or similar) without
```
ignore_above
```
on large analyzed text.
Flag dynamic
```
object
```
on high-cardinality free-form maps; recommend
```
flattened
```
.
Propose retrieve-only and numeric storage optimizations (
```
doc_values: false
```
,
```
scaled_float
```
).
State that type changes require a new index and
```
POST /_reindex
```
, then show the corrected mapping and reindex plan.

当用户提供映射JSON和使用说明时，按以下顺序执行检查：

匹配每个字段的类型与其指定的访问模式（见步骤2）。
标记仅用于过滤/聚合的字段使用
```
text
```
类型的情况；标记同一逻辑字段需同时支持搜索和排序/聚合但缺少多字段的情况。
标记大型已分析文本上无
```
ignore_above
```
的
```
message.keyword
```
（或类似字段）。
标记高基数自由形式映射使用动态
```
object
```
的情况；建议使用
```
flattened
```
。
提出仅检索字段和数值存储优化方案（
```
doc_values: false
```
、
```
scaled_float
```
）。
说明类型变更需要新索引和
```
POST /_reindex
```
，然后展示修正后的映射和重新索引计划。

Examples

示例

"Users search product names and also sort and aggregate on them" — one logical field, two access patterns, so use a

text

field with a

keyword

multi-field:

json

{
  "mappings": {
    "properties": {
      "product_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }
    }
  }
}

"A
status
field is only ever filtered and aggregated, never full-text searched" — use

keyword

, not

text

json

{ "mappings": { "properties": { "status": { "type": "keyword" } } } }

"Free-form
labels
object with unbounded keys" — avoid mapping explosion with

flattened

json

{ "mappings": { "properties": { "labels": { "type": "flattened" } } } }

"用户搜索产品名称，同时也对其进行排序和聚合"——一个逻辑字段，两种访问模式，因此使用带

keyword

多字段的

text

字段：

json

{
  "mappings": {
    "properties": {
      "product_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }
    }
  }
}

"
status
字段仅用于过滤和聚合，从不进行全文搜索"——使用

keyword

而非

text

：

json

{ "mappings": { "properties": { "status": { "type": "keyword" } } } }

"包含无限键的自由形式
labels
对象"——使用

flattened

避免映射爆炸：

json

{ "mappings": { "properties": { "labels": { "type": "flattened" } } } }

Guidelines

指南

Minimal mapping — Map only what access patterns require; every sub-field and analyzed form adds indexed data.
Never guess access patterns — Wrong type choice is expensive to fix at scale.
Verify after create — Always confirm with
```
GET /{index}/_mapping
```
; use
```
GET /{index}/_count
```
after reindex.
Cross-skill boundary — Copying documents between indices is
```
POST /_reindex
```
(see the reindex skill for slicing, throttling, and task tracking). Loading files into a new index is bulk ingest, not index design.

最小化映射——仅映射访问模式所需的内容；每个子字段和已分析形式都会增加索引数据量。
绝不猜测访问模式——错误的类型选择在大规模场景下修复成本极高。
创建后验证——始终使用
```
GET /{index}/_mapping
```
确认；重新索引后使用
```
GET /{index}/_count
```
验证。
跨技能边界——在索引之间复制文档使用
```
POST /_reindex
```
（有关分片、限流和任务跟踪，请参阅重新索引技能）。将文件加载到新索引属于批量导入，而非索引设计范畴。

Reference material

参考资料

Field Type Decisions — access-pattern-to-type table and common mistakes
Multi-Field Patterns — text+keyword,
```
ignore_above
```
, anti-patterns
Mapping Explosion and Storage Bloat —
```
flattened
```
,
```
doc_values
```
, dynamic objects

字段类型决策——访问模式到类型的对应表及常见错误
多字段模式——text+keyword、
```
ignore_above
```
、反模式
映射爆炸与存储膨胀——
```
flattened
```
、
```
doc_values
```
、动态对象

Operations

操作

HTTP API (shorthand)	`elastic` CLI command
`GET /`	`elastic es info`
`GET /{index}/_mapping`	`elastic es indices get-mapping --index '<index>'`
`PUT /{index}`	`elastic es indices create --index '<index>' --mappings '<json>' --settings '<json>'`
`POST /_reindex`	`elastic es reindex --source '<json>' --dest '<json>'`
`POST /_reindex?wait_for_completion=false`	`elastic es reindex --wait-for-completion false --source '<json>' --dest '<json>'`
`GET /{index}/_count`	`elastic es count --index '<index>'`

HTTP API（简写形式）	`elastic` CLI命令
`GET /`	`elastic es info`
`GET /{index}/_mapping`	`elastic es indices get-mapping --index '<index>'`
`PUT /{index}`	`elastic es indices create --index '<index>' --mappings '<json>' --settings '<json>'`
`POST /_reindex`	`elastic es reindex --source '<json>' --dest '<json>'`
`POST /_reindex?wait_for_completion=false`	`elastic es reindex --wait-for-completion false --source '<json>' --dest '<json>'`
`GET /{index}/_count`	`elastic es count --index '<index>'`