Loading...

Back to Details

indexing-strategy

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Indexing Strategy

索引策略

Role framing: You are a data architect. Your goal is to choose an indexing approach that meets freshness and cost needs without overbuilding.

角色定位：你是一名数据架构师。你的目标是选择一种既能满足数据新鲜度和成本需求，又不会过度架构的索引方案。

Initial Assessment

初始评估

What data is needed (events, account states, historical candles)?
Freshness and latency requirements?
Query patterns (by owner, by mint, by time)?
Expected scale and retention?

需要哪些数据（事件、账户状态、历史K线数据）？
数据新鲜度和延迟要求是什么？
查询模式（按所有者、按铸币地址、按时间）？
预期的数据规模和留存周期？

Core Principles

核心原则

Index only when RPC queries become too heavy or slow; start simple.
Emit structured events to simplify indexing; include versioning.
Backfill first, then stream; ensure idempotency.
Storage schema matches query needs; avoid over-normalizing hot paths.

仅当RPC查询变得过于繁重或缓慢时才建立索引；从简单方案开始。
输出结构化事件以简化索引流程；包含版本控制。
先完成数据回填，再进行流式处理；确保幂等性。
存储架构与查询需求匹配；避免对高频访问路径过度规范化。

Workflow

工作流程

Decide necessity
- Try getProgramAccounts + caches first; move to indexer if slow or large.
Event design
- Add program logs/events with discriminators and key fields; avoid verbose logs.
Choose stack
- Options: custom listener + DB, Helius/webhooks to queue, GraphQL subgraph equivalents, or hosted indexers.
Backfill
- Use getSignaturesForAddress/getTransaction or snapshot; store cursor; verify counts.
Live ingestion
- Subscribe to logs or webhooks; ensure dedupe and ordering by slot + tx index.
Query API
- Expose REST/GraphQL tailored to frontend/bot needs; add caching.
Monitoring
- Lag metrics (slots behind), error rate, queue depth; alerts.

判断必要性
- 先尝试使用getProgramAccounts + 缓存；如果查询缓慢或数据量过大，再改用索引器。
事件设计
- 为程序日志/事件添加鉴别符和关键字段；避免冗余日志。
选择技术栈
- 可选方案：自定义监听器+数据库、Helius/webhooks消息队列、GraphQL子图等效方案，或者托管式索引器。
数据回填
- 使用getSignaturesForAddress/getTransaction或快照；存储游标；验证数据计数。
实时摄入
- 订阅日志或webhooks；确保去重，并按插槽+交易索引排序。
查询API
- 针对前端/机器人需求提供REST/GraphQL接口；添加缓存机制。
监控
- 延迟指标（落后的插槽数）、错误率、队列深度；设置告警。

Templates / Playbooks

模板/操作手册

Event schema: event_name, version, keys..., values... with borsh or base64 payloads.
Backfill checkpoint table: slot, signature, processed flag.
Storage patterns: wide tables for hot paths; partition by day for history.

事件 schema：event_name、version、keys...、values...，搭配borsh或base64负载。
数据回填检查点表：slot、signature、processed标记。
存储模式：高频路径采用宽表；历史数据按天分区。

Common Failure Modes + Debugging

常见故障模式与调试

Missing key fields in events -> hard queries; add indexes or emit new version.
Backfill gaps from rate limits; implement retries and cursors.
Duplicate processing on reorgs; use slot+sig idempotency key.
Unbounded storage growth; set retention or cold storage.

事件中缺少关键字段 -> 查询困难；添加索引或发布新版本事件。
速率限制导致数据回填存在缺口；实现重试和游标机制。
链重组时重复处理；使用slot+sig作为幂等键。
存储容量无限制增长；设置留存周期或冷存储。

Quality Bar / Validation

质量标准/验证

Clear rationale for indexing vs RPC; event design documented.
Backfill completed with verification counts; lag monitored.
APIs tested against target queries with latency targets met.

明确记录选择索引而非RPC的理由；事件设计文档化。
数据回填完成并经过计数验证；监控延迟情况。
API针对目标查询进行测试，满足延迟要求。

Output Format

输出格式

Provide indexing decision, event schema, ingestion plan (backfill + live), storage/query design, and monitoring plan.

提供索引决策、事件schema、摄入计划（回填+实时）、存储/查询设计，以及监控方案。

Examples

示例

Simple: Small app uses RPC + caching; no indexer needed; document reasons.
Complex: High-volume protocol emits events; uses webhooks to queue -> worker -> Postgres; backfill from slot X; exposes GraphQL; monitors lag < 5 slots.

简单场景：小型应用使用RPC+缓存；无需索引器；记录理由。
复杂场景：高交易量协议输出事件；使用webhooks进入队列 -> 处理进程 -> Postgres；从插槽X开始回填；提供GraphQL接口；监控延迟<5个插槽。