sequencer

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Sequencer

ID序列生成器

Hand out identifiers that are unique across every node without a central bottleneck — and decide whether those IDs must also be sortable or monotonic. Getting this wrong shows up late and hard: collisions corrupt data, a single allocator caps write throughput, and IDs that leak a creation time or a sequential count expose business secrets and enable enumeration attacks.
在无中心瓶颈的情况下,为所有节点分配全局唯一的标识符——同时可决定这些ID是否需要具备可排序性或单调性。如果这部分设计出错,问题会出现得晚且影响严重:ID冲突会损坏数据,单一分配器会限制写入吞吐量,而泄露创建时间或序列计数的ID会暴露业务机密并引发枚举攻击。

When to reach for this

适用场景

A system writes new records across multiple nodes and each needs a primary key (orders, messages, uploads, events). Reach for this when a single auto-increment column would serialize all writes, when IDs must be generated before a DB round trip (client-side, offline), or when records must be roughly time-ordered without a separate sort field.
系统需要在多个节点上写入新记录,且每条记录都需要主键(如订单、消息、上传文件、事件)。当单一自增列会序列化所有写入操作、需要在数据库往返前生成ID(客户端、离线场景),或需要记录大致按时间排序而无需单独排序字段时,可使用本方案。

When NOT to

不适用场景

A single relational node still comfortably serves the write load (→
back-of-the-envelope
) — then a plain
BIGINT AUTO_INCREMENT
/
SERIAL
is the cheapest correct answer; do not build a distributed ID service for it (YAGNI). If a natural unique key already exists (email, ISBN, content hash), use it. Don't demand global monotonicity unless an invariant truly needs it — it is the most expensive property here and usually only per-entity ordering is required.
单一关系型节点仍能轻松处理写入负载(参考
back-of-the-envelope
)——此时普通的
BIGINT AUTO_INCREMENT
/
SERIAL
是最经济的正确方案;无需为其构建分布式ID服务(YAGNI原则)。如果已存在天然唯一键(如邮箱、ISBN、内容哈希),则直接使用该键。除非确实需要全局不变的单调性,否则不要强求——这是成本最高的特性,通常仅需「按实体排序」即可。

Clarify first

先明确需求

  • Generation point — client/edge, app server, or database? (Decides whether a DB round trip per ID is acceptable.)
  • Ordering need — none, time-sortable (k-sorted is fine), or strictly monotonic? Per-entity or global? This is the single biggest fork.
  • Write rate & node count — IDs/sec at peak and how many generators (→
    back-of-the-envelope
    ). Sets the bits needed for a sequence counter.
  • Size & encoding budget — 64-bit int (fits an indexed key cheaply) vs 128-bit (no coordination ever) vs short URL-safe string?
  • Leakage tolerance — may the ID reveal creation time or a guessable count (enumeration / competitor signal)?
  • 生成节点——客户端/边缘节点、应用服务器还是数据库?(决定是否接受每次生成ID都需数据库往返。)
  • 排序需求——无、「可按时间排序」(k排序即可)还是「严格单调」?是按实体还是全局?这是最关键的分支点。
  • 写入速率与节点数量——峰值时每秒生成的ID数量,以及生成器节点的数量(参考
    back-of-the-envelope
    )。这决定了序列计数器所需的位数。
  • 大小与编码预算——64位整数(可低成本作为索引键)、128位(无需任何协调)还是短URL安全字符串?
  • 泄露容忍度——ID是否可以暴露创建时间或可猜测的计数(枚举攻击/竞争对手信号)?

The options

可选方案

  • Auto-increment / SQL sequence — one DB column hands out IDs. Use when a single node owns the writes and you want zero new infrastructure.
  • UUIDv4 (random 128-bit) — generate anywhere, no coordination, effectively zero collision risk. Use when you only need uniqueness and never sort by ID.
  • ULID / UUIDv7 (time-prefixed 128-bit) — random but with a millisecond timestamp prefix, so IDs sort by creation time. Use when you want UUIDv4's zero-coordination and time-ordering (the modern default for new keys).
  • Snowflake-style (timestamp + node + sequence, 64-bit) — pack a timestamp, a node ID, and a per-ms counter into a sortable 64-bit int. Use at high write rates where a compact, k-sorted integer key matters.
  • DB ticket / range allocation (Flickr-style) — a central table hands out blocks of IDs (e.g. 1000 at a time); each node serves from its block in memory. Use when you want simple monotonic-ish integers without per-ID coordination.
  • 自增/SQL序列——由数据库的一列分配ID。适用于单一节点处理所有写入操作,且不想新增任何基础设施的场景。
  • UUIDv4(随机128位)——可在任意位置生成,无需协调,几乎无冲突风险。仅需唯一性且无需按ID排序时使用。
  • ULID / UUIDv7(时间前缀128位)——随机生成但带有毫秒时间戳前缀,因此ID可按创建时间排序。当你需要UUIDv4的无协调特性+时间排序时使用(这是新键的现代默认方案)。
  • Snowflake风格(时间戳+节点+序列,64位)——将时间戳、节点ID和每毫秒计数器打包成可排序的64位整数。适用于高写入速率场景,且需要紧凑、k排序的整数键时使用。
  • 数据库票据/范围分配(Flickr风格)——由中心表分配ID块(例如一次分配1000个);每个节点从内存中的块中分配ID。当你需要简单的近似单调整数且无需为每个ID进行协调时使用。

Trade-offs

权衡对比

OptionWhat it solvesWhat it worsensChange it when
Auto-increment / sequenceTrivial, monotonic, compact intSerializes writes; single node caps throughput; leaks countWrites outgrow one node, or you need client-side IDs → ticket/Snowflake
UUIDv4 (random)Generate anywhere, no coordination, no leakage128-bit; random order kills index locality (page splits); not sortableYou need time-ordering → ULID/UUIDv7
ULID / UUIDv7Zero coordination + time-sortable + index-friendlyStill 128-bit; only ms-sortable (not strict); leaks creation timeYou need a 64-bit key or strict order → Snowflake / sequence
Snowflake-style (64-bit)Compact, k-sorted, ~4M IDs/node/secNeeds node-ID assignment + clock-skew handling; epoch/bit budget caps lifespanClock sync is unreliable, or you can't assign node IDs → ULID
DB ticket / rangeMonotonic-ish ints, low coordination, simpleAllocator table is a SPOF; gaps on restart; only loosely ordered across nodesAllocator becomes a bottleneck or SPOF → Snowflake/ULID
方案解决的问题带来的问题何时更换
自增/序列实现简单、单调、紧凑整数序列化写入操作;单一节点限制吞吐量;泄露计数写入量超出单一节点处理能力,或需要客户端ID → 票据/Snowflake方案
UUIDv4(随机)可在任意位置生成、无需协调、无泄露128位长度;随机顺序破坏索引局部性(页分裂);不可排序需要时间排序 → ULID/UUIDv7
ULID / UUIDv7无协调+可按时间排序+索引友好仍为128位;仅支持毫秒级排序(非严格);泄露创建时间需要64位键或严格排序 → Snowflake/序列
Snowflake风格(64位)紧凑、k排序、单节点每秒约400万ID需要分配节点ID+处理时钟偏移;纪元/位预算限制生命周期时钟同步不可靠,或无法分配节点ID → ULID
数据库票据/范围近似单调整数、低协调、实现简单分配器表是单点故障;重启时会产生ID间隙;跨节点仅松散排序分配器成为瓶颈或单点故障 → Snowflake/ULID

Behavior under stress

压力下的表现

The whole point of distributed ID schemes is to avoid a single allocator, so the failure modes cluster around coordination shortcuts.
  • Allocator as SPOF/bottleneck (ticket, single sequence): every write blocks on one row/node. A spike or its failure stalls all inserts. Mitigate: hand out larger ranges, replicate the allocator, or move to Snowflake/ULID (no central hop). Larger ranges trade away monotonicity and waste IDs on restart.
  • Clock skew & rewind (Snowflake/time-prefixed): if a node's clock jumps backward (NTP correction, VM pause), it can re-emit a timestamp it already used and collide within its node+sequence space. Mitigate: refuse to emit while
    now < last_timestamp
    (block or error), use a monotonic clock source, and alarm on skew. Never silently trust wall-clock time.
  • Sequence-bits exhaustion: more than
    2^seq_bits
    IDs in one millisecond on one node overflows the counter. Mitigate: spin-wait to the next ms, or size the bit budget to peak rate up front.
  • Node-ID collision: two generators boot with the same node ID (bad config, autoscaling reuse) and silently mint duplicates. Mitigate: lease node IDs from a coordinator (→
    consistency-coordination
    ) instead of static config.
  • Hot shard from sequential keys: monotonic IDs as a shard/partition key send all new writes to one shard. Mitigate: hash the key or prefix-shard — the partitioning fix lives in
    data-storage
    .
Monitor: ID issuance rate per node, clock-skew/rewind events, allocator latency and range-exhaustion rate, and duplicate-key errors (should be zero).
分布式ID方案的核心是避免单一分配器,因此故障模式集中在「协调捷径」上。
  • 分配器作为单点故障/瓶颈(票据、单一序列):每次写入都依赖单一行/节点。流量峰值或节点故障会导致所有插入停滞。缓解措施:分配更大的ID范围、复制分配器,或切换到Snowflake/ULID(无需中心节点)。更大的范围会牺牲单调性,且重启时会浪费ID。
  • 时钟偏移与回退(Snowflake/时间前缀):如果节点时钟回退(NTP校正、VM暂停),可能会重新发出已使用过的时间戳,导致节点+序列空间内的ID冲突。缓解措施:当
    now < last_timestamp
    时拒绝生成ID(阻塞或报错),使用单调时钟源,并对偏移发出告警。永远不要盲目信任墙上时钟时间。
  • 序列位耗尽:单个节点在一毫秒内生成超过
    2^seq_bits
    个ID,会导致计数器溢出。缓解措施:自旋等待到下一毫秒,或提前根据峰值速率设置位预算。
  • 节点ID冲突:两个生成器启动时使用相同的节点ID(配置错误、自动扩缩重用),会静默生成重复ID。缓解措施:从协调器租赁节点ID(参考
    consistency-coordination
    ),而非静态配置。
  • 顺序键导致热点分片:单调ID作为分片/分区键会将所有新写入发送到同一个分片。缓解措施:对键进行哈希或前缀分片——该修复属于
    data-storage
    范畴。
监控指标:每个节点的ID生成速率、时钟偏移/回退事件、分配器延迟和范围耗尽速率,以及重复键错误(应为零)。

How to apply

实施步骤

  1. Clarify the inputs — pin down the generation point, the ordering need (none / time-sortable / strict; per-entity vs global), peak IDs/sec, the size budget, and leakage tolerance (see Clarify first). If one DB node serves the writes, stop — use auto-increment (→
    back-of-the-envelope
    ).
  2. Pick from the trade-off table — no ordering and 128-bit is fine → UUIDv4; want time-sortable with zero coordination → ULID/UUIDv7; need a compact 64-bit k-sorted int at high rate → Snowflake; want simple monotonic ints → ticket/range.
  3. Set the key knobs — for Snowflake fix the epoch and the timestamp/node/seq bit split against your node count and peak rate; for ticket pick the block size; decide the node-ID assignment method (lease vs static); choose the encoding (raw int, Base62, Crockford Base32).
  4. Stress-test the choice — walk Behavior under stress: clock rewind, sequence exhaustion, node-ID collision, allocator failure, and the hot-shard effect of sequential keys. Confirm a mitigation for each one your profile hits.
  5. Size it with numbers — confirm the bit budget covers peak IDs/sec/node and the epoch gives enough years; confirm the encoded length fits the key/URL constraint (→ Numbers that matter).
  6. Pick a provider — default to the generic recipe (a Snowflake library, a ticket row, or ULID/UUIDv7); only open a provider file if the user named a cloud (see Choosing a provider).
  1. 明确输入条件——确定生成节点、排序需求(无/时间排序/严格;按实体或全局)、峰值每秒ID数、大小预算和泄露容忍度(见「先明确需求」)。如果单一数据库节点可处理写入负载,就此停止——使用自增方案(参考
    back-of-the-envelope
    )。
  2. 根据权衡表选择方案——无需排序且128位长度可接受 → UUIDv4;需要无协调+时间排序 → ULID/UUIDv7;高速率下需要紧凑的64位k排序整数 → Snowflake;需要简单的近似单调整数 → 票据/范围分配。
  3. 设置关键参数——对于Snowflake方案,根据节点数量和峰值速率确定纪元以及时间戳/节点/序列的位分配;对于票据方案,选择块大小;确定节点ID分配方式(租赁 vs 静态);选择编码方式(原始整数、Base62、Crockford Base32)。
  4. 压力测试所选方案——检查「压力下的表现」中的场景:时钟回退、序列耗尽、节点ID冲突、分配器故障、顺序键导致的热点分片。确保每个命中的场景都有缓解措施。
  5. 量化规模——确认位预算可覆盖单节点峰值每秒ID数,且纪元提供足够的使用年限;确认编码后的长度符合键/URL约束(参考「关键数值」)。
  6. 选择实现方式——默认使用通用方案(Snowflake库、票据行或ULID/UUIDv7);仅当用户指定云厂商时,才查看
    references/providers/<provider>.md
    获取托管服务映射、配额/限制和厂商特定的权衡。如果该厂商没有对应文件,通用方案即为答案(大多数云厂商没有专门的ID服务——你需要自行运行库或序列)。

Dos and don'ts

注意事项

Do
  • Default to ULID/UUIDv7 for new keys when you want zero coordination plus rough time-ordering — it dodges UUIDv4's random-index pain.
  • Size the Snowflake bit budget (timestamp/node/sequence) against peak rate and required lifespan before picking it; write the epoch down.
  • Lease node IDs from a coordinator instead of static config when nodes autoscale.
  • Refuse to emit on clock rewind and alarm on skew; treat duplicate-key errors as a P1.
  • Separate the internal key (sortable, may leak time) from any external opaque ID when enumeration or leakage matters.
Don't
  • Don't build a distributed ID service before a number shows one DB node can't keep up (YAGNI).
  • Don't use a random UUIDv4 as a clustered/primary index on a hot table — random order causes page splits and write amplification.
  • Don't make a single sequence or ticket row the allocator for the whole fleet without replication — it's a SPOF and a write bottleneck.
  • Don't use a globally monotonic ID as a shard key — it creates a hot shard (fix in
    data-storage
    ).
  • Don't trust wall-clock time for ordering; ms-sortable is k-sorted, not strict.
建议
  • 当你需要无协调+大致时间排序的新键时,默认选择ULID/UUIDv7——它避免了UUIDv4的随机索引问题。
  • 在选择Snowflake方案前,根据峰值速率和所需寿命设置位预算(时间戳/节点/序列);记录纪元时间。
  • 当节点自动扩缩时,从协调器租赁节点ID而非使用静态配置。
  • 时钟回退时拒绝生成ID并发出告警;将重复键错误视为P1级问题。
  • 当枚举或泄露风险重要时,将「内部」键(可排序、可能泄露时间)与「外部」透明ID分开。
禁忌
  • 在数据显示单一数据库节点无法处理负载前,不要构建分布式ID服务(YAGNI原则)。
  • 不要在热点表上使用随机UUIDv4作为聚集/主键——随机顺序会导致页分裂和写入放大。
  • 不要让单一序列或票据行成为整个集群的分配器而不进行复制——这是单点故障和写入瓶颈。
  • 不要使用全局单调ID作为分片键——会导致热点分片(修复见
    data-storage
    )。
  • 不要信任墙上时钟时间进行排序;毫秒级排序是k排序,而非严格排序。

Numbers that matter

关键数值

A 64-bit Snowflake layout (≈41 timestamp bits + 10 node + 12 sequence) gives ~69 years from its epoch, 1024 nodes, and 4096 IDs/node/ms ≈ 4M IDs/node/sec — ample for almost any single service. UUID/ULID are 128 bits = 16 bytes (vs 8 for a 64-bit int), doubling index key size. A ticket block of N IDs cuts allocator hits by N× but risks losing up to N IDs on a node restart. For peak-rate and storage sizing, see
back-of-the-envelope
; restate only the figure a decision turns on.
64位Snowflake布局(约41位时间戳 + 10位节点 + 12位序列)从纪元开始可使用约69年,支持1024个节点,单节点每毫秒可生成4096个ID ≈ 单节点每秒400万ID——几乎适用于任何单一服务。UUID/ULID为128位=16字节(64位整数为8字节),索引键大小翻倍。票据块大小为N时,分配器访问次数减少N倍,但节点重启时最多可能丢失N个ID。如需峰值速率和存储规模计算,参考
back-of-the-envelope
;仅需说明决策依赖的数值。

Interface sketch

接口示例

An issued ID is a contract. State its width (64 vs 128 bit), its layout (e.g. Snowflake
[timestamp:41 | node:10 | seq:12]
), its ordering guarantee (unordered / k-sorted by ms / strict), and its encoding (raw int, Base62, Crockford Base32 — case-insensitive, URL-safe). A generator endpoint, if any, is minimal:
next(entity) -> {id, issued_at}
. Document whether the ID is the storage key, the sort key, or both — that choice is consumed by
data-storage
.
生成的ID是一种约定。需说明其长度(64位 vs 128位)、布局(例如Snowflake
[timestamp:41 | node:10 | seq:12]
)、排序保证(无序/按毫秒k排序/严格)和编码方式(原始整数、Base62、Crockford Base32——大小写不敏感、URL安全)。如果有生成器端点,应尽量简洁:
next(entity) -> {id, issued_at}
。需说明该ID是存储键、排序键还是两者皆是——此选择会被
data-storage
使用。

Choosing a provider

选择实现提供商

Default to the generic recipe above. If the user names a cloud, read
references/providers/<provider>.md
for the managed-service mapping, quotas/limits, and provider-specific trade-offs. If no file exists for that provider, the generic recipe is the answer (most clouds have no dedicated ID service — you run a library or a sequence yourself).
默认使用上述通用方案。如果用户指定云厂商,查看
references/providers/<provider>.md
获取托管服务映射、配额/限制和厂商特定的权衡。如果该厂商没有对应文件,通用方案即为答案(大多数云厂商没有专门的ID服务——你需要自行运行库或序列)。

Diagram

图表

To visualize the issuance path (generator nodes → 64-bit layout → record key) or the ticket-server block-allocation flow, use the in-plugin
architecture-diagram
skill; an inline
[ts | node | seq]
sketch is enough for quick bit-budget reasoning. Do not embed Mermaid.
如需可视化生成路径(生成器节点 → 64位布局 → 记录键)或票据服务器块分配流程,使用插件内的
architecture-diagram
Skill;快速进行位预算推理时,使用内联
[ts | node | seq]
示意图即可。不要嵌入Mermaid图表。

Related building blocks

相关组件

  • data-storage
    feeds into it: the ID becomes the primary/sort key, and the sharding/partitioning that a sequential key can hot-spot is owned there.
  • consistency-coordination
    depends on it for the causality, ordering, and leader-election theory behind monotonic guarantees and node-ID leasing; link, don't re-teach.
  • messaging-streaming
    pairs with it for message ordering and dedup, where time-sortable IDs give a natural sequence and idempotency anchor.
  • api-design
    pairs with it: ID generation underpins idempotency keys (owned there) for safe retries.
  • system-design
    owned-concept lives in the orchestrator: the reasoning loop, the trade-off method, and the ten failure modes.
  • data-storage
    依赖本组件:ID会成为主键/排序键,而顺序键可能导致的热点分片问题由该组件处理。
  • consistency-coordination
    本组件依赖它:单调性保证和节点ID租赁背后的因果性、排序性和领导者选举理论;仅需链接,无需重复讲解。
  • messaging-streaming
    与本组件配合使用:用于消息排序和去重,可按时间排序的ID提供了天然的序列和幂等性锚点。
  • api-design
    与本组件配合使用:ID生成是幂等键(由该组件负责)的基础,用于安全重试。
  • system-design
    核心概念归属:推理流程、权衡方法和十种故障模式属于系统设计范畴。

References

参考资料

  • references/deep-dive.md
    — Snowflake bit-layout math and epoch choice, clock-skew/monotonic-clock handling, ticket-server range allocation, ULID vs UUIDv7 byte layout, encoding (Base62/Crockford), and node-ID leasing. Read when designing the generator in detail.
  • references/providers/{generic,aws,gcp}.md
    — library/service mappings, limits, and pitfalls per environment.
  • references/deep-dive.md
    — Snowflake位布局计算和纪元选择、时钟偏移/单调时钟处理、票据服务器范围分配、ULID与UUIDv7字节布局、编码(Base62/Crockford)和节点ID租赁。详细设计生成器时阅读。
  • references/providers/{generic,aws,gcp}.md
    — 各环境下的库/服务映射、限制和陷阱。