clickhouse-js-node-rowbinary

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

ClickHouse JS RowBinary Codec Generator for Node.js

Node.js 版 ClickHouse JS RowBinary 编解码器生成工具

This skill generates both directions of the wire format: readers (decode bytes → values) and writers (encode values → bytes, the mirror). A given task normally needs only one side. This file is the shared entry point — the format gate plus the principles common to both directions; the per-direction decisions, guidance, and the per-type reference tables live in two sibling files.

Pick your side — read only the one you need:

Decoding a
RowBinary*
response from ClickHouse into JS values → reader.md. Streaming vs whole-buffer, row-objects vs columnar, fixed vs runtime schema, and the per-type reader reference.
Encoding JS values into a
RowBinary
payload to send to ClickHouse → writer.md. The
```
Sink
```
/
```
writeX
```
building blocks,
```
writeRows
```
streaming, and the per-type writer reference.

The per-type code is real, split by direction under

src/readers/

and

src/writers/

此技能可生成双向的有线格式处理代码：读取器（将字节解码为值）和写入器（将值编码为字节，与读取器互为镜像）。通常一项任务只需要其中一方。本文件是共享入口点——包含格式网关以及双向通用的原则；各方向的决策、指南和按类型划分的参考表则位于两个同级文件中。

选择你需要的方向——仅阅读对应内容即可：

将ClickHouse返回的
```
RowBinary*
```
响应解码为JS值 → reader.md。涵盖流式处理与全缓冲、行对象与列存储、固定模式与运行时模式，以及按类型划分的读取器参考。
将JS值编码为
```
RowBinary
```
负载以发送至ClickHouse → writer.md。包含
```
Sink
```
/
```
writeX
```
构建块、
```
writeRows
```
流式处理，以及按类型划分的写入器参考。

按类型划分的代码是真实可用的，按方向分别存放在

src/readers/

和

src/writers/

目录下。

First: is RowBinary even the right format?

首先：RowBinary 是合适的格式吗？

RowBinary exists for throughput, but it is not automatically the fastest path — match the format to the shape of the data before committing to a bespoke parser.

Prefer a
JSON*
format (e.g.
JSONEachRow
) when the result is mostly strings / JSON-like values that you consume wholesale — randomly accessing essentially every field, running string/regexp methods on them, treating values as text. V8's native

JSON.parse

is heavily optimized C++ and builds JS strings and objects faster than a JS-level RowBinary decoder can; pair it with HTTP response compression (

gzip

zstd

, which crushes JSON's repetitive keys) and the wire cost shrinks too.

RowBinary clearly wins when the result is dominated by:

Wide numerics —

Int128

Int256

UInt128

UInt256

Decimal128

Decimal256

Binary / fixed-width blobs —
```
IPv4
```
,
```
IPv6
```
,
```
UUID
```
,
```
FixedString
```
.
High-volume fixed-width numeric columns generally, where each value is a single
```
DataView
```
read.

Prefer the
Native
format when columnar load and client-side analytics are the main goal (fold/scan/filter columns, feed typed arrays to a Worker or WASM).

Native

is column-major, so it loads straight into one typed array per column with no transpose.

For help choosing and consuming a

JSON*

format (or CSV / TSV) instead, use the clickhouse-js-node-coding
skill.

RowBinary专为吞吐量设计，但它并非自动是最快的路径——在决定定制解析器之前，要让格式与数据形态相匹配。

当结果主要是字符串/类JSON值且你需要整体消费时，优先选择
JSON*
格式（如
JSONEachRow
）——比如需要随机访问几乎每个字段、对其执行字符串/正则表达式方法、将值视为文本处理。V8原生的

JSON.parse

是经过高度优化的C++实现，构建JS字符串和对象的速度比JS层面的RowBinary解码器更快；搭配HTTP响应压缩（

gzip

zstd

，可大幅压缩JSON重复的键），还能降低网络传输成本。

当结果主要包含以下内容时，RowBinary 明显更具优势：

宽数值类型——

Int128

Int256

UInt128

UInt256

、

Decimal128

Decimal256

。

二进制/固定宽度二进制大对象——
```
IPv4
```
、
```
IPv6
```
、
```
UUID
```
、
```
FixedString
```
。
高容量固定宽度数值列，通常每个值只需一次
```
DataView
```
读取操作。

当列加载和客户端分析是主要目标时，优先选择
Native
格式（对列进行折叠/扫描/过滤，将类型化数组传入Worker或WASM）。

Native

是列优先的格式，因此可以直接加载到每列对应的类型化数组中，无需转置。

如果需要帮助选择和使用

JSON*

格式（或CSV/TSV），请使用**

clickhouse-js-node-coding

**技能。

Core guidance (both directions)

核心指南（双向通用）

These principles apply whether you are generating a reader or a writer; the side-specific operational guidance is in reader.md / writer.md.

Little-endian only. RowBinary is little-endian; target x86/ARM. Read and write every multi-byte number with
```
DataView
```
accessors passing a literal
```
true
```
for the
```
littleEndian
```
flag.
Correct first, then optimize. First emit a correct codec built from the plain per-type API. Only after it's correct (and tested) specialize it. Don't bake performance assumptions in before correctness.
Monomorphize generic/composite types. Emit specialized, inlined code per type combination instead of passing functions as arguments where the type is known ahead of time.
Inline the leaf ops. The per-type
```
readX
```
/
```
writeX
```
functions are the correct, composable reference; the generated codec should INLINE their bodies, not call them, so the row loop is straight-line with no per-field indirection (and so the fixed-width coalescing can fold the offset arithmetic together).
Annotate the type per column. Inlining erases the type structure, so put a short comment above each column's encode/decode block naming the ClickHouse type it handles.
Shared scratch is not reentrant. Some hot methods reuse a module-level scratch buffer as a write-then-read pair — correct only because the access is fully synchronous. An
```
async
```
/
```
yield
```
boundary between populating and reading it corrupts the value.
TypeScript by default. Generate TypeScript code and helpers unless the user explicitly asks for plain JavaScript.

这些原则适用于生成读取器或写入器的场景；各方向的操作指南请查看reader.md/writer.md。

仅支持小端序。RowBinary采用小端序；目标平台为x86/ARM。读取和写入每个多字节数字时，使用
```
DataView
```
访问器并为
```
littleEndian
```
参数传入字面量
true
。
先保证正确性，再优化。首先基于简单的按类型API生成正确的编解码器。只有在确保正确（并经过测试）之后，再进行优化。不要在确保正确性之前就加入性能假设。
单态化泛型/复合类型。针对每种类型组合生成专门的内联代码，而不是在类型提前已知的情况下将函数作为参数传递。
内联叶子操作。按类型划分的
```
readX
```
/
```
writeX
```
函数是正确、可组合的参考实现；生成的编解码器应内联它们的主体代码，而非调用它们，这样行循环就是直线执行的，没有每个字段的间接调用（并且固定宽度合并可以将偏移量运算合并在一起）。
为每列标注类型。内联会擦除类型结构，因此在每列的编码/解码块上方添加简短注释，说明它处理的ClickHouse类型。
共享临时缓冲区不可重入。一些高频方法会复用模块级别的临时缓冲区作为写入-读取对——这仅在访问完全同步时才正确。如果在填充和读取之间存在
```
async
```
/
```
yield
```
边界，会导致值损坏。
默认使用TypeScript。生成TypeScript代码及辅助工具，除非用户明确要求纯JavaScript。

Worked examples

实战示例

Six end-to-end examples with real speedup are catalogued in EXAMPLES.md.

EXAMPLES.md中收录了六个端到端的示例，均实现了实际的性能提升。

Out of scope

超出范围的内容

JSON / CSV / TSV / Parquet parsing → use
```
clickhouse-js-node-coding
```
.
Connection errors, hangs, type mismatches → use
```
clickhouse-js-node-troubleshooting
```
.
Browser / Web Worker / Edge →
```
@clickhouse/client-web
```
.

JSON/CSV/TSV/Parquet解析 → 使用
```
clickhouse-js-node-coding
```
技能。
连接错误、挂起、类型不匹配 → 使用
```
clickhouse-js-node-troubleshooting
```
技能。
浏览器/Web Worker/Edge环境 → 使用
```
@clickhouse/client-web
```
。