ClickHouse JS RowBinary Codec Generator for Node.js
This skill generates both directions of the wire format: readers (decode
bytes → values) and writers (encode values → bytes, the mirror). A given
task normally needs only one side. This file is the shared entry point — the
format gate plus the principles common to both directions; the per-direction
decisions, guidance, and the per-type reference tables live in two sibling files.
Pick your side — read only the one you need:
- Decoding a response from ClickHouse into JS values →
reader.md. Streaming vs whole-buffer, row-objects vs columnar,
fixed vs runtime schema, and the per-type reader reference.
- Encoding JS values into a payload to send to ClickHouse →
writer.md. The / building blocks,
streaming, and the per-type writer reference.
The per-type code is real, split by direction under
and
.
First: is RowBinary even the right format?
RowBinary exists for throughput, but it is not automatically the fastest
path — match the format to the shape of the data before committing to a
bespoke parser.
Prefer a format (e.g. ) when the result is mostly
strings / JSON-like values that you consume wholesale — randomly accessing
essentially every field, running string/regexp methods on them, treating values
as text. V8's native
is heavily optimized C++ and builds JS strings
and objects faster than a JS-level RowBinary decoder can; pair it with HTTP
response compression (
/
, which crushes JSON's repetitive keys) and
the wire cost shrinks too.
RowBinary clearly wins when the result is dominated by:
- Wide numerics — ///,
/.
- Binary / fixed-width blobs — , , , .
- High-volume fixed-width numeric columns generally, where each value is a
single read.
Prefer the format when columnar load and client-side analytics are
the main goal (fold/scan/filter columns, feed typed arrays to a Worker or WASM).
is column-major, so it loads straight into one typed array per column
with no transpose.
For help choosing and consuming a
format (or CSV / TSV) instead, use the
clickhouse-js-node-coding
skill.
Core guidance (both directions)
These principles apply whether you are generating a reader or a writer; the
side-specific operational guidance is in reader.md /
writer.md.
-
Little-endian only. RowBinary is little-endian; target x86/ARM. Read and
write every multi-byte number with
accessors passing a
literal
for the
flag.
-
Correct first, then optimize. First emit a correct codec built from the
plain per-type API. Only after it's correct (and tested) specialize it. Don't
bake performance assumptions in before correctness.
-
Monomorphize generic/composite types. Emit specialized, inlined code per
type combination instead of passing functions as arguments where the type is
known ahead of time.
-
Inline the leaf ops. The per-type
/
functions are the
correct, composable reference; the generated codec should INLINE their bodies,
not call them, so the row loop is straight-line with no per-field indirection
(and so the fixed-width coalescing can fold the offset arithmetic together).
-
Annotate the type per column. Inlining erases the type structure, so put a
short comment above each column's encode/decode block naming the ClickHouse
type it handles.
-
Shared scratch is not reentrant. Some hot methods reuse a module-level
scratch buffer as a write-then-read pair — correct only because the access is
fully synchronous. An
/
boundary between populating and reading
it corrupts the value.
-
TypeScript by default. Generate TypeScript code and helpers unless the user
explicitly asks for plain JavaScript.
Worked examples
Six end-to-end examples with real speedup are catalogued in EXAMPLES.md.
Out of scope
- JSON / CSV / TSV / Parquet parsing → use
clickhouse-js-node-coding
- Connection errors, hangs, type mismatches → use
clickhouse-js-node-troubleshooting
- Browser / Web Worker / Edge → .
Still Stuck?