jq - JSON Processing

jq - JSON 处理

IMPORTANT:

jq

is the STRONGLY PREFERRED tool for ALL JSON formatting, parsing, manipulation, and analysis tasks. Use jq instead of Python/Node.js scripts, grep, awk, or other text processing tools when working with JSON data.

重要提示：

jq

是所有JSON格式化、解析、操作和分析任务的首选工具。处理JSON数据时，请使用jq替代Python/Node.js脚本、grep、awk或其他文本处理工具。

Core Philosophy

核心理念

Always use jq for JSON: If the data is JSON or can be converted to JSON, use jq
Streaming-friendly: jq processes JSON as a stream, making it memory-efficient for large files
Composable: jq filters can be chained with pipes, just like shell commands
Pure and functional: jq transformations are predictable and side-effect-free

始终使用jq处理JSON：如果数据是JSON格式或可转换为JSON，就使用jq
支持流处理：jq以流方式处理JSON，处理大文件时内存效率更高
可组合性：jq过滤器可以通过管道符链式调用，就像Shell命令一样
纯函数式：jq的数据转换可预测且无副作用

Basic Usage Patterns

基础使用模式

Pretty-Printing and Formatting

格式化输出与排版

bash

undefined

bash

undefined

Pretty-print JSON (most common use case)

格式化输出JSON（最常用场景）

cat file.json | jq '.' jq '.' file.json

Compact output (remove whitespace)

紧凑输出（移除空白字符）

jq -c '.' file.json

Sort keys alphabetically

按字母顺序排序键

jq -S '.' file.json

Raw output (no quotes for strings)

原始输出（字符串不带引号）

jq -r '.field' file.json

Null input (construct JSON from scratch)

空输入（从头构建JSON）

jq -n '{name: "value", count: 42}'

undefined

jq -n '{name: "value", count: 42}'

undefined

Selecting and Filtering

选择与过滤

bash

undefined

bash

undefined

Select a single field

选择单个字段

jq '.field' file.json

Select nested field

选择嵌套字段

jq '.user.email' file.json

Select array element by index

按索引选择数组元素

jq '.[0]' array.json jq '.[2:5]' array.json # Array slice

jq '.[0]' array.json jq '.[2:5]' array.json # 数组切片

Select multiple fields (create new object)

选择多个字段（创建新对象）

jq '{name: .name, email: .email}' file.json

Filter array elements

过滤数组元素

jq '.[] | select(.age > 21)' users.json

Filter with multiple conditions

多条件过滤

jq '.[] | select(.active == true and .role == "admin")' users.json

Filter and select fields

过滤并选择字段

jq '.[] | select(.price < 100) | {name, price}' products.json

undefined

jq '.[] | select(.price < 100) | {name, price}' products.json

undefined

Array Operations

数组操作

bash

undefined

bash

undefined

Map over array (transform each element)

映射数组（转换每个元素）

jq 'map(.name)' users.json jq '[.[] | .email]' users.json # Alternative syntax

jq 'map(.name)' users.json jq '[.[] | .email]' users.json # 替代语法

Filter then map

过滤后映射

jq 'map(select(.active)) | map(.name)' users.json

Get array length

获取数组长度

jq 'length' array.json jq '.items | length' file.json

Sort array

排序数组

jq 'sort' numbers.json jq 'sort_by(.created_at)' items.json

Reverse array

反转数组

jq 'reverse' array.json

Unique values

去重

jq 'unique' array.json jq 'unique_by(.category)' items.json

Group by field

按字段分组

jq 'group_by(.category)' items.json

Flatten nested arrays

展平嵌套数组

jq 'flatten' nested.json jq 'flatten(2)' deeply_nested.json # Flatten 2 levels

undefined

jq 'flatten' nested.json jq 'flatten(2)' deeply_nested.json # 展平2层

undefined

Aggregations and Statistics

聚合与统计

bash

undefined

bash

undefined

Sum values

求和

jq 'map(.price) | add' items.json jq '[.[] | .count] | add' data.json

Average

求平均值

jq 'map(.score) | add / length' scores.json

Min/max

最小值/最大值

jq 'map(.price) | min' products.json jq 'map(.price) | max' products.json jq 'min_by(.created_at)' items.json jq 'max_by(.score)' results.json

Count occurrences

统计出现次数

jq 'group_by(.status) | map({status: .[0].status, count: length})' items.json

undefined

jq 'group_by(.status) | map({status: .[0].status, count: length})' items.json

undefined

Transforming Data

数据转换

bash

undefined

bash

undefined

Add field

添加字段

jq '. + {new_field: "value"}' file.json

Update field

更新字段

jq '.price = .price * 1.1' product.json jq '.updated_at = now' record.json

Rename field

重命名字段

jq '{name: .old_name, other: .other}' file.json

Delete field

删除字段

jq 'del(.sensitive_data)' file.json

Conditional updates

条件更新

jq 'if .price > 100 then .category = "premium" else . end' product.json

Map with transformation

映射并转换

jq 'map(. + {full_name: "(.first_name) (.last_name)"})' users.json

undefined

jq 'map(. + {full_name: "(.first_name) (.last_name)"})' users.json

undefined

Combining and Merging

合并与组合

bash

undefined

bash

undefined

Merge objects

合并对象

jq '. + {extra: "data"}' file.json jq '. * {override: "value"}' file.json # Recursive merge

jq '. + {extra: "data"}' file.json jq '. * {override: "value"}' file.json # 递归合并

Combine arrays

合并数组

jq '. + [1,2,3]' array.json

Merge multiple files

合并多个文件

jq -s '.[0] + .[1]' file1.json file2.json

Slurp mode (combine into array)

合并模式（组合为数组）

jq -s '.' file1.json file2.json file3.json jq -s 'map(.items) | flatten' *.json

undefined

jq -s '.' file1.json file2.json file3.json jq -s 'map(.items) | flatten' *.json

undefined

Working with Keys

键操作

bash

undefined

bash

undefined

Get all keys

获取所有键

jq 'keys' object.json jq 'keys_unsorted' object.json

Check if key exists

检查键是否存在

jq 'has("field")' file.json

Get values

获取值

jq 'values' object.json jq '.[] | values' array.json # Filter out nulls

jq 'values' object.json jq '.[] | values' array.json # 过滤空值

Convert object to array of key-value pairs

将对象转换为键值对数组

jq 'to_entries' object.json jq 'to_entries | map({key: .key, value: .value})' object.json

Convert array of pairs back to object

将键值对数组转换回对象

jq 'from_entries' pairs.json

undefined

jq 'from_entries' pairs.json

undefined

String Operations

字符串操作

bash

undefined

bash

undefined

String interpolation

字符串插值

jq '"(.first_name) (.last_name)"' user.json

String functions

字符串函数

jq '.name | ascii_downcase' file.json jq '.email | ascii_upcase' file.json jq '.text | ltrimstr("prefix:")' file.json jq '.url | rtrimstr(".html")' file.json

Split and join

分割与连接

jq '.tags | split(",")' file.json jq '.words | join(" ")' file.json

Regular expressions

正则表达式

jq '.email | test("@example\.com$")' user.json jq '.text | match("\b\w+@\w+\.\w+\b")' content.json jq '.name | sub("old"; "new")' file.json jq '.text | gsub("\s+"; " ")' file.json # Replace all

undefined

jq '.email | test("@example\.com$")' user.json jq '.text | match("\b\w+@\w+\.\w+\b")' content.json jq '.name | sub("old"; "new")' file.json jq '.text | gsub("\s+"; " ")' file.json # 替换所有匹配项

undefined

Conditional Logic

条件逻辑

bash

undefined

bash

undefined

Simple if-then-else

简单if-then-else

jq 'if .age >= 18 then "adult" else "minor" end' user.json

Multiple conditions

多条件判断

jq 'if .score > 90 then "A" elif .score > 80 then "B" else "C" end' result.json

Alternative operator (handle null/false)

备选运算符（处理null/false）

jq '.optional_field // "default"' file.json

Try-catch (handle errors gracefully)

异常捕获（优雅处理错误）

jq 'try .field.nested catch "not found"' file.json

undefined

jq 'try .field.nested catch "not found"' file.json

undefined

Type Conversions

类型转换

bash

undefined

bash

undefined

Convert to string

转换为字符串

jq 'tostring' number.json

Convert to number

转换为数字

jq 'tonumber' string.json

Type checking

类型检查

jq 'type' value.json jq '.[] | select(type == "number")' mixed.json

Array/object detection

数组/对象检测

jq 'if type == "array" then length else 1 end' value.json

undefined

jq 'if type == "array" then length else 1 end' value.json

undefined

Output Formatting

输出格式化

bash

undefined

bash

undefined

Tab-separated values

制表符分隔值

jq -r '.[] | [.name, .email, .age] | @tsv' users.json

CSV output

CSV输出

jq -r '.[] | [.name, .email, .age] | @csv' users.json

URL encoding

URL编码

jq -r '@uri' string.json

Base64 encoding/decoding

Base64编码/解码

jq -r '@base64' string.json jq -r '@base64d' encoded.json

HTML encoding

HTML编码

jq -r '@html' string.json

Shell escaping

Shell转义

jq -r '@sh' command.json

undefined

jq -r '@sh' command.json

undefined

Advanced Patterns

高级模式

bash

undefined

bash

undefined

Recursive descent (search all levels)

递归遍历（搜索所有层级）

jq '.. | .id? | select(. != null)' nested.json

Walk (apply transformation recursively)

递归转换（对所有层级应用转换）

jq 'walk(if type == "string" then ascii_downcase else . end)' file.json

Path queries

路径查询

jq 'path(.items[].name)' file.json jq 'getpath(["items", 0, "name"])' file.json

Limit output

限制输出数量

jq 'limit(5; .[] | select(.active))' items.json

Until condition

循环直到满足条件

jq 'until(. > 100; . * 2)' number.json

Reduce (fold)

归约（折叠）

jq 'reduce .[] as $item (0; . + $item.count)' items.json jq 'reduce .[] as $item ({}; . + {($item.key): $item.value})' pairs.json

undefined

jq 'reduce .[] as $item (0; . + $item.count)' items.json jq 'reduce .[] as $item ({}; . + {($item.key): $item.value})' pairs.json

undefined

Working with API Responses

API响应处理

bash

undefined

bash

undefined

Pretty-print API response

格式化输出API响应

curl -s api.example.com/data | jq '.'

Extract specific fields from API

从API响应中提取特定字段

curl -s api.example.com/users | jq '.[] | {id, name, email}'

Filter and transform

过滤与转换

curl -s api.example.com/items | jq 'map(select(.active)) | sort_by(.created_at) | reverse | .[0:10]'

Pagination handling

分页处理

for page in {1..5}; do curl -s "api.example.com/items?page=$page" | jq '.items[]' done | jq -s '.'

Error handling

错误处理

curl -s api.example.com/data | jq 'if .error then .error.message else .data end'

undefined

curl -s api.example.com/data | jq 'if .error then .error.message else .data end'

undefined

Log Analysis

日志分析

bash

undefined

bash

undefined

Parse JSON logs

解析JSON日志

cat app.log | jq -R 'fromjson? | select(.level == "error")'

Group errors by type

按类型分组错误

cat app.log | jq -R 'fromjson? | select(.level == "error")' | jq -s 'group_by(.error_type) | map({type: .[0].error_type, count: length})'

Time-based filtering

按时间过滤

cat app.log | jq -R 'fromjson? | select(.timestamp > "2024-01-01")'

Extract stack traces

提取堆栈跟踪

cat app.log | jq -R 'fromjson? | select(.stack_trace) | .stack_trace'

undefined

cat app.log | jq -R 'fromjson? | select(.stack_trace) | .stack_trace'

undefined

Multi-File Processing

多文件处理

bash

undefined

bash

undefined

Merge multiple JSON files

合并多个JSON文件

jq -s 'add' file1.json file2.json file3.json

Process each file separately, collect results

单独处理每个文件，收集结果

jq -s 'map(.)' *.json

Cross-file analysis

跨文件分析

jq -s 'map(.items) | flatten | group_by(.category)' *.json

Join files (like SQL join)

连接文件（类似SQL连接）

jq -s '.[0].users as $users | .[1].orders | map(. + {user: ($users[] | select(.id == .user_id))})' users.json orders.json

undefined

jq -s '.[0].users as $users | .[1].orders | map(. + {user: ($users[] | select(.id == .user_id))})' users.json orders.json

undefined

Performance Optimization

性能优化

bash

undefined

bash

undefined

Streaming mode for large files (don't load entire file)

大文件流处理模式（不加载整个文件）

jq --stream 'select(length == 2)' huge.json

Process line-by-line for NDJSON (newline-delimited JSON)

逐行处理NDJSON（换行分隔的JSON）

cat data.ndjson | jq -c '.'

Use compact output to reduce size

使用紧凑输出减少文件大小

jq -c '.' file.json > compressed.json

Limit memory by processing incrementally

增量处理以限制内存使用

cat stream.json | jq -c 'select(.important)' >> filtered.json

undefined

cat stream.json | jq -c 'select(.important)' >> filtered.json

undefined

Debugging

调试

bash

undefined

bash

undefined

Debug mode (show intermediate values)

调试模式（显示中间值）

jq --debug '.field' file.json

Print to stderr while continuing

输出到stderr同时继续执行

jq 'debug | .field' file.json

Add debugging output

添加调试输出

jq '. as $orig | .field | debug | $orig' file.json

Validate JSON

验证JSON格式

jq empty file.json # No output = valid JSON

jq empty file.json # 无输出表示JSON有效

Check for null fields

检查空字段

jq 'paths(. == null)' file.json

undefined

jq 'paths(. == null)' file.json

undefined

Common Patterns for Claude Code Tasks

Claude代码任务常用模式

Extract Configuration Values

提取配置值

bash

undefined

bash

undefined

Get specific config value

获取特定配置值

jq -r '.database.host' config.json

Get all environment variables

获取所有环境变量

jq -r '.env | to_entries | .[] | "(.key)=(.value)"' config.json

undefined

jq -r '.env | to_entries | .[] | "(.key)=(.value)"' config.json

undefined

Process API Responses

处理API响应

bash

undefined

bash

undefined

Extract IDs from response

从响应中提取ID

gh api /repos/owner/repo/issues | jq '.[].number'

Format for display

格式化输出用于展示

gh api /repos/owner/repo/pulls | jq -r '.[] | "(.number): (.title)"'

undefined

gh api /repos/owner/repo/pulls | jq -r '.[] | "(.number): (.title)"'

undefined

Transform Test Results

转换测试结果

bash

undefined

bash

undefined

Parse test output

解析测试输出

cat test-results.json | jq '{total: .stats.tests, passed: .stats.passes, failed: .stats.failures}'

Find failed tests

查找失败的测试用例

cat test-results.json | jq '.tests[] | select(.state == "failed") | .title'

undefined

cat test-results.json | jq '.tests[] | select(.state == "failed") | .title'

undefined

Modify Package Files

修改包文件

bash

undefined

bash

undefined

Update package.json version

更新package.json版本

jq '.version = "2.0.0"' package.json > package.json.tmp && mv package.json.tmp package.json

Add dependency

添加依赖

jq '.dependencies["new-package"] = "^1.0.0"' package.json > package.json.tmp && mv package.json.tmp package.json

Remove dev dependency

删除开发依赖

jq 'del(.devDependencies["old-package"])' package.json > package.json.tmp && mv package.json.tmp package.json

undefined

jq 'del(.devDependencies["old-package"])' package.json > package.json.tmp && mv package.json.tmp package.json

undefined

When NOT to Use jq

不适合使用jq的场景

Binary data: jq is for text-based JSON only
Very simple extraction: For trivial field access where grep would suffice
Modifying files in place: jq doesn't support in-place editing (use temp files)
Non-JSON data: Use appropriate tools (xsv for CSV, yq for YAML, etc.)

二进制数据：jq仅适用于基于文本的JSON
极简单的提取：对于非常简单的字段提取，grep足够的情况下
原地修改文件：jq不支持原地编辑文件（需使用临时文件）
非JSON数据：使用对应工具（如xsv处理CSV，yq处理YAML等）

Error Messages and Troubleshooting

错误信息与故障排除

Common Errors

常见错误

bash

undefined

bash

undefined

"parse error: Invalid numeric literal"

→ JSON has invalid syntax, validate with: jq empty file.json

→ JSON语法无效，使用以下命令验证：jq empty file.json

"jq: error: Cannot index string with string"

→ Trying to access field on non-object, check types first

→ 尝试在非对象上访问字段，请先检查类型

"jq: error: null cannot be parsed as JSON"

→ Input is null or empty, use: jq -R 'fromjson?'

→ 输入为空或null，使用：jq -R 'fromjson?'

"Cannot iterate over null"

→ Field doesn't exist, use: jq '.field // []'

→ 字段不存在，使用：jq '.field // []'

undefined

undefined

Validation

验证

bash

undefined

bash

undefined

Validate JSON syntax

验证JSON语法

jq empty file.json && echo "Valid JSON" || echo "Invalid JSON"

Pretty-print to find errors

格式化输出以查找错误

jq '.' file.json

undefined

jq '.' file.json

undefined

Performance Considerations

性能注意事项

Use
-c
for compact output when piping to other commands
Stream large files with
```
--stream
```
or process line-by-line for NDJSON
Filter early to reduce data size before complex transformations
Avoid unnecessary array creation - use streaming operations when possible
Use built-in functions like
```
map
```
,
```
select
```
,
```
group_by
```
instead of manual loops

管道传输时使用
-c
生成紧凑输出
处理大文件时使用流模式或逐行处理NDJSON
尽早过滤数据，在复杂转换前减少数据量
避免不必要的数组创建 - 尽可能使用流操作
使用内置函数如
```
map
```
、
```
select
```
、
```
group_by
```
替代手动循环

Integration with Other Tools

与其他工具集成

bash

undefined

bash

undefined

With curl

与curl集成

curl -s api.example.com/data | jq '.results[]'

With grep (for NDJSON)

与grep集成（处理NDJSON）

cat logs.ndjson | grep "error" | jq '.'

With awk

与awk集成

cat data.json | jq -r '.[] | @tsv' | awk -F'\t' '{print $1, $3}'

With xargs

与xargs集成

jq -r '.files[]' list.json | xargs -I {} cp {} dest/

With parallel processing

与并行处理集成

cat items.json | jq -c '.[]' | parallel -j4 'echo {} | jq ".id"'

undefined

cat items.json | jq -c '.[]' | parallel -j4 'echo {} | jq ".id"'

undefined

Resources

资源

Official manual: https://stedolan.github.io/jq/manual/
jq play (online tester): https://jqplay.org/
Cookbook: https://github.com/stedolan/jq/wiki/Cookbook

官方手册：https://stedolan.github.io/jq/manual/
jq在线测试工具：https://jqplay.org/
参考手册：https://github.com/stedolan/jq/wiki/Cookbook

jq

Original

Translation

jq - JSON Processing

jq - JSON 处理

Core Philosophy

核心理念

Basic Usage Patterns

基础使用模式

Pretty-Printing and Formatting

格式化输出与排版

Pretty-print JSON (most common use case)

格式化输出JSON（最常用场景）

Compact output (remove whitespace)

紧凑输出（移除空白字符）

Sort keys alphabetically

按字母顺序排序键

Raw output (no quotes for strings)

原始输出（字符串不带引号）

Null input (construct JSON from scratch)

空输入（从头构建JSON）

Selecting and Filtering

选择与过滤

Select a single field

选择单个字段

Select nested field

选择嵌套字段

Select array element by index

按索引选择数组元素

Select multiple fields (create new object)

选择多个字段（创建新对象）

Filter array elements

过滤数组元素

Filter with multiple conditions

多条件过滤

Filter and select fields

过滤并选择字段

Array Operations

数组操作

Map over array (transform each element)

映射数组（转换每个元素）

Filter then map

过滤后映射

Get array length

获取数组长度

Sort array

排序数组

Reverse array

反转数组

Unique values

去重

Group by field

按字段分组

Flatten nested arrays

展平嵌套数组

Aggregations and Statistics

聚合与统计

Sum values

求和

Average

求平均值

Min/max

最小值/最大值

Count occurrences

统计出现次数

Transforming Data

数据转换

Add field

添加字段

Update field

更新字段

Rename field

重命名字段

Delete field

删除字段

Conditional updates

条件更新

Map with transformation

映射并转换

Combining and Merging