database-query-and-export
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseDatabase Query and Export
数据库查询与导出
Query relational databases (SQLite, PostgreSQL, MySQL) and export results to CSV, JSON, or other formats. Essential for data extraction, reporting, backup automation, and analytics pipelines.
查询关系型数据库(SQLite、PostgreSQL、MySQL)并将结果导出为CSV、JSON或其他格式。这是数据提取、报告生成、备份自动化和分析流程中的必备操作。
When to use
适用场景
- Use case 1: When the user asks to query a database and export results
- Use case 2: When you need to extract data for analysis or reporting
- Use case 3: For backup and data migration workflows
- Use case 4: When building automated database monitoring and alerts
- 场景1:用户要求查询数据库并导出结果时
- 场景2:需要提取数据用于分析或报告时
- 场景3:用于备份和数据迁移流程
- 场景4:构建自动化数据库监控与告警时
Required tools / APIs
所需工具/API
- SQLite — Lightweight file-based database (often pre-installed)
- PostgreSQL client — For PostgreSQL databases
- MySQL client — For MySQL/MariaDB databases
- No external API required
Install options:
bash
undefined- SQLite — 轻量级文件型数据库(通常已预装)
- PostgreSQL client — 用于PostgreSQL数据库
- MySQL client — 用于MySQL/MariaDB数据库
- 无需外部API
安装选项:
bash
undefinedUbuntu/Debian
Ubuntu/Debian
sudo apt-get install -y sqlite3 postgresql-client mysql-client
sudo apt-get install -y sqlite3 postgresql-client mysql-client
macOS
macOS
brew install sqlite3 postgresql mysql-client
brew install sqlite3 postgresql mysql-client
Node.js (database drivers)
Node.js (数据库驱动)
npm install better-sqlite3 # SQLite
npm install pg # PostgreSQL
npm install mysql2 # MySQL
undefinednpm install better-sqlite3 # SQLite
npm install pg # PostgreSQL
npm install mysql2 # MySQL
undefinedSkills
技能
query_sqlite_to_json
query_sqlite_to_json
Query SQLite database and export to JSON format.
bash
undefined查询SQLite数据库并导出为JSON格式。
bash
undefinedBasic query to JSON
基础查询导出为JSON
sqlite3 database.db "SELECT * FROM users LIMIT 10;" -json
sqlite3 database.db "SELECT * FROM users LIMIT 10;" -json
With pretty formatting using jq
使用jq进行格式化输出
sqlite3 database.db "SELECT * FROM users WHERE active=1;" -json | jq '.'
sqlite3 database.db "SELECT * FROM users WHERE active=1;" -json | jq '.'
Export entire table to JSON file
将整张表导出为JSON文件
sqlite3 database.db "SELECT * FROM orders;" -json > orders.json
sqlite3 database.db "SELECT * FROM orders;" -json > orders.json
Query with WHERE clause
带WHERE子句的查询
sqlite3 database.db "SELECT id, name, email FROM users WHERE created_at > '2024-01-01';" -json
**Node.js:**
```javascript
const Database = require('better-sqlite3');
function querySQLiteToJSON(dbPath, query) {
const db = new Database(dbPath, { readonly: true });
const rows = db.prepare(query).all();
db.close();
return rows;
}
// Usage
// const users = querySQLiteToJSON('./database.db', 'SELECT * FROM users LIMIT 10');
// console.log(JSON.stringify(users, null, 2));sqlite3 database.db "SELECT id, name, email FROM users WHERE created_at > '2024-01-01';" -json
**Node.js:**
```javascript
const Database = require('better-sqlite3');
function querySQLiteToJSON(dbPath, query) {
const db = new Database(dbPath, { readonly: true });
const rows = db.prepare(query).all();
db.close();
return rows;
}
// 使用示例
// const users = querySQLiteToJSON('./database.db', 'SELECT * FROM users LIMIT 10');
// console.log(JSON.stringify(users, null, 2));query_sqlite_to_csv
query_sqlite_to_csv
Query SQLite database and export to CSV format.
bash
undefined查询SQLite数据库并导出为CSV格式。
bash
undefinedBasic query to CSV
基础查询导出为CSV
sqlite3 database.db <<EOF
.mode csv
.headers on
SELECT * FROM users LIMIT 10;
EOF
sqlite3 database.db <<EOF
.mode csv
.headers on
SELECT * FROM users LIMIT 10;
EOF
Export to CSV file
导出为CSV文件
sqlite3 database.db <<EOF
.mode csv
.headers on
.output users.csv
SELECT id, name, email, created_at FROM users WHERE active=1;
EOF
sqlite3 database.db <<EOF
.mode csv
.headers on
.output users.csv
SELECT id, name, email, created_at FROM users WHERE active=1;
EOF
Query multiple tables with JOIN
多表JOIN查询
sqlite3 database.db <<EOF
.mode csv
.headers on
SELECT u.name, o.order_id, o.total
FROM users u
JOIN orders o ON u.id = o.user_id
WHERE o.created_at > '2024-01-01';
EOF
**Node.js:**
```javascript
const Database = require('better-sqlite3');
const fs = require('fs');
function querySQLiteToCSV(dbPath, query, outputPath) {
const db = new Database(dbPath, { readonly: true });
const rows = db.prepare(query).all();
db.close();
if (rows.length === 0) {
return 'No results';
}
// Generate CSV
const headers = Object.keys(rows[0]).join(',');
const csvRows = rows.map(row =>
Object.values(row).map(val =>
typeof val === 'string' && val.includes(',') ? `"${val}"` : val
).join(',')
);
const csv = [headers, ...csvRows].join('\n');
if (outputPath) {
fs.writeFileSync(outputPath, csv);
return `Exported ${rows.length} rows to ${outputPath}`;
}
return csv;
}
// Usage
// querySQLiteToCSV('./database.db', 'SELECT * FROM users LIMIT 10', './users.csv');sqlite3 database.db <<EOF
.mode csv
.headers on
SELECT u.name, o.order_id, o.total
FROM users u
JOIN orders o ON u.id = o.user_id
WHERE o.created_at > '2024-01-01';
EOF
**Node.js:**
```javascript
const Database = require('better-sqlite3');
const fs = require('fs');
function querySQLiteToCSV(dbPath, query, outputPath) {
const db = new Database(dbPath, { readonly: true });
const rows = db.prepare(query).all();
db.close();
if (rows.length === 0) {
return '无查询结果';
}
// 生成CSV
const headers = Object.keys(rows[0]).join(',');
const csvRows = rows.map(row =>
Object.values(row).map(val =>
typeof val === 'string' && val.includes(',') ? `"${val}"` : val
).join(',')
);
const csv = [headers, ...csvRows].join('\n');
if (outputPath) {
fs.writeFileSync(outputPath, csv);
return `已将${rows.length}条数据导出至${outputPath}`;
}
return csv;
}
// 使用示例
// querySQLiteToCSV('./database.db', 'SELECT * FROM users LIMIT 10', './users.csv');query_postgresql
query_postgresql
Query PostgreSQL database and export results.
bash
undefined查询PostgreSQL数据库并导出结果。
bash
undefinedSet connection string (alternative: use individual flags)
设置连接字符串(替代方案:使用单独的参数)
export PGHOST=localhost
export PGPORT=5432
export PGDATABASE=mydb
export PGUSER=postgres
export PGPASSWORD=mypassword
export PGHOST=localhost
export PGPORT=5432
export PGDATABASE=mydb
export PGUSER=postgres
export PGPASSWORD=mypassword
Query to JSON (using psql with formatted output)
查询导出为JSON(使用psql格式化输出)
psql -t -A -F"," -c "SELECT row_to_json(t) FROM (SELECT * FROM users LIMIT 10) t;"
psql -t -A -F"," -c "SELECT row_to_json(t) FROM (SELECT * FROM users LIMIT 10) t;"
Query to CSV
查询导出为CSV
psql -c "COPY (SELECT * FROM users WHERE active=true) TO STDOUT WITH CSV HEADER;" > users.csv
psql -c "COPY (SELECT * FROM users WHERE active=true) TO STDOUT WITH CSV HEADER;" > users.csv
Query with connection string
使用连接字符串查询
psql "postgresql://user:password@localhost:5432/mydb" -c "SELECT * FROM users LIMIT 5;"
psql "postgresql://user:password@localhost:5432/mydb" -c "SELECT * FROM users LIMIT 5;"
Query to formatted table
查询导出为格式化表格
psql -c "SELECT id, name, email FROM users ORDER BY created_at DESC LIMIT 10;"
**Node.js:**
```javascript
const { Pool } = require('pg');
async function queryPostgreSQL(connectionString, query) {
const pool = new Pool({ connectionString });
try {
const result = await pool.query(query);
return result.rows;
} finally {
await pool.end();
}
}
// Usage
// const connStr = 'postgresql://user:password@localhost:5432/mydb';
// queryPostgreSQL(connStr, 'SELECT * FROM users LIMIT 10')
// .then(rows => console.log(JSON.stringify(rows, null, 2)));psql -c "SELECT id, name, email FROM users ORDER BY created_at DESC LIMIT 10;"
**Node.js:**
```javascript
const { Pool } = require('pg');
async function queryPostgreSQL(connectionString, query) {
const pool = new Pool({ connectionString });
try {
const result = await pool.query(query);
return result.rows;
} finally {
await pool.end();
}
}
// 使用示例
// const connStr = 'postgresql://user:password@localhost:5432/mydb';
// queryPostgreSQL(connStr, 'SELECT * FROM users LIMIT 10')
// .then(rows => console.log(JSON.stringify(rows, null, 2)));query_mysql
query_mysql
Query MySQL/MariaDB database and export results.
bash
undefined查询MySQL/MariaDB数据库并导出结果。
bash
undefinedQuery to CSV with headers
查询导出为带表头的CSV
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users WHERE active=1;"
--batch --silent
| cat > users.csv
-e "SELECT * FROM users WHERE active=1;"
--batch --silent
| cat > users.csv
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users WHERE active=1;"
--batch --silent
| cat > users.csv
-e "SELECT * FROM users WHERE active=1;"
--batch --silent
| cat > users.csv
Query to JSON-like format (requires jq for proper formatting)
查询导出为类JSON格式(需要jq进行正确格式化)
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users LIMIT 10;"
--batch --silent
-e "SELECT * FROM users LIMIT 10;"
--batch --silent
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users LIMIT 10;"
--batch --silent
-e "SELECT * FROM users LIMIT 10;"
--batch --silent
Export entire table to CSV
将整张表导出为CSV
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM orders INTO OUTFILE '/tmp/orders.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';"
-e "SELECT * FROM orders INTO OUTFILE '/tmp/orders.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';"
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM orders INTO OUTFILE '/tmp/orders.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';"
-e "SELECT * FROM orders INTO OUTFILE '/tmp/orders.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';"
Query with timeout
带超时设置的查询
mysql -h localhost -u root -p'mypassword' -D mydb
--connect-timeout=10
-e "SELECT COUNT(*) as total FROM users;"
--connect-timeout=10
-e "SELECT COUNT(*) as total FROM users;"
**Node.js:**
```javascript
const mysql = require('mysql2/promise');
async function queryMySQL(config, query) {
const connection = await mysql.createConnection({
host: config.host || 'localhost',
user: config.user,
password: config.password,
database: config.database,
connectTimeout: 10000
});
try {
const [rows] = await connection.execute(query);
return rows;
} finally {
await connection.end();
}
}
// Usage
// const config = {
// host: 'localhost',
// user: 'root',
// password: 'mypassword',
// database: 'mydb'
// };
// queryMySQL(config, 'SELECT * FROM users LIMIT 10')
// .then(rows => console.log(JSON.stringify(rows, null, 2)));mysql -h localhost -u root -p'mypassword' -D mydb
--connect-timeout=10
-e "SELECT COUNT(*) as total FROM users;"
--connect-timeout=10
-e "SELECT COUNT(*) as total FROM users;"
**Node.js:**
```javascript
const mysql = require('mysql2/promise');
async function queryMySQL(config, query) {
const connection = await mysql.createConnection({
host: config.host || 'localhost',
user: config.user,
password: config.password,
database: config.database,
connectTimeout: 10000
});
try {
const [rows] = await connection.execute(query);
return rows;
} finally {
await connection.end();
}
}
// 使用示例
// const config = {
// host: 'localhost',
// user: 'root',
// password: 'mypassword',
// database: 'mydb'
// };
// queryMySQL(config, 'SELECT * FROM users LIMIT 10')
// .then(rows => console.log(JSON.stringify(rows, null, 2)));advanced_sqlite_export_with_error_handling
advanced_sqlite_export_with_error_handling
Production-ready SQLite export with validation and error handling.
bash
#!/bin/bash
DB_PATH="database.db"
QUERY="SELECT * FROM users WHERE active=1;"
OUTPUT_FILE="users.csv"带验证和错误处理的生产级SQLite导出功能。
bash
#!/bin/bash
DB_PATH="database.db"
QUERY="SELECT * FROM users WHERE active=1;"
OUTPUT_FILE="users.csv"Check if database exists
检查数据库是否存在
if [ ! -f "$DB_PATH" ]; then
echo "Error: Database file not found: $DB_PATH" >&2
exit 1
fi
if [ ! -f "$DB_PATH" ]; then
echo "错误:未找到数据库文件:$DB_PATH" >&2
exit 1
fi
Check if table exists
检查表是否存在
if ! sqlite3 "$DB_PATH" "SELECT name FROM sqlite_master WHERE type='table' AND name='users';" | grep -q "users"; then
echo "Error: Table 'users' not found in database" >&2
exit 1
fi
if ! sqlite3 "$DB_PATH" "SELECT name FROM sqlite_master WHERE type='table' AND name='users';" | grep -q "users"; then
echo "错误:数据库中不存在'users'表" >&2
exit 1
fi
Execute query and export to CSV
执行查询并导出为CSV
if sqlite3 "$DB_PATH" <<EOF > "$OUTPUT_FILE" 2>&1
.mode csv
.headers on
$QUERY
EOF
then
ROW_COUNT=$(wc -l < "$OUTPUT_FILE")
echo "Success: Exported $((ROW_COUNT - 1)) rows to $OUTPUT_FILE"
else
echo "Error: Query failed" >&2
exit 1
fi
**Node.js:**
```javascript
const Database = require('better-sqlite3');
const fs = require('fs');
async function exportSQLiteWithValidation(options) {
const { dbPath, query, outputPath, format = 'json' } = options;
// Validate database exists
if (!fs.existsSync(dbPath)) {
throw new Error(`Database file not found: ${dbPath}`);
}
let db;
try {
db = new Database(dbPath, { readonly: true, timeout: 10000 });
// Prepare and execute query
const stmt = db.prepare(query);
const rows = stmt.all();
if (rows.length === 0) {
return { success: true, rowCount: 0, message: 'No rows returned' };
}
// Export based on format
let output;
if (format === 'json') {
output = JSON.stringify(rows, null, 2);
} else if (format === 'csv') {
const headers = Object.keys(rows[0]).join(',');
const csvRows = rows.map(row =>
Object.values(row).map(val =>
typeof val === 'string' && val.includes(',') ? `"${val.replace(/"/g, '""')}"` : val
).join(',')
);
output = [headers, ...csvRows].join('\n');
} else {
throw new Error(`Unsupported format: ${format}`);
}
// Write to file
fs.writeFileSync(outputPath, output);
return {
success: true,
rowCount: rows.length,
outputPath,
format,
message: `Exported ${rows.length} rows to ${outputPath}`
};
} catch (err) {
throw new Error(`Database export failed: ${err.message}`);
} finally {
if (db) db.close();
}
}
// Usage
// exportSQLiteWithValidation({
// dbPath: './database.db',
// query: 'SELECT * FROM users WHERE active=1',
// outputPath: './users.json',
// format: 'json'
// }).then(result => console.log(result));if sqlite3 "$DB_PATH" <<EOF > "$OUTPUT_FILE" 2>&1
.mode csv
.headers on
$QUERY
EOF
then
ROW_COUNT=$(wc -l < "$OUTPUT_FILE")
echo "成功:已将$((ROW_COUNT - 1))条数据导出至$OUTPUT_FILE"
else
echo "错误:查询失败" >&2
exit 1
fi
**Node.js:**
```javascript
const Database = require('better-sqlite3');
const fs = require('fs');
async function exportSQLiteWithValidation(options) {
const { dbPath, query, outputPath, format = 'json' } = options;
// 验证数据库是否存在
if (!fs.existsSync(dbPath)) {
throw new Error(`未找到数据库文件:${dbPath}`);
}
let db;
try {
db = new Database(dbPath, { readonly: true, timeout: 10000 });
// 准备并执行查询
const stmt = db.prepare(query);
const rows = stmt.all();
if (rows.length === 0) {
return { success: true, rowCount: 0, message: '无数据返回' };
}
// 根据格式导出
let output;
if (format === 'json') {
output = JSON.stringify(rows, null, 2);
} else if (format === 'csv') {
const headers = Object.keys(rows[0]).join(',');
const csvRows = rows.map(row =>
Object.values(row).map(val =>
typeof val === 'string' && val.includes(',') ? `"${val.replace(/"/g, '""')}"` : val
).join(',')
);
output = [headers, ...csvRows].join('\n');
} else {
throw new Error(`不支持的格式:${format}`);
}
// 写入文件
fs.writeFileSync(outputPath, output);
return {
success: true,
rowCount: rows.length,
outputPath,
format,
message: `已将${rows.length}条数据导出至${outputPath}`
};
} catch (err) {
throw new Error(`数据库导出失败:${err.message}`);
} finally {
if (db) db.close();
}
}
// 使用示例
// exportSQLiteWithValidation({
// dbPath: './database.db',
// query: 'SELECT * FROM users WHERE active=1',
// outputPath: './users.json',
// format: 'json'
// }).then(result => console.log(result));Rate limits / Best practices
速率限制/最佳实践
- ✅ Use readonly connections — Open databases in readonly mode when only querying
- ✅ Set connection timeouts — Use 10-30 second timeouts to prevent hanging
- ✅ Validate inputs — Check that database files/tables exist before querying
- ✅ Escape user inputs — Use parameterized queries to prevent SQL injection
- ✅ Handle large datasets — Use LIMIT/OFFSET for pagination on large tables
- ✅ Close connections — Always close database connections after queries
- ⚠️ Secure credentials — Store database passwords in environment variables, never hardcode
- ⚠️ Export file permissions — Ensure export directories have proper write permissions
- ✅ 使用只读连接 — 仅查询时以只读模式打开数据库
- ✅ 设置连接超时 — 使用10-30秒超时时间避免进程挂起
- ✅ 验证输入 — 查询前检查数据库文件/表是否存在
- ✅ 转义用户输入 — 使用参数化查询防止SQL注入
- ✅ 处理大型数据集 — 对大表使用LIMIT/OFFSET进行分页
- ✅ 关闭连接 — 查询后始终关闭数据库连接
- ⚠️ 安全存储凭证 — 将数据库密码存储在环境变量中,切勿硬编码
- ⚠️ 设置导出文件权限 — 确保导出目录拥有正确的写入权限
Agent prompt
Agent提示词
text
You have database query and export capability. When a user asks to query a database:
1. Identify the database type (SQLite, PostgreSQL, MySQL) from:
- File extension (.db, .sqlite, .sqlite3 = SQLite)
- Connection string (postgresql://, mysql://)
- User specification
2. For SQLite:
- Use `sqlite3 database.db "QUERY" -json` for JSON output
- Use `.mode csv` with `.headers on` for CSV output
- Always check if the database file exists first
3. For PostgreSQL:
- Use `psql` with connection string or environment variables
- Use `COPY ... TO STDOUT WITH CSV HEADER` for CSV export
- Export JSON using `row_to_json()` function
4. For MySQL:
- Use `mysql` with `-e` flag for queries
- Use `--batch --silent` for CSV-like output
- Set connection timeout with `--connect-timeout=10`
5. Always:
- Validate database/table exists before querying
- Use readonly connections when only reading
- Handle errors gracefully with clear messages
- Sanitize outputs (escape commas in CSV, quote strings)
6. For large datasets:
- Add LIMIT clause to queries
- Use pagination with OFFSET for very large tables
- Warn user if result set is likely to be hugetext
你具备数据库查询与导出能力。当用户要求查询数据库时:
1. 从以下信息识别数据库类型(SQLite、PostgreSQL、MySQL):
- 文件扩展名(.db、.sqlite、.sqlite3 = SQLite)
- 连接字符串(postgresql://、mysql://)
- 用户指定
2. 对于SQLite:
- 使用 `sqlite3 database.db "QUERY" -json` 导出JSON格式
- 使用 `.mode csv` 搭配 `.headers on` 导出CSV格式
- 始终先检查数据库文件是否存在
3. 对于PostgreSQL:
- 使用 `psql` 搭配连接字符串或环境变量
- 使用 `COPY ... TO STDOUT WITH CSV HEADER` 导出CSV
- 使用 `row_to_json()` 函数导出JSON
4. 对于MySQL:
- 使用 `mysql` 搭配 `-e` 参数执行查询
- 使用 `--batch --silent` 导出类CSV格式
- 使用 `--connect-timeout=10` 设置连接超时
5. 始终遵循:
- 查询前验证数据库/表是否存在
- 仅读取时使用只读连接
- 优雅处理错误并给出清晰提示
- 清理输出内容(CSV中转义逗号,字符串添加引号)
6. 对于大型数据集:
- 在查询中添加LIMIT子句
- 对超大型表使用OFFSET进行分页
- 若结果集可能过大,提前告知用户Troubleshooting
故障排查
Error: "unable to open database file"
- Symptom: SQLite cannot find or access the database file
- Solution: Check file path is correct and file has read permissions
Error: "connection refused"
- Symptom: Cannot connect to PostgreSQL or MySQL server
- Solution: Verify host/port are correct, database service is running, and firewall allows connections
Error: "authentication failed"
- Symptom: Database rejects username/password
- Solution: Verify credentials are correct, user has necessary privileges
Error: "table does not exist"
- Symptom: Query references non-existent table
- Solution: List available tables first (or
sqlite3 db.db ".tables"in psql)\dt
CSV output has broken formatting:
- Symptom: Commas in data break CSV columns
- Solution: Properly escape values with commas using quotes, escape existing quotes
Query takes too long:
- Symptom: Query hangs or runs for minutes
- Solution: Add LIMIT clause, optimize query with indexes, increase timeout
错误:"unable to open database file"
- 症状:SQLite无法找到或访问数据库文件
- 解决方案:检查文件路径是否正确,以及文件是否拥有读取权限
错误:"connection refused"
- 症状:无法连接到PostgreSQL或MySQL服务器
- 解决方案:验证主机/端口是否正确,数据库服务是否运行,防火墙是否允许连接
错误:"authentication failed"
- 症状:数据库拒绝用户名/密码
- 解决方案:验证凭证是否正确,用户是否拥有必要权限
错误:"table does not exist"
- 症状:查询引用了不存在的表
- 解决方案:先列出可用表(或在psql中使用
sqlite3 db.db ".tables")\dt
CSV输出格式混乱:
- 症状:数据中的逗号破坏了CSV列结构
- 解决方案:使用引号正确转义包含逗号的值,转义已有的引号
查询耗时过长:
- 症状:查询挂起或运行数分钟
- 解决方案:添加LIMIT子句,使用索引优化查询,增加超时时间
See also
相关链接
- ../json-and-csv-data-transformation/SKILL.md — Transform exported data between formats
- ../file-tracker/SKILL.md — Track database file changes over time
- ../chat-logger/SKILL.md — Example of SQLite usage for logging
- ../json-and-csv-data-transformation/SKILL.md — 在不同格式间转换导出的数据
- ../file-tracker/SKILL.md — 跟踪数据库文件随时间的变化
- ../chat-logger/SKILL.md — SQLite用于日志记录的示例