database-query-and-export

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Database Query and Export

数据库查询与导出

Query relational databases (SQLite, PostgreSQL, MySQL) and export results to CSV, JSON, or other formats. Essential for data extraction, reporting, backup automation, and analytics pipelines.
查询关系型数据库(SQLite、PostgreSQL、MySQL)并将结果导出为CSV、JSON或其他格式。这是数据提取、报告生成、备份自动化和分析流程中的必备操作。

When to use

适用场景

  • Use case 1: When the user asks to query a database and export results
  • Use case 2: When you need to extract data for analysis or reporting
  • Use case 3: For backup and data migration workflows
  • Use case 4: When building automated database monitoring and alerts
  • 场景1:用户要求查询数据库并导出结果时
  • 场景2:需要提取数据用于分析或报告时
  • 场景3:用于备份和数据迁移流程
  • 场景4:构建自动化数据库监控与告警时

Required tools / APIs

所需工具/API

  • SQLite — Lightweight file-based database (often pre-installed)
  • PostgreSQL client — For PostgreSQL databases
  • MySQL client — For MySQL/MariaDB databases
  • No external API required
Install options:
bash
undefined
  • SQLite — 轻量级文件型数据库(通常已预装)
  • PostgreSQL client — 用于PostgreSQL数据库
  • MySQL client — 用于MySQL/MariaDB数据库
  • 无需外部API
安装选项:
bash
undefined

Ubuntu/Debian

Ubuntu/Debian

sudo apt-get install -y sqlite3 postgresql-client mysql-client
sudo apt-get install -y sqlite3 postgresql-client mysql-client

macOS

macOS

brew install sqlite3 postgresql mysql-client
brew install sqlite3 postgresql mysql-client

Node.js (database drivers)

Node.js (数据库驱动)

npm install better-sqlite3 # SQLite npm install pg # PostgreSQL npm install mysql2 # MySQL
undefined
npm install better-sqlite3 # SQLite npm install pg # PostgreSQL npm install mysql2 # MySQL
undefined

Skills

技能

query_sqlite_to_json

query_sqlite_to_json

Query SQLite database and export to JSON format.
bash
undefined
查询SQLite数据库并导出为JSON格式。
bash
undefined

Basic query to JSON

基础查询导出为JSON

sqlite3 database.db "SELECT * FROM users LIMIT 10;" -json
sqlite3 database.db "SELECT * FROM users LIMIT 10;" -json

With pretty formatting using jq

使用jq进行格式化输出

sqlite3 database.db "SELECT * FROM users WHERE active=1;" -json | jq '.'
sqlite3 database.db "SELECT * FROM users WHERE active=1;" -json | jq '.'

Export entire table to JSON file

将整张表导出为JSON文件

sqlite3 database.db "SELECT * FROM orders;" -json > orders.json
sqlite3 database.db "SELECT * FROM orders;" -json > orders.json

Query with WHERE clause

带WHERE子句的查询

sqlite3 database.db "SELECT id, name, email FROM users WHERE created_at > '2024-01-01';" -json

**Node.js:**

```javascript
const Database = require('better-sqlite3');

function querySQLiteToJSON(dbPath, query) {
  const db = new Database(dbPath, { readonly: true });
  const rows = db.prepare(query).all();
  db.close();
  return rows;
}

// Usage
// const users = querySQLiteToJSON('./database.db', 'SELECT * FROM users LIMIT 10');
// console.log(JSON.stringify(users, null, 2));
sqlite3 database.db "SELECT id, name, email FROM users WHERE created_at > '2024-01-01';" -json

**Node.js:**

```javascript
const Database = require('better-sqlite3');

function querySQLiteToJSON(dbPath, query) {
  const db = new Database(dbPath, { readonly: true });
  const rows = db.prepare(query).all();
  db.close();
  return rows;
}

// 使用示例
// const users = querySQLiteToJSON('./database.db', 'SELECT * FROM users LIMIT 10');
// console.log(JSON.stringify(users, null, 2));

query_sqlite_to_csv

query_sqlite_to_csv

Query SQLite database and export to CSV format.
bash
undefined
查询SQLite数据库并导出为CSV格式。
bash
undefined

Basic query to CSV

基础查询导出为CSV

sqlite3 database.db <<EOF .mode csv .headers on SELECT * FROM users LIMIT 10; EOF
sqlite3 database.db <<EOF .mode csv .headers on SELECT * FROM users LIMIT 10; EOF

Export to CSV file

导出为CSV文件

sqlite3 database.db <<EOF .mode csv .headers on .output users.csv SELECT id, name, email, created_at FROM users WHERE active=1; EOF
sqlite3 database.db <<EOF .mode csv .headers on .output users.csv SELECT id, name, email, created_at FROM users WHERE active=1; EOF

Query multiple tables with JOIN

多表JOIN查询

sqlite3 database.db <<EOF .mode csv .headers on SELECT u.name, o.order_id, o.total FROM users u JOIN orders o ON u.id = o.user_id WHERE o.created_at > '2024-01-01'; EOF

**Node.js:**

```javascript
const Database = require('better-sqlite3');
const fs = require('fs');

function querySQLiteToCSV(dbPath, query, outputPath) {
  const db = new Database(dbPath, { readonly: true });
  const rows = db.prepare(query).all();
  db.close();
  
  if (rows.length === 0) {
    return 'No results';
  }
  
  // Generate CSV
  const headers = Object.keys(rows[0]).join(',');
  const csvRows = rows.map(row => 
    Object.values(row).map(val => 
      typeof val === 'string' && val.includes(',') ? `"${val}"` : val
    ).join(',')
  );
  
  const csv = [headers, ...csvRows].join('\n');
  
  if (outputPath) {
    fs.writeFileSync(outputPath, csv);
    return `Exported ${rows.length} rows to ${outputPath}`;
  }
  
  return csv;
}

// Usage
// querySQLiteToCSV('./database.db', 'SELECT * FROM users LIMIT 10', './users.csv');
sqlite3 database.db <<EOF .mode csv .headers on SELECT u.name, o.order_id, o.total FROM users u JOIN orders o ON u.id = o.user_id WHERE o.created_at > '2024-01-01'; EOF

**Node.js:**

```javascript
const Database = require('better-sqlite3');
const fs = require('fs');

function querySQLiteToCSV(dbPath, query, outputPath) {
  const db = new Database(dbPath, { readonly: true });
  const rows = db.prepare(query).all();
  db.close();
  
  if (rows.length === 0) {
    return '无查询结果';
  }
  
  // 生成CSV
  const headers = Object.keys(rows[0]).join(',');
  const csvRows = rows.map(row => 
    Object.values(row).map(val => 
      typeof val === 'string' && val.includes(',') ? `"${val}"` : val
    ).join(',')
  );
  
  const csv = [headers, ...csvRows].join('\n');
  
  if (outputPath) {
    fs.writeFileSync(outputPath, csv);
    return `已将${rows.length}条数据导出至${outputPath}`;
  }
  
  return csv;
}

// 使用示例
// querySQLiteToCSV('./database.db', 'SELECT * FROM users LIMIT 10', './users.csv');

query_postgresql

query_postgresql

Query PostgreSQL database and export results.
bash
undefined
查询PostgreSQL数据库并导出结果。
bash
undefined

Set connection string (alternative: use individual flags)

设置连接字符串(替代方案:使用单独的参数)

export PGHOST=localhost export PGPORT=5432 export PGDATABASE=mydb export PGUSER=postgres export PGPASSWORD=mypassword
export PGHOST=localhost export PGPORT=5432 export PGDATABASE=mydb export PGUSER=postgres export PGPASSWORD=mypassword

Query to JSON (using psql with formatted output)

查询导出为JSON(使用psql格式化输出)

psql -t -A -F"," -c "SELECT row_to_json(t) FROM (SELECT * FROM users LIMIT 10) t;"
psql -t -A -F"," -c "SELECT row_to_json(t) FROM (SELECT * FROM users LIMIT 10) t;"

Query to CSV

查询导出为CSV

psql -c "COPY (SELECT * FROM users WHERE active=true) TO STDOUT WITH CSV HEADER;" > users.csv
psql -c "COPY (SELECT * FROM users WHERE active=true) TO STDOUT WITH CSV HEADER;" > users.csv

Query with connection string

使用连接字符串查询

psql "postgresql://user:password@localhost:5432/mydb" -c "SELECT * FROM users LIMIT 5;"
psql "postgresql://user:password@localhost:5432/mydb" -c "SELECT * FROM users LIMIT 5;"

Query to formatted table

查询导出为格式化表格

psql -c "SELECT id, name, email FROM users ORDER BY created_at DESC LIMIT 10;"

**Node.js:**

```javascript
const { Pool } = require('pg');

async function queryPostgreSQL(connectionString, query) {
  const pool = new Pool({ connectionString });
  
  try {
    const result = await pool.query(query);
    return result.rows;
  } finally {
    await pool.end();
  }
}

// Usage
// const connStr = 'postgresql://user:password@localhost:5432/mydb';
// queryPostgreSQL(connStr, 'SELECT * FROM users LIMIT 10')
//   .then(rows => console.log(JSON.stringify(rows, null, 2)));
psql -c "SELECT id, name, email FROM users ORDER BY created_at DESC LIMIT 10;"

**Node.js:**

```javascript
const { Pool } = require('pg');

async function queryPostgreSQL(connectionString, query) {
  const pool = new Pool({ connectionString });
  
  try {
    const result = await pool.query(query);
    return result.rows;
  } finally {
    await pool.end();
  }
}

// 使用示例
// const connStr = 'postgresql://user:password@localhost:5432/mydb';
// queryPostgreSQL(connStr, 'SELECT * FROM users LIMIT 10')
//   .then(rows => console.log(JSON.stringify(rows, null, 2)));

query_mysql

query_mysql

Query MySQL/MariaDB database and export results.
bash
undefined
查询MySQL/MariaDB数据库并导出结果。
bash
undefined

Query to CSV with headers

查询导出为带表头的CSV

mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users WHERE active=1;"
--batch --silent
| cat > users.csv
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users WHERE active=1;"
--batch --silent
| cat > users.csv

Query to JSON-like format (requires jq for proper formatting)

查询导出为类JSON格式(需要jq进行正确格式化)

mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users LIMIT 10;"
--batch --silent
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM users LIMIT 10;"
--batch --silent

Export entire table to CSV

将整张表导出为CSV

mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM orders INTO OUTFILE '/tmp/orders.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';"
mysql -h localhost -u root -p'mypassword' -D mydb
-e "SELECT * FROM orders INTO OUTFILE '/tmp/orders.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';"

Query with timeout

带超时设置的查询

mysql -h localhost -u root -p'mypassword' -D mydb
--connect-timeout=10
-e "SELECT COUNT(*) as total FROM users;"

**Node.js:**

```javascript
const mysql = require('mysql2/promise');

async function queryMySQL(config, query) {
  const connection = await mysql.createConnection({
    host: config.host || 'localhost',
    user: config.user,
    password: config.password,
    database: config.database,
    connectTimeout: 10000
  });
  
  try {
    const [rows] = await connection.execute(query);
    return rows;
  } finally {
    await connection.end();
  }
}

// Usage
// const config = {
//   host: 'localhost',
//   user: 'root',
//   password: 'mypassword',
//   database: 'mydb'
// };
// queryMySQL(config, 'SELECT * FROM users LIMIT 10')
//   .then(rows => console.log(JSON.stringify(rows, null, 2)));
mysql -h localhost -u root -p'mypassword' -D mydb
--connect-timeout=10
-e "SELECT COUNT(*) as total FROM users;"

**Node.js:**

```javascript
const mysql = require('mysql2/promise');

async function queryMySQL(config, query) {
  const connection = await mysql.createConnection({
    host: config.host || 'localhost',
    user: config.user,
    password: config.password,
    database: config.database,
    connectTimeout: 10000
  });
  
  try {
    const [rows] = await connection.execute(query);
    return rows;
  } finally {
    await connection.end();
  }
}

// 使用示例
// const config = {
//   host: 'localhost',
//   user: 'root',
//   password: 'mypassword',
//   database: 'mydb'
// };
// queryMySQL(config, 'SELECT * FROM users LIMIT 10')
//   .then(rows => console.log(JSON.stringify(rows, null, 2)));

advanced_sqlite_export_with_error_handling

advanced_sqlite_export_with_error_handling

Production-ready SQLite export with validation and error handling.
bash
#!/bin/bash
DB_PATH="database.db"
QUERY="SELECT * FROM users WHERE active=1;"
OUTPUT_FILE="users.csv"
带验证和错误处理的生产级SQLite导出功能。
bash
#!/bin/bash
DB_PATH="database.db"
QUERY="SELECT * FROM users WHERE active=1;"
OUTPUT_FILE="users.csv"

Check if database exists

检查数据库是否存在

if [ ! -f "$DB_PATH" ]; then echo "Error: Database file not found: $DB_PATH" >&2 exit 1 fi
if [ ! -f "$DB_PATH" ]; then echo "错误:未找到数据库文件:$DB_PATH" >&2 exit 1 fi

Check if table exists

检查表是否存在

if ! sqlite3 "$DB_PATH" "SELECT name FROM sqlite_master WHERE type='table' AND name='users';" | grep -q "users"; then echo "Error: Table 'users' not found in database" >&2 exit 1 fi
if ! sqlite3 "$DB_PATH" "SELECT name FROM sqlite_master WHERE type='table' AND name='users';" | grep -q "users"; then echo "错误:数据库中不存在'users'表" >&2 exit 1 fi

Execute query and export to CSV

执行查询并导出为CSV

if sqlite3 "$DB_PATH" <<EOF > "$OUTPUT_FILE" 2>&1 .mode csv .headers on $QUERY EOF then ROW_COUNT=$(wc -l < "$OUTPUT_FILE") echo "Success: Exported $((ROW_COUNT - 1)) rows to $OUTPUT_FILE" else echo "Error: Query failed" >&2 exit 1 fi

**Node.js:**

```javascript
const Database = require('better-sqlite3');
const fs = require('fs');

async function exportSQLiteWithValidation(options) {
  const { dbPath, query, outputPath, format = 'json' } = options;
  
  // Validate database exists
  if (!fs.existsSync(dbPath)) {
    throw new Error(`Database file not found: ${dbPath}`);
  }
  
  let db;
  try {
    db = new Database(dbPath, { readonly: true, timeout: 10000 });
    
    // Prepare and execute query
    const stmt = db.prepare(query);
    const rows = stmt.all();
    
    if (rows.length === 0) {
      return { success: true, rowCount: 0, message: 'No rows returned' };
    }
    
    // Export based on format
    let output;
    if (format === 'json') {
      output = JSON.stringify(rows, null, 2);
    } else if (format === 'csv') {
      const headers = Object.keys(rows[0]).join(',');
      const csvRows = rows.map(row => 
        Object.values(row).map(val => 
          typeof val === 'string' && val.includes(',') ? `"${val.replace(/"/g, '""')}"` : val
        ).join(',')
      );
      output = [headers, ...csvRows].join('\n');
    } else {
      throw new Error(`Unsupported format: ${format}`);
    }
    
    // Write to file
    fs.writeFileSync(outputPath, output);
    
    return {
      success: true,
      rowCount: rows.length,
      outputPath,
      format,
      message: `Exported ${rows.length} rows to ${outputPath}`
    };
    
  } catch (err) {
    throw new Error(`Database export failed: ${err.message}`);
  } finally {
    if (db) db.close();
  }
}

// Usage
// exportSQLiteWithValidation({
//   dbPath: './database.db',
//   query: 'SELECT * FROM users WHERE active=1',
//   outputPath: './users.json',
//   format: 'json'
// }).then(result => console.log(result));
if sqlite3 "$DB_PATH" <<EOF > "$OUTPUT_FILE" 2>&1 .mode csv .headers on $QUERY EOF then ROW_COUNT=$(wc -l < "$OUTPUT_FILE") echo "成功:已将$((ROW_COUNT - 1))条数据导出至$OUTPUT_FILE" else echo "错误:查询失败" >&2 exit 1 fi

**Node.js:**

```javascript
const Database = require('better-sqlite3');
const fs = require('fs');

async function exportSQLiteWithValidation(options) {
  const { dbPath, query, outputPath, format = 'json' } = options;
  
  // 验证数据库是否存在
  if (!fs.existsSync(dbPath)) {
    throw new Error(`未找到数据库文件:${dbPath}`);
  }
  
  let db;
  try {
    db = new Database(dbPath, { readonly: true, timeout: 10000 });
    
    // 准备并执行查询
    const stmt = db.prepare(query);
    const rows = stmt.all();
    
    if (rows.length === 0) {
      return { success: true, rowCount: 0, message: '无数据返回' };
    }
    
    // 根据格式导出
    let output;
    if (format === 'json') {
      output = JSON.stringify(rows, null, 2);
    } else if (format === 'csv') {
      const headers = Object.keys(rows[0]).join(',');
      const csvRows = rows.map(row => 
        Object.values(row).map(val => 
          typeof val === 'string' && val.includes(',') ? `"${val.replace(/"/g, '""')}"` : val
        ).join(',')
      );
      output = [headers, ...csvRows].join('\n');
    } else {
      throw new Error(`不支持的格式:${format}`);
    }
    
    // 写入文件
    fs.writeFileSync(outputPath, output);
    
    return {
      success: true,
      rowCount: rows.length,
      outputPath,
      format,
      message: `已将${rows.length}条数据导出至${outputPath}`
    };
    
  } catch (err) {
    throw new Error(`数据库导出失败:${err.message}`);
  } finally {
    if (db) db.close();
  }
}

// 使用示例
// exportSQLiteWithValidation({
//   dbPath: './database.db',
//   query: 'SELECT * FROM users WHERE active=1',
//   outputPath: './users.json',
//   format: 'json'
// }).then(result => console.log(result));

Rate limits / Best practices

速率限制/最佳实践

  • Use readonly connections — Open databases in readonly mode when only querying
  • Set connection timeouts — Use 10-30 second timeouts to prevent hanging
  • Validate inputs — Check that database files/tables exist before querying
  • Escape user inputs — Use parameterized queries to prevent SQL injection
  • Handle large datasets — Use LIMIT/OFFSET for pagination on large tables
  • Close connections — Always close database connections after queries
  • ⚠️ Secure credentials — Store database passwords in environment variables, never hardcode
  • ⚠️ Export file permissions — Ensure export directories have proper write permissions
  • 使用只读连接 — 仅查询时以只读模式打开数据库
  • 设置连接超时 — 使用10-30秒超时时间避免进程挂起
  • 验证输入 — 查询前检查数据库文件/表是否存在
  • 转义用户输入 — 使用参数化查询防止SQL注入
  • 处理大型数据集 — 对大表使用LIMIT/OFFSET进行分页
  • 关闭连接 — 查询后始终关闭数据库连接
  • ⚠️ 安全存储凭证 — 将数据库密码存储在环境变量中,切勿硬编码
  • ⚠️ 设置导出文件权限 — 确保导出目录拥有正确的写入权限

Agent prompt

Agent提示词

text
You have database query and export capability. When a user asks to query a database:

1. Identify the database type (SQLite, PostgreSQL, MySQL) from:
   - File extension (.db, .sqlite, .sqlite3 = SQLite)
   - Connection string (postgresql://, mysql://)
   - User specification

2. For SQLite:
   - Use `sqlite3 database.db "QUERY" -json` for JSON output
   - Use `.mode csv` with `.headers on` for CSV output
   - Always check if the database file exists first

3. For PostgreSQL:
   - Use `psql` with connection string or environment variables
   - Use `COPY ... TO STDOUT WITH CSV HEADER` for CSV export
   - Export JSON using `row_to_json()` function

4. For MySQL:
   - Use `mysql` with `-e` flag for queries
   - Use `--batch --silent` for CSV-like output
   - Set connection timeout with `--connect-timeout=10`

5. Always:
   - Validate database/table exists before querying
   - Use readonly connections when only reading
   - Handle errors gracefully with clear messages
   - Sanitize outputs (escape commas in CSV, quote strings)

6. For large datasets:
   - Add LIMIT clause to queries
   - Use pagination with OFFSET for very large tables
   - Warn user if result set is likely to be huge
text
你具备数据库查询与导出能力。当用户要求查询数据库时:

1. 从以下信息识别数据库类型(SQLite、PostgreSQL、MySQL):
   - 文件扩展名(.db、.sqlite、.sqlite3 = SQLite)
   - 连接字符串(postgresql://、mysql://)
   - 用户指定

2. 对于SQLite:
   - 使用 `sqlite3 database.db "QUERY" -json` 导出JSON格式
   - 使用 `.mode csv` 搭配 `.headers on` 导出CSV格式
   - 始终先检查数据库文件是否存在

3. 对于PostgreSQL:
   - 使用 `psql` 搭配连接字符串或环境变量
   - 使用 `COPY ... TO STDOUT WITH CSV HEADER` 导出CSV
   - 使用 `row_to_json()` 函数导出JSON

4. 对于MySQL:
   - 使用 `mysql` 搭配 `-e` 参数执行查询
   - 使用 `--batch --silent` 导出类CSV格式
   - 使用 `--connect-timeout=10` 设置连接超时

5. 始终遵循:
   - 查询前验证数据库/表是否存在
   - 仅读取时使用只读连接
   - 优雅处理错误并给出清晰提示
   - 清理输出内容(CSV中转义逗号,字符串添加引号)

6. 对于大型数据集:
   - 在查询中添加LIMIT子句
   - 对超大型表使用OFFSET进行分页
   - 若结果集可能过大,提前告知用户

Troubleshooting

故障排查

Error: "unable to open database file"
  • Symptom: SQLite cannot find or access the database file
  • Solution: Check file path is correct and file has read permissions
Error: "connection refused"
  • Symptom: Cannot connect to PostgreSQL or MySQL server
  • Solution: Verify host/port are correct, database service is running, and firewall allows connections
Error: "authentication failed"
  • Symptom: Database rejects username/password
  • Solution: Verify credentials are correct, user has necessary privileges
Error: "table does not exist"
  • Symptom: Query references non-existent table
  • Solution: List available tables first (
    sqlite3 db.db ".tables"
    or
    \dt
    in psql)
CSV output has broken formatting:
  • Symptom: Commas in data break CSV columns
  • Solution: Properly escape values with commas using quotes, escape existing quotes
Query takes too long:
  • Symptom: Query hangs or runs for minutes
  • Solution: Add LIMIT clause, optimize query with indexes, increase timeout
错误:"unable to open database file"
  • 症状:SQLite无法找到或访问数据库文件
  • 解决方案:检查文件路径是否正确,以及文件是否拥有读取权限
错误:"connection refused"
  • 症状:无法连接到PostgreSQL或MySQL服务器
  • 解决方案:验证主机/端口是否正确,数据库服务是否运行,防火墙是否允许连接
错误:"authentication failed"
  • 症状:数据库拒绝用户名/密码
  • 解决方案:验证凭证是否正确,用户是否拥有必要权限
错误:"table does not exist"
  • 症状:查询引用了不存在的表
  • 解决方案:先列出可用表(
    sqlite3 db.db ".tables"
    或在psql中使用
    \dt
CSV输出格式混乱:
  • 症状:数据中的逗号破坏了CSV列结构
  • 解决方案:使用引号正确转义包含逗号的值,转义已有的引号
查询耗时过长:
  • 症状:查询挂起或运行数分钟
  • 解决方案:添加LIMIT子句,使用索引优化查询,增加超时时间

See also

相关链接

  • ../json-and-csv-data-transformation/SKILL.md — Transform exported data between formats
  • ../file-tracker/SKILL.md — Track database file changes over time
  • ../chat-logger/SKILL.md — Example of SQLite usage for logging
  • ../json-and-csv-data-transformation/SKILL.md — 在不同格式间转换导出的数据
  • ../file-tracker/SKILL.md — 跟踪数据库文件随时间的变化
  • ../chat-logger/SKILL.md — SQLite用于日志记录的示例