configuring-log-export

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Configuring Log Export

配置日志导出

Configures log and metric export for CockroachDB Cloud clusters to deliver cluster logs, audit logs, and performance metrics to external monitoring services. Supports AWS CloudWatch, GCP Cloud Logging, and third-party integrations like Datadog.

配置CockroachDB Cloud集群的日志与指标导出功能，将集群日志、审计日志和性能指标发送至外部监控服务。支持AWS CloudWatch、GCP Cloud Logging以及Datadog等第三方集成。

When to Use This Skill

何时使用该技能

Setting up log export to AWS CloudWatch or GCP Cloud Logging
Configuring metric export to CloudWatch or Datadog
Verifying that SQL audit logs are being exported to an external sink
Troubleshooting log export errors or missing logs
Estimating CloudWatch or logging service costs for CockroachDB log volume
Configuring log export via Terraform

设置日志导出至AWS CloudWatch或GCP Cloud Logging
配置指标导出至CloudWatch或Datadog
验证SQL审计日志是否导出至外部接收器
排查日志导出错误或日志缺失问题
估算CockroachDB日志量对应的CloudWatch或日志服务成本
通过Terraform配置日志导出

Prerequisites

前置条件

CockroachDB Cloud Advanced plan — Log export is not available on Basic or Standard plans
ccloud CLI authenticated with Cluster Admin role
Cloud provider setup:
- AWS: CloudWatch Logs group created, IAM role granting CockroachDB Cloud cross-account access
- GCP: Cloud Logging API enabled, service account with Logs Writer role
Cluster ID: Available from
```
ccloud cluster list
```

Verify access:

bash

ccloud auth whoami
ccloud cluster info <cluster-name> -o json

CockroachDB Cloud高级版套餐 — 基础版或标准版不支持日志导出
ccloud CLI 已通过集群管理员角色认证
云提供商配置：
- AWS： 已创建CloudWatch Logs日志组，拥有授予CockroachDB Cloud跨账户访问权限的IAM角色
- GCP： 已启用Cloud Logging API，拥有Logs Writer角色的服务账户
集群ID： 可通过
```
ccloud cluster list
```
获取

验证访问权限：

bash

ccloud auth whoami
ccloud cluster info <cluster-name> -o json

Look for "plan": "ADVANCED"

查找 "plan": "ADVANCED"

undefined

undefined

Configuration Decisions

配置决策

Before proceeding, determine which export destinations apply to the user's environment. Ask which options are relevant, then follow only the corresponding sections below.

Decision 1 — Log export destination:

AWS CloudWatch: Use when the cluster runs on AWS and logs should go to CloudWatch Logs. Requires IAM cross-account role setup.
GCP Cloud Logging: Use when the cluster runs on GCP. Requires a service account with Logs Writer role.

Decision 2 — Metric export destination:

CloudWatch: Use when metrics should go to AWS CloudWatch. Requires IAM role with
```
cloudwatch:PutMetricData
```
permission.
Datadog: Use when metrics should go to Datadog. Requires a Datadog API key and site.
Skip: No metric export needed at this time.

开始操作前，请确定适用于用户环境的导出目标。询问用户相关选项，然后仅遵循对应部分的步骤。

决策1 — 日志导出目标：

AWS CloudWatch： 当集群运行在AWS上且日志需发送至CloudWatch Logs时使用。需要配置IAM跨账户角色。
GCP Cloud Logging： 当集群运行在GCP上时使用。需要拥有Logs Writer角色的服务账户。

决策2 — 指标导出目标：

CloudWatch： 当指标需发送至AWS CloudWatch时使用。需要拥有
```
cloudwatch:PutMetricData
```
权限的IAM角色。
Datadog： 当指标需发送至Datadog时使用。需要Datadog API密钥和站点信息。
跳过： 当前无需指标导出。

Steps

步骤

1. Check Current Log Export Configuration

1. 检查当前日志导出配置

bash

undefined

bash

undefined

Check if log export is currently configured

检查日志导出是否已配置

ccloud cluster info <cluster-name> -o json

Look for "log_export_config" in the output

在输出中查找 "log_export_config"

undefined

undefined

2. Set Up Log Export to AWS CloudWatch

2. 设置日志导出至AWS CloudWatch

Follow this section only if the user selected AWS CloudWatch in Decision 1. Skip to Step 3 if using GCP Cloud Logging.

仅当用户在决策1中选择AWS CloudWatch时遵循此部分。若使用GCP Cloud Logging，请跳至步骤3。

2.1 Create a CloudWatch Log Group

2.1 创建CloudWatch日志组

bash

undefined

bash

undefined

Create a log group in AWS (if it doesn't exist)

在AWS中创建日志组（若不存在）

aws logs create-log-group
--log-group-name cockroachdb-<cluster-name>
--region <aws-region>

Set retention policy (recommended)

设置保留策略（推荐）

aws logs put-retention-policy
--log-group-name cockroachdb-<cluster-name>
--retention-in-days 90
--region <aws-region>

undefined

aws logs put-retention-policy
--log-group-name cockroachdb-<cluster-name>
--retention-in-days 90
--region <aws-region>

undefined

2.2 Create an IAM Role for CockroachDB Cloud

2.2 为CockroachDB Cloud创建IAM角色

See cloud provider setup reference for the complete IAM role policy.

The IAM role must:

Trust the CockroachDB Cloud AWS account as an allowed principal

Grant

logs:CreateLogStream

logs:PutLogEvents

logs:DescribeLogGroups

logs:DescribeLogStreams

permissions

Be scoped to the specific log group

完整的IAM角色策略请参见云提供商配置参考。

IAM角色必须：

允许CockroachDB Cloud AWS账户作为可信主体

授予

logs:CreateLogStream

、

logs:PutLogEvents

、

logs:DescribeLogGroups

、

logs:DescribeLogStreams

权限

限定于特定日志组

2.3 Enable Log Export

2.3 启用日志导出

bash

undefined

bash

undefined

Enable log export to CloudWatch

启用日志导出至CloudWatch

ccloud cluster log-export create <cluster-id>
--log-group-name cockroachdb-<cluster-name>
--auth-principal <iam-role-arn>
--type AWS_CLOUDWATCH
--region <aws-region>

undefined

ccloud cluster log-export create <cluster-id>
--log-group-name cockroachdb-<cluster-name>
--auth-principal <iam-role-arn>
--type AWS_CLOUDWATCH
--region <aws-region>

undefined

3. Set Up Log Export to GCP Cloud Logging

3. 设置日志导出至GCP Cloud Logging

Follow this section only if the user selected GCP Cloud Logging in Decision 1. Skip if using AWS CloudWatch.

仅当用户在决策1中选择GCP Cloud Logging时遵循此部分。若使用AWS CloudWatch，请跳过。

3.1 Enable Cloud Logging API

3.1 启用Cloud Logging API

bash

gcloud services enable logging.googleapis.com

bash

gcloud services enable logging.googleapis.com

3.2 Grant CockroachDB Cloud Service Account Access

3.2 授予CockroachDB Cloud服务账户访问权限

bash

undefined

bash

undefined

Get the CockroachDB Cloud service account from ccloud cluster info

从ccloud cluster info中获取CockroachDB Cloud服务账户

Grant Logs Writer role

授予Logs Writer角色

gcloud projects add-iam-policy-binding <gcp-project-id>
--member="serviceAccount:<cockroachdb-service-account>"
--role="roles/logging.logWriter"

undefined

gcloud projects add-iam-policy-binding <gcp-project-id>
--member="serviceAccount:<cockroachdb-service-account>"
--role="roles/logging.logWriter"

undefined

3.3 Enable Log Export

3.3 启用日志导出

bash

ccloud cluster log-export create <cluster-id> \
  --auth-principal <gcp-project-id> \
  --type GCP_CLOUD_LOGGING

bash

ccloud cluster log-export create <cluster-id> \
  --auth-principal <gcp-project-id> \
  --type GCP_CLOUD_LOGGING

4. Configure Metric Export

4. 配置指标导出

Skip this section if the user selected Skip in Decision 2. Follow only the relevant subsection (4.1 or 4.2) based on the selected metric export destination.

Metric export sends CockroachDB performance metrics to CloudWatch or Datadog.

若用户在决策2中选择跳过，请跳过此部分。根据所选指标导出目标，仅遵循相关子部分（4.1或4.2）。

指标导出会将CockroachDB性能指标发送至CloudWatch或Datadog。

4.1 Metric Export to CloudWatch

4.1 指标导出至CloudWatch

bash

ccloud cluster metric-export create cloudwatch <cluster-id> \
  --role-arn <iam-role-arn> \
  --target-region <aws-region>

The IAM role for metric export needs

cloudwatch:PutMetricData

permission.

bash

ccloud cluster metric-export create cloudwatch <cluster-id> \
  --role-arn <iam-role-arn> \
  --target-region <aws-region>

指标导出的IAM角色需要

cloudwatch:PutMetricData

权限。

4.2 Metric Export to Datadog

4.2 指标导出至Datadog

bash

ccloud cluster metric-export create datadog <cluster-id> \
  --api-key <datadog-api-key> \
  --site <datadog-site>

Datadog site values:

datadoghq.com

(US),

datadoghq.eu

(EU),

us3.datadoghq.com

(US3),

us5.datadoghq.com

(US5)

bash

ccloud cluster metric-export create datadog <cluster-id> \
  --api-key <datadog-api-key> \
  --site <datadog-site>

Datadog站点取值：

datadoghq.com

（美国）、

datadoghq.eu

（欧盟）、

us3.datadoghq.com

（美国3区）、

us5.datadoghq.com

（美国5区）

5. Verify Log and Metric Export

5. 验证日志与指标导出

bash

undefined

bash

undefined

Check log export status

检查日志导出状态

ccloud cluster log-export list <cluster-id> -o json

Status should be ENABLED

状态应为ENABLED

Check metric export status

检查指标导出状态

ccloud cluster metric-export list <cluster-id> -o json


**Verify log delivery in CloudWatch:**
```bash

ccloud cluster metric-export list <cluster-id> -o json


**在CloudWatch中验证日志交付：**
```bash

Check for recent log streams

检查最近的日志流

aws logs describe-log-streams
--log-group-name cockroachdb-<cluster-name>
--order-by LastEventTime
--descending
--limit 5
--region <aws-region>

Tail recent log events

查看最近的日志事件

aws logs tail cockroachdb-<cluster-name>
--since 1h
--region <aws-region>


**Verify audit logs are being exported:**

Audit logs are included in the log export if SQL audit logging is enabled on the cluster. To confirm:

```sql
-- Check audit logging is enabled
SHOW CLUSTER SETTING sql.log.admin_audit.enabled;
SHOW CLUSTER SETTING sql.log.user_audit;

If audit logging is enabled but audit events are not appearing in CloudWatch, check:

Log export status is ENABLED
The IAM role has correct permissions
Log group name matches the configured export target
Allow 5-10 minutes for initial log delivery

aws logs tail cockroachdb-<cluster-name>
--since 1h
--region <aws-region>


**验证审计日志是否导出：**

如果集群已启用SQL审计日志，审计日志会包含在导出内容中。确认方法：

```sql
-- 检查审计日志是否启用
SHOW CLUSTER SETTING sql.log.admin_audit.enabled;
SHOW CLUSTER SETTING sql.log.user_audit;

若审计日志已启用但未出现在CloudWatch中，请检查：

日志导出状态为ENABLED
IAM角色拥有正确权限
日志组名称与配置的导出目标匹配
初始日志交付需要等待5-10分钟

6. Configure Log Export via Terraform

6. 通过Terraform配置日志导出

hcl

resource "cockroach_log_export_config" "main" {
  id         = cockroach_cluster.main.id
  auth_principal = "<iam-role-arn>"
  log_name       = "cockroachdb-${cockroach_cluster.main.name}"
  type           = "AWS_CLOUDWATCH"
  region         = "<aws-region>"
}

resource "cockroach_metric_export_cloudwatch_config" "main" {
  id         = cockroach_cluster.main.id
  role_arn       = "<iam-role-arn>"
  target_region  = "<aws-region>"
}

Known Terraform issue: Creating a cluster with log export and CMEK in the same

terraform apply

can cause a race condition. Apply the cluster first, then add log export and CMEK configurations in a subsequent apply.

hcl

resource "cockroach_log_export_config" "main" {
  id         = cockroach_cluster.main.id
  auth_principal = "<iam-role-arn>"
  log_name       = "cockroachdb-${cockroach_cluster.main.name}"
  type           = "AWS_CLOUDWATCH"
  region         = "<aws-region>"
}

resource "cockroach_metric_export_cloudwatch_config" "main" {
  id         = cockroach_cluster.main.id
  role_arn       = "<iam-role-arn>"
  target_region  = "<aws-region>"
}

已知Terraform问题： 在同一次

terraform apply

中创建集群并配置日志导出和CMEK会导致竞争条件。请先创建集群，再在后续的apply中添加日志导出和CMEK配置。

Safety Considerations

安全注意事项

Impact Type	Severity	Recommendation
Log export enabling	Low	No impact on cluster operation
Log export disabling	Low	Stops log delivery but does not affect cluster
IAM misconfiguration	Medium	Log export will fail silently; monitor for delivery gaps
Cost impact	Medium	High-volume clusters can generate significant CloudWatch/logging costs
Terraform race condition	Medium	Apply cluster creation before log/CMEK config

Cost planning:

CockroachDB Cloud can generate 1-10 GB of logs per day per node depending on query volume and audit settings
CloudWatch Logs pricing: ~$0.50/GB ingestion + $0.03/GB storage/month (varies by region)
Enable log retention policies to control storage costs
Audit logging significantly increases log volume — plan accordingly

Do not:

Delete the CloudWatch log group while log export is active (will cause delivery errors)
Revoke IAM permissions without disabling log export first
Enable cluster-wide SQL audit logging without considering the log volume increase

影响类型	严重程度	建议
启用日志导出	低	对集群运行无影响
禁用日志导出	低	停止日志交付但不影响集群
IAM配置错误	中	日志导出会静默失败；监控交付间隙
成本影响	中	高负载集群会产生显著的CloudWatch/日志服务成本
Terraform竞争条件	中	先执行集群创建，再配置日志/CMEK

成本规划：

根据查询量和审计设置，CockroachDB Cloud每个节点每天可生成1-10GB日志
CloudWatch Logs定价：约$0.50/GB ingestion + $0.03/GB存储/月（因区域而异）
启用日志保留策略以控制存储成本
审计日志会显著增加日志量 — 请提前规划

请勿：

在日志导出处于活跃状态时删除CloudWatch日志组（会导致交付错误）
未先禁用日志导出就撤销IAM权限
未考虑日志量增加就启用集群级SQL审计日志

Rollback

回滚

bash

undefined

bash

undefined

Disable log export

禁用日志导出

ccloud cluster log-export delete <cluster-id>

Disable metric export

禁用指标导出

ccloud cluster metric-export delete cloudwatch <cluster-id> ccloud cluster metric-export delete datadog <cluster-id>


Log export can be re-enabled at any time with the same or different configuration. Historical logs are not re-sent — only new logs are exported after re-enabling.

ccloud cluster metric-export delete cloudwatch <cluster-id> ccloud cluster metric-export delete datadog <cluster-id>


日志导出可随时重新启用，使用相同或不同配置。重新启用后仅会导出新日志，不会补发历史日志。

References

参考资料

Skill references:

Cloud provider IAM setup

Related skills:

configuring-audit-logging — Enable SQL audit logging (must be enabled for audit logs to appear in export)
auditing-cloud-cluster-security — Run a full security posture audit

Official CockroachDB Documentation:

技能参考：

云提供商IAM配置

相关技能：

configuring-audit-logging — 启用SQL审计日志（必须启用才能在导出中包含审计日志）
auditing-cloud-cluster-security — 运行完整的安全态势审计

官方CockroachDB文档：