querying-aws-redshift

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Query AWS Redshift System Tables

查询AWS Redshift系统表

Overview

概述

Works best with the AWS MCP server for sandboxed execution and audit logging. All commands below use the AWS CLI and work in any environment with configured AWS credentials. Use IAM roles or temporary credentials; avoid long-lived access keys.
Redshift can publish system tables — the
SYS_*
monitoring data such as
sys_query_history
,
sys_query_detail
, and
sys_connection_log
— to S3 Tables as continuously-updated Apache Iceberg tables.
Terminology used throughout: system table refers to a
SYS_*
dataset generally, and each one maps 1:1 to a published Iceberg table. Where this skill says
SYS_
view
, it means specifically the live in-cluster object you query on the cluster itself — that is a view, and it is a different thing from the published S3 Tables copy. This applies to both Provisioned clusters and Serverless namespaces. It is an opt-in extension of the existing logging APIs. Published tables are read-only, stored in the AWS-managed
aws-redshift
table bucket, and queryable via any Iceberg-compatible engine including Amazon Athena and Amazon Redshift itself.
Querying the S3 Tables copy is preferred over the live in-cluster
SYS_
views when analyzing historical or high-volume system-table data because:
  • The in-cluster
    SYS_
    views have a limited retention window; S3 Tables retains history well beyond it.
  • Querying S3 Tables adds no load to the running Redshift cluster.
  • The logs are Iceberg tables, so they can be queried at scale from any Iceberg-compatible engine and joined with other lake data.
此功能与AWS MCP服务器配合使用效果最佳,可实现沙箱执行和审计日志记录。以下所有命令均使用AWS CLI,可在任何配置了AWS凭证的环境中运行。建议使用IAM角色或临时凭证,避免使用长期访问密钥。
Redshift可将系统表(即
SYS_*
监控数据,如
sys_query_history
sys_query_detail
sys_connection_log
)以持续更新的Apache Iceberg表形式发布到S3 Tables
全文使用的术语说明:系统表泛指
SYS_*
数据集,每个系统表与一个已发布的Iceberg表一一对应。当本技能提及**
SYS_
视图时,特指你在集群上直接查询的集群内实时对象——它是一个视图,与已发布的S3 Tables副本并非同一对象。此功能适用于预置集群无服务器命名空间**,是现有日志API的可选扩展。已发布的表为只读状态,存储在AWS托管的
aws-redshift
表存储桶中,可通过Amazon Athena、Amazon Redshift等任何兼容Iceberg的引擎进行查询。
在分析历史或高容量系统表数据时,查询S3 Tables副本比查询集群内实时
SYS_
视图更具优势,原因如下:
  • 集群内
    SYS_
    视图的保留窗口有限;而S3 Tables的历史数据保留时间远超出该窗口。
  • 查询S3 Tables不会对运行中的Redshift集群造成任何负载。
  • 日志以Iceberg表形式存储,可通过任何兼容Iceberg的引擎进行大规模查询,还能与其他湖仓数据关联。

Decision Tree

决策树

User intentUse this skill?Alternative
Turn on S3 Tables log publishing for a cluster or namespaceYes
Confirm a cluster/namespace is publishing / find its S3 Tables namespaceYes
Querying non-realtime data from Redshift system tablesYes
Build daily/weekly/monthly dashboard for Redshift monitoring and auditingYes
Selectively stop S3 Tables publishingYes
Query published system tables from Redshift (cross-database)Yes
Query published system tables from AthenaYes
Inspect the current, real-time
SYS_
state on a live cluster
NoQuery the
SYS_
view on the cluster directly
Query data inside customer tablesNoDirect Redshift SQL on the cluster
用户意图是否使用本技能替代方案
为集群或命名空间开启S3 Tables日志发布
确认集群/命名空间是否正在发布日志 / 查找其S3 Tables命名空间
查询Redshift系统表的非实时数据
构建Redshift监控和审计的每日/每周/每月仪表板
选择性停止S3 Tables日志发布
从Redshift(跨数据库)查询已发布的系统表
从Athena查询已发布的系统表
检查集群上当前实时的
SYS_
状态
直接在集群上查询
SYS_
视图
查询用户表内的数据在集群上直接执行Redshift SQL

Supported Data Sources

支持的数据源

Compute typeEnable / disable APIStatus APIGranularity options
Redshift Provisioned cluster
redshift enable-logging
/
redshift disable-logging
redshift describe-logging-status
cluster
(default),
account
Redshift Serverless namespace
redshift-serverless update-namespace
with
--s3-table-action Enable
/
Disable
redshift-serverless get-namespace
namespace
(default),
account
Both compute types publish into the same AWS-managed
aws-redshift
table bucket and are queried identically once published. They differ only in the enable/disable API surface and in the casing of the status response — see the flag and field tables in Common Tasks.
Not covered by this skill: Redshift audit logs delivered to S3 or CloudWatch (
useractivitylog
,
userlog
,
connectionlog
), which use the separate
--log-exports
mechanism on Serverless and are not
SYS_*
system tables.
计算类型启用/禁用API状态API粒度选项
Redshift预置集群
redshift enable-logging
/
redshift disable-logging
redshift describe-logging-status
cluster
(默认)、
account
Redshift无服务器命名空间
--s3-table-action Enable
/
Disable
参数的
redshift-serverless update-namespace
redshift-serverless get-namespace
namespace
(默认)、
account
两种计算类型都会将数据发布到同一个AWS托管的
aws-redshift
表存储桶,发布完成后的查询方式完全相同。二者仅在启用/禁用API接口以及状态响应的字段格式上存在差异——详情请参见常见任务中的标志和字段表。
本技能不涵盖以下内容:交付到S3或CloudWatch的Redshift审计日志(
useractivitylog
userlog
connectionlog
),这些日志使用无服务器环境中独立的
--log-exports
机制,不属于
SYS_*
系统表范畴。

Common Tasks

常见任务

1. Check If Configured

1. 检查配置状态

Before querying, confirm the cluster or namespace is publishing to S3 Tables.
bash
undefined
查询前,请确认集群或命名空间是否已配置为向S3 Tables发布日志。
bash
undefined

Provisioned

预置集群

aws redshift describe-logging-status --region <REGION> --cluster-identifier <CLUSTER_ID>
aws redshift describe-logging-status --region <REGION> --cluster-identifier <CLUSTER_ID>

Serverless

无服务器命名空间

aws redshift-serverless get-namespace --region <REGION> --namespace-name <NAMESPACE_NAME>

**Interpret the response.** The two compute types return the *same* information under **different field names and casing** — Provisioned uses PascalCase under `S3Tables`, Serverless uses camelCase under `namespace.s3TablePublishStatus`:

| Meaning | Provisioned (`describe-logging-status`) | Serverless (`get-namespace`) |
|---|---|---|
| Not enabled | `LoggingEnabled: false` or no `S3Tables` block | no `s3TablePublishStatus` block |
| Destination includes S3 Tables | `LogDestinationType` contains `s3table` | `logDestinationType` contains `s3table` |
| List of published `SYS_*` tables | `S3Tables.S3Tables` | `namespace.s3TablePublishStatus.s3Tables` |
| **The exact S3 Tables namespace** (required for querying) | `S3Tables.S3TableNamespace` | `namespace.s3TablePublishStatus.s3TableNamespace` |
| Granularity | `S3Tables.S3TableGranularity` (`cluster`/`account`) | `namespace.s3TablePublishStatus.s3TableGranularity` (`namespace`/`account`) |
| Per-table last ingest time | `S3Tables.LastIngestionTimes` | `namespace.s3TablePublishStatus.lastIngestionTimes` |
| All available system tables published | `S3Tables.EnabledAll` | `namespace.s3TablePublishStatus.enabledAll` |

Notes:

- `LogDestinationType` is a **comma-joined list** when more than one destination is active — e.g. `"cloudwatch,s3table"`. Test with a substring/contains check, not equality against `s3table`.
- An empty `LastIngestionTimes` / `lastIngestionTimes` map, or a table listed as published but absent from the map, means data for that table may still be in flight. Compare successive values to confirm new data is landing.
- On Serverless, do **not** read the top-level `logExports` field for this feature — that field carries the CloudWatch/S3 audit logs (`useractivitylog`, `userlog`, `connectionlog`) and is unrelated to `SYS_*` S3 Tables publishing.
aws redshift-serverless get-namespace --region <REGION> --namespace-name <NAMESPACE_NAME>

**解读响应结果**。两种计算类型返回的是**相同信息**,但字段名称和格式不同——预置集群在`S3Tables`下使用 PascalCase,无服务器命名空间在`namespace.s3TablePublishStatus`下使用 camelCase:

| 含义 | 预置集群(`describe-logging-status`) | 无服务器命名空间(`get-namespace`) |
|---|---|---|
| 未启用 | `LoggingEnabled: false` 或无`S3Tables`块 | 无`s3TablePublishStatus`块 |
| 目标包含S3 Tables | `LogDestinationType`包含`s3table` | `logDestinationType`包含`s3table` |
| 已发布的`SYS_*`表列表 | `S3Tables.S3Tables` | `namespace.s3TablePublishStatus.s3Tables` |
| **准确的S3 Tables命名空间**(查询必需) | `S3Tables.S3TableNamespace` | `namespace.s3TablePublishStatus.s3TableNamespace` |
| 粒度 | `S3Tables.S3TableGranularity`(`cluster`/`account`) | `namespace.s3TablePublishStatus.s3TableGranularity`(`namespace`/`account`) |
| 各表的最后摄取时间 | `S3Tables.LastIngestionTimes` | `namespace.s3TablePublishStatus.lastIngestionTimes` |
| 已发布所有可用系统表 | `S3Tables.EnabledAll` | `namespace.s3TablePublishStatus.enabledAll` |

注意事项:

- 当存在多个目标时,`LogDestinationType`是**逗号分隔的列表**——例如`"cloudwatch,s3table"`。请使用子字符串/包含性检查,而非直接判断是否等于`s3table`。
- `LastIngestionTimes` / `lastIngestionTimes`为空,或某张表已列在已发布列表中但未出现在该映射里,意味着该表的数据可能仍在传输中。可对比连续多次查询的结果,确认新数据是否已落地。
- 在无服务器环境中,请勿通过顶层`logExports`字段查看此功能——该字段承载的是CloudWatch/S3审计日志(`useractivitylog`、`userlog`、`connectionlog`),与`SYS_*` S3 Tables发布无关。

2. Enable (if not configured)

2. 启用日志发布(若未配置)

bash
undefined
bash
undefined

Provisioned

预置集群

aws redshift enable-logging --region <REGION> --cluster-identifier <CLUSTER_ID> --log-destination-type s3table --log-exports <SYS_TABLE>... --s3-table-granularity <cluster|account> --s3-table-kms-key-id <KMS_KEY_ARN>
aws redshift enable-logging --region <REGION> --cluster-identifier <CLUSTER_ID> --log-destination-type s3table --log-exports <SYS_TABLE>... --s3-table-granularity <cluster|account> --s3-table-kms-key-id <KMS_KEY_ARN>

Serverless

无服务器命名空间

aws redshift-serverless update-namespace --region <REGION> --namespace-name <NAMESPACE_NAME> --log-destination-type s3table --s3-table-names <SYS_TABLE>... --s3-table-action Enable --s3-table-granularity <namespace|account> --s3-table-kms-key-id <KMS_KEY_ARN>

`--s3-table-kms-key-id` is part of both commands deliberately, not an optional add-on. Omitting it does not fail — the tables fall back to an AWS-owned key you cannot audit, restrict by policy, or revoke. Because `SYS_*` tables carry `query_text`, `user_name`, and `remote_host`, treat the customer-managed key as the default and drop the flag only for throwaway environments.

Enable from AWS console Amazon Redshift Console > Clusters > select your cluster > Tabs > Integrations / System table integration

**The two compute types take different flags for the same feature.** Do not carry Provisioned flag names over to Serverless:

| Purpose | Provisioned (`enable-logging`) | Serverless (`update-namespace`) |
|---|---|---|
| Which system tables to publish | `--log-exports` | `--s3-table-names` |
| Enable vs disable | separate `enable-logging` / `disable-logging` operations | `--s3-table-action Enable` \| `Disable` |
| Granularity | `--s3-table-granularity` `cluster` \| `account` | `--s3-table-granularity` `namespace` \| `account` |
| Customer-managed KMS key | `--s3-table-kms-key-id` | `--s3-table-kms-key-id` |
| Validate without applying | `--dry-run` | `--dry-run` |

Notes:

- Granularity: Provisioned supports `cluster` (default) or `account`; Serverless supports `namespace` (default) or `account`.
- `cluster`/`namespace` granularity → one S3 table per cluster/namespace; `account` → one shared table for all clusters/namespaces per account per region.
- Use `all` to publish all available `SYS_*` tables — `--log-exports all` on Provisioned, `--s3-table-names all` on Serverless.
- **Encryption at rest is strongly recommended for production.** Without `--s3-table-kms-key-id` the published tables are encrypted with an AWS-owned key, which you cannot audit, restrict by policy, or revoke. `SYS_*` tables carry `query_text`, `user_name`, and `remote_host` (see [Security Considerations](#security-considerations)), so pass a customer-managed key. Grant key access using the complete key policy in `${SKILL_DIR}/references/security.md` rather than an abbreviated action list — it needs **two** service principals (`systemtables.redshift.amazonaws.com` for publishing and `maintenance.s3tables.amazonaws.com` for table maintenance/compaction). Provisioning only the publishing principal lets writes succeed while compaction silently fails.
- Both operations accept `--dry-run` to validate the request without changing anything. Provisioned returns a `DryRunOperation` error on success ("Request would have succeeded, but DryRun flag is set"); Serverless returns an empty body and exit code 0. Note that the Serverless dry-run validates request *shape* only, not parameter values, so a successful dry-run there does not guarantee the values are accepted.

**Disable selectively:**

```bash
aws redshift-serverless update-namespace --region <REGION> --namespace-name <NAMESPACE_NAME> --log-destination-type s3table --s3-table-names <SYS_TABLE>... --s3-table-action Enable --s3-table-granularity <namespace|account> --s3-table-kms-key-id <KMS_KEY_ARN>

`--s3-table-kms-key-id`是两个命令的必填参数,而非可选附加项。省略该参数不会导致命令失败——表会默认使用AWS托管密钥,但你无法对其进行审计、策略限制或撤销操作。由于`SYS_*`表包含`query_text`、`user_name`和`remote_host`等敏感信息,建议默认使用客户托管密钥,仅在临时测试环境中可省略该参数。

也可通过AWS控制台启用:Amazon Redshift控制台 > 集群 > 选择你的集群 > 标签页 > 集成 / 系统表集成

**两种计算类型针对同一功能使用不同的命令参数**。请勿将预置集群的参数名称直接用于无服务器环境:

| 用途 | 预置集群(`enable-logging`) | 无服务器命名空间(`update-namespace`) |
|---|---|---|
| 指定要发布的系统表 | `--log-exports` | `--s3-table-names` |
| 启用/禁用 | 分别使用`enable-logging` / `disable-logging`操作 | `--s3-table-action Enable` \| `Disable` |
| 粒度 | `--s3-table-granularity` `cluster` \| `account` | `--s3-table-granularity` `namespace` \| `account` |
| 客户托管KMS密钥 | `--s3-table-kms-key-id` | `--s3-table-kms-key-id` |
| 验证请求而不执行 | `--dry-run` | `--dry-run` |

注意事项:

- 粒度:预置集群支持`cluster`(默认)或`account`;无服务器命名空间支持`namespace`(默认)或`account`。
- `cluster`/`namespace`粒度 → 每个集群/命名空间对应一个S3表;`account`粒度 → 每个区域内的所有集群/命名空间共享一个表。
- 使用`all`可发布所有可用的`SYS_*`表——预置集群使用`--log-exports all`,无服务器命名空间使用`--s3-table-names all`。
- **生产环境强烈建议启用静态加密**。若未指定`--s3-table-kms-key-id`,已发布的表将使用AWS托管密钥加密,你无法对其进行审计、策略限制或撤销操作。`SYS_*`表包含`query_text`、`user_name`和`remote_host`等敏感信息(详见[安全注意事项](#security-considerations)),因此请传入客户托管密钥。请使用`${SKILL_DIR}/references/security.md`中的完整密钥策略授予密钥访问权限,而非仅使用简化的操作列表——该策略需要**两个**服务主体(`systemtables.redshift.amazonaws.com`用于发布,`maintenance.s3tables.amazonaws.com`用于表维护/压缩)。若仅配置发布主体,写入操作可成功执行,但压缩操作会静默失败。
- 两个命令均支持`--dry-run`参数,可验证请求的有效性而不实际更改任何配置。预置集群在验证成功时会返回`DryRunOperation`错误("Request would have succeeded, but DryRun flag is set");无服务器命名空间会返回空响应体且退出码为0。请注意,无服务器环境的dry-run仅验证请求的格式,不验证参数值,因此dry-run成功并不代表参数值会被接受。

**选择性禁用日志发布:**

```bash

Provisioned

预置集群

aws redshift disable-logging --region <REGION> --cluster-identifier <CLUSTER_ID> --log-destination-type s3table --log-exports <SYS_TABLE>...
aws redshift disable-logging --region <REGION> --cluster-identifier <CLUSTER_ID> --log-destination-type s3table --log-exports <SYS_TABLE>...

Serverless

无服务器命名空间

aws redshift-serverless update-namespace --region <REGION> --namespace-name <NAMESPACE_NAME> --log-destination-type s3table --s3-table-names <SYS_TABLE>... --s3-table-action Disable
undefined
aws redshift-serverless update-namespace --region <REGION> --namespace-name <NAMESPACE_NAME> --log-destination-type s3table --s3-table-names <SYS_TABLE>... --s3-table-action Disable
undefined

3. Verify Permissions

3. 验证权限

Full setup commands for both paths:
${SKILL_DIR}/references/permissions-setup.md
. Load it before creating roles or registering resources.
Athena path — needs the
s3tablescatalog/aws-redshift
catalog registered in Glue, a workgroup with an output location, and S3 Tables read permissions. Confirm the catalog is queryable:
bash
aws glue get-databases --region <REGION> \
  --catalog-id "<ACCOUNT>:s3tablescatalog/aws-redshift"
Namespaces returned → registered and queryable.
EntityNotFoundException
/
CATALOG_NOT_FOUND
→ the S3 Tables integration is not enabled (S3 console > Table buckets > Enable integration). Encrypt the workgroup output location — Athena writes full result sets, including
query_text
and
user_name
, to S3.
Redshift auto-mount path — needs a Provisioned RA3 cluster and a four-step setup: create the
query_s3_tables
role (trust policy must name both
redshift.amazonaws.com
and
lakeformation.amazonaws.com
, the latter with all four of
sts:AssumeRole
,
sts:SetContext
,
sts:SetSourceIdentity
,
sts:TagSession
), attach it to the cluster, register the table bucket with Lake Formation, and add the Redshift service-linked roles to
ReadOnlyAdmins
. Constraints that cause most failures:
  • Condition both trust statements on
    aws:SourceAccount
    — a bare service principal is a confused-deputy risk.
  • Do not attach
    AWSLakeFormationDataAdmin
    to the cluster's query role.
    It is needed only by the principal performing setup, and only during setup. The cluster's role needs read access alone.
  • Auto-mount is a poll, not a callback — the catalog can take up to 300 seconds to appear in
    pg_database
    . A cluster reboot forces immediate discovery.
两种路径的完整设置命令:
${SKILL_DIR}/references/permissions-setup.md
。创建角色或注册资源前,请先查看该文件。
Athena路径——需要在Glue中注册
s3tablescatalog/aws-redshift
目录、配置带有输出位置的工作组,以及S3 Tables读取权限。确认目录可查询:
bash
aws glue get-databases --region <REGION> \
  --catalog-id "<ACCOUNT>:s3tablescatalog/aws-redshift"
返回命名空间 → 目录已注册且可查询。返回
EntityNotFoundException
/
CATALOG_NOT_FOUND
→ S3 Tables集成未启用(S3控制台 > 表存储桶 > 启用集成)。加密工作组输出位置——Athena会将完整结果集(包括
query_text
user_name
)写入S3。
Redshift自动挂载路径——需要预置RA3集群,并完成四步设置:创建
query_s3_tables
角色(信任策略必须同时包含
redshift.amazonaws.com
lakeformation.amazonaws.com
,后者需拥有
sts:AssumeRole
sts:SetContext
sts:SetSourceIdentity
sts:TagSession
这四个权限)、将角色附加到集群、在Lake Formation中注册表存储桶、将Redshift服务关联角色添加到
ReadOnlyAdmins
。最常导致失败的约束条件:
  • 为两个信任语句添加
    aws:SourceAccount
    条件
    ——仅指定服务主体存在混淆代理风险。
  • 请勿将
    AWSLakeFormationDataAdmin
    附加到集群的查询角色
    。该权限仅在设置期间由执行设置的主体需要,集群角色仅需读取权限。
  • 自动挂载是轮询机制,而非回调——目录可能需要最多300秒才会出现在
    pg_database
    中。重启集群可强制立即发现目录。

4. Identify the Target Table

4. 确定目标表

Namespace — resolve it from the API, do not construct it:
  • Read
    S3Tables.S3TableNamespace
    from
    describe-logging-status
    (Provisioned) or
    s3TablePublishStatus.s3TableNamespace
    from
    get-namespace
    (Serverless) and use it verbatim.
  • Optional sanity check only: the API value typically follows
    <namespace_arn_id>_sys
    for
    cluster
    /
    namespace
    granularity and
    <account>_sys
    for
    account
    granularity. Use this only to verify the value looks right — never to generate the namespace when the API response is unavailable.
Table — each publishable system table maps 1:1 to a table in the
aws-redshift
table bucket. Do not work from a memorized list — resolve it at runtime, in this order:
  1. The published set for this cluster/namespace
    S3Tables.S3Tables
    (Provisioned) or
    s3TablePublishStatus.s3Tables
    (Serverless) from the status call above, e.g.
    sys_query_history
    . This is the only authoritative answer to "what can I query right now".
  2. The set this API accepts
    aws redshift enable-logging help
    (accepted
    --log-exports
    values) or
    aws redshift-serverless update-namespace help
    (accepted
    --s3-table-names
    values).
  3. What each table contains — the public Redshift SYS monitoring views reference, which documents every
    SYS_*
    view and its columns. AWS adds views over time, so treat the docs as the current list rather than hardcoding one.
Column names and types come from the same public reference, or from the live table:
bash
aws glue get-table --region <REGION> \
  --catalog-id "<ACCOUNT>:s3tablescatalog/aws-redshift" \
  --database-name "<NAMESPACE>" --name "<SYS_TABLE>"
Two caveats when reading the public docs against a published table: enum-valued columns (
query_type
,
status
,
event
) gain values over time, so confirm with
SELECT DISTINCT
rather than filtering on an assumed set; and the published Iceberg table prepends warehouse-identity columns (
warehouse_name
,
warehouse_namespace_arn
, and peers) that the in-cluster
SYS_
view does not have — they are how you tell apart multiple clusters publishing at
account
granularity.
命名空间——从API响应中获取,请勿自行构造:
  • describe-logging-status
    (预置集群)的
    S3Tables.S3TableNamespace
    get-namespace
    (无服务器命名空间)的
    s3TablePublishStatus.s3TableNamespace
    中读取命名空间,并直接使用该值。
  • 可选的 sanity check:API返回的值通常遵循以下格式——
    cluster
    /
    namespace
    粒度为
    <namespace_arn_id>_sys
    account
    粒度为
    <account>_sys
    。仅用此格式验证返回值是否合理——当API响应不可用时,切勿自行生成命名空间。
——每个可发布的系统表与
aws-redshift
表存储桶中的一个表一一对应。请勿依赖记忆的列表——请按以下顺序在运行时确定:
  1. 此集群/命名空间的已发布表集合——上述状态查询中的
    S3Tables.S3Tables
    (预置集群)或
    s3TablePublishStatus.s3Tables
    (无服务器命名空间),例如
    sys_query_history
    。这是“我现在可以查询哪些表”的唯一权威答案。
  2. API接受的表集合——运行
    aws redshift enable-logging help
    查看可接受的
    --log-exports
    值,或运行
    aws redshift-serverless update-namespace help
    查看可接受的
    --s3-table-names
    值。
  3. 各表包含的内容——参考公开的Redshift SYS监控视图文档,该文档记录了每个
    SYS_*
    视图及其列。AWS会不断添加新视图,请以文档为准,而非硬编码列表。
列名和类型可从上述公开文档或实时表中获取:
bash
aws glue get-table --region <REGION> \
  --catalog-id "<ACCOUNT>:s3tablescatalog/aws-redshift" \
  --database-name "<NAMESPACE>" --name "<SYS_TABLE>"
对照公开文档查看已发布表时的两个注意事项:枚举值列(
query_type
status
event
)会不断新增取值,请使用
SELECT DISTINCT
确认,而非基于假设的集合进行过滤;已发布的Iceberg表会添加集群标识列(
warehouse_name
warehouse_namespace_arn
等),而集群内的
SYS_
视图没有这些列——这些列用于区分在
account
粒度下发布数据的多个集群。

5. Query

5. 查询

Query from Athena

从Athena查询

Query syntax:
sql
"s3tablescatalog/aws-redshift"."<NAMESPACE>"."<SYS_TABLE>"
查询语法:
sql
"s3tablescatalog/aws-redshift"."<NAMESPACE>"."<SYS_TABLE>"

Query from Redshift (Auto-Mounted Catalog)

从Redshift查询(自动挂载目录)

Once the auto-mounted catalog is set up (see
${SKILL_DIR}/references/permissions-setup.md
), query using cross-database notation:
sql
"aws-redshift@s3tablescatalog"."<NAMESPACE>".<SYS_TABLE>
完成自动挂载目录的设置后(详见
${SKILL_DIR}/references/permissions-setup.md
),使用跨数据库语法查询:
sql
"aws-redshift@s3tablescatalog"."<NAMESPACE>".<SYS_TABLE>

Query from Redshift (External Schema)

从Redshift查询(外部模式)

Alternatively, create an external schema pointing to the S3 Tables catalog:
sql
CREATE EXTERNAL SCHEMA <schema_name>
FROM DATA CATALOG
DATABASE '<NAMESPACE>'
CATALOG_ID '<ACCOUNT>:s3tablescatalog/aws-redshift'
IAM_ROLE 'arn:aws:iam::<ACCOUNT>:role/query_s3_tables'
REGION '<REGION>';

SELECT * FROM <schema_name>.<SYS_TABLE> LIMIT 10;
或者,创建指向S3 Tables目录的外部模式:
sql
CREATE EXTERNAL SCHEMA <schema_name>
FROM DATA CATALOG
DATABASE '<NAMESPACE>'
CATALOG_ID '<ACCOUNT>:s3tablescatalog/aws-redshift'
IAM_ROLE 'arn:aws:iam::<ACCOUNT>:role/query_s3_tables'
REGION '<REGION>';

SELECT * FROM <schema_name>.<SYS_TABLE> LIMIT 10;

Constraints

约束条件

  • You MUST run
    describe-logging-status
    or
    get-namespace
    to get the namespace before writing any SQL query — never construct it manually
  • For Athena queries, you MUST confirm workgroup and output location before executing
  • Timing columns are in microseconds. Divide by
    1000000.0
    for seconds
  • Tables are read-only — no
    INSERT
    /
    UPDATE
    /
    DELETE
  • Always add a
    LIMIT
    when the user doesn't specify one; filter on
    start_time
    /
    record_time
    where possible
  • 编写任何SQL查询前,必须运行
    describe-logging-status
    get-namespace
    获取命名空间——切勿手动构造
  • 执行Athena查询前,必须确认工作组和输出位置已配置
  • 时间列以微秒为单位。需除以
    1000000.0
    转换为秒
  • 表为只读——不支持
    INSERT
    /
    UPDATE
    /
    DELETE
    操作
  • 当用户未指定时,务必添加
    LIMIT
    ;尽可能根据
    start_time
    /
    record_time
    进行过滤

Examples

示例

Worked SQL for the common asks — longest-running queries, error analysis, connection auditing, queue-time trends, cross-table joins — is in
${SKILL_DIR}/references/example-queries.md
. Two rules that apply to every one of them:
  • Timing columns are microseconds. Divide by 1,000,000 for seconds. Reporting
    elapsed_time
    as-is overstates durations by 10^6.
  • Filter on the Iceberg partition columns (
    year
    /
    month
    /
    day
    or the table's own partitioning) in addition to any timestamp predicate, or the engine scans the full history.
常见需求的SQL示例(最长运行时间查询、错误分析、连接审计、队列时间趋势、跨表关联)请参见**
${SKILL_DIR}/references/example-queries.md
**。所有示例均需遵循以下两条规则:
  • 时间列以微秒为单位。需除以1,000,000转换为秒。直接使用
    elapsed_time
    会将持续时间夸大10^6倍。
  • 除了时间戳谓词外,还需根据Iceberg分区列过滤
    year
    /
    month
    /
    day
    或表自身的分区),否则引擎会扫描全部历史数据。

Routing: Athena vs Redshift vs Direct SYS_ Access

路由选择:Athena vs Redshift vs 直接访问SYS_

ScenarioUse
Historical/high-volume log analysis, no cluster loadAthena or Redshift on S3 Tables
Already connected to a Redshift cluster, want to query S3 Tables logsRedshift cross-database or external schema
Join system table logs with other lake dataAthena or Redshift Spectrum
Real-time current state of the clusterDirect
SYS_
view on the cluster
Quick ad-hoc query without Redshift cluster accessAthena
场景使用方式
历史/高容量日志分析,不希望给集群带来负载Athena或Redshift查询S3 Tables
已连接到Redshift集群,希望查询S3 Tables日志Redshift跨数据库或外部模式
将系统表日志与其他湖仓数据关联Athena或Redshift Spectrum
集群的实时当前状态直接在集群上查询
SYS_
视图
无Redshift集群访问权限的临时即席查询Athena

Key Behaviors

核心特性

  • No backfill — only events recorded after enabling are delivered to S3 Tables
  • Namespace from the API — always read the namespace from
    describe-logging-status
    (
    S3Tables.S3TableNamespace
    ) or
    get-namespace
    (
    s3TablePublishStatus.s3TableNamespace
    ); never construct it manually
  • Microsecond timing — all duration columns are in microseconds; divide by 1000000.0 for seconds
  • Read-only — published tables cannot be written to
  • Both Provisioned and Serverless — same table bucket (
    aws-redshift
    ), different enable APIs
  • Any Iceberg-compatible engine — query from Athena, Redshift, or any engine that reads Iceberg
  • 无回填——仅会将启用后记录的事件交付到S3 Tables
  • 从API获取命名空间——始终从
    describe-logging-status
    S3Tables.S3TableNamespace
    get-namespace
    s3TablePublishStatus.s3TableNamespace
    中读取命名空间;切勿手动构造
  • 微秒级计时——所有持续时间列均以微秒为单位;需除以1000000.0转换为秒
  • 只读——已发布的表无法写入
  • 同时支持预置和无服务器环境——使用同一个表存储桶(
    aws-redshift
    ),但启用API不同
  • 兼容任何Iceberg引擎——可从Athena、Redshift或任何支持Iceberg的引擎查询

Troubleshooting

故障排除

ErrorCauseFix
aws-redshift
bucket not found
S3 Tables integration not enabled or logging not startedRun
enable-logging
(Provisioned) or
update-namespace
(Serverless) with
--log-destination-type s3table
CATALOG_NOT_FOUND
in Athena
S3 Tables not registered in GlueEnable integration: S3 console > Table buckets > Enable integration
Athena table empty after enablingIngestion still in flightCheck
LastIngestionTimes
(Provisioned) /
lastIngestionTimes
(Serverless); wait and re-query
SYS_*
table missing from the namespace
System table not included when enablingRe-run enable with that table included, or use
all
--log-exports
(Provisioned),
--s3-table-names
(Serverless)
Wrong / empty namespaceNamespace constructed instead of read from APIUse the namespace from the describe/get response —
S3Tables.S3TableNamespace
(Provisioned) or
s3TablePublishStatus.s3TableNamespace
(Serverless)
Status response has no
S3Tables
/
s3TablePublishStatus
field at all, even though publishing is on
Outdated AWS CLI / SDK. The field is silently omitted rather than raising an error, so this looks identical to the feature being disabledUpgrade the CLI/SDK, then re-run. Confirm publishing is actually off before acting on the absence — check the
aws-redshift
table bucket for the namespace, or that
LogDestinationType
includes
s3table
Unknown options: --log-exports, --log-export-action
on Serverless
Provisioned flag names used against
update-namespace
Use
--s3-table-names
and
--s3-table-action
— see the flag table in the Enable section
AccessDenied
querying the table
Missing
s3tables:GetTable
or
GetTableData
See
references/security.md
Empty results from
sys_connection_log
Querying identity lacks visibilityUse an identity with superuser-level access
Catalog doesn't appear in
pg_database
LF resource not registered, or SLRs not ReadOnlyAdminsComplete the Lake Formation steps in
references/permissions-setup.md
, wait 5 min or reboot
"Unable to assume role" from GlueMissing
sts:SetContext
/
sts:SetSourceIdentity
in trust policy, or missing
AWSLakeFormationDataAdmin
Fix trust policy and attach
AWSLakeFormationDataAdmin
VerificationStatus: NOT_VERIFIED
Normal after Lake Formation registrationNo action needed if queries work
Query fails with "does not exist" in RedshiftCatalog not yet auto-mounted (poll delay)Wait up to 300s or reboot cluster
错误原因修复方案
aws-redshift
存储桶未找到
S3 Tables集成未启用或日志发布未启动运行
enable-logging
(预置集群)或
update-namespace
(无服务器命名空间)并指定
--log-destination-type s3table
Athena中返回
CATALOG_NOT_FOUND
S3 Tables未在Glue中注册启用集成:S3控制台 > 表存储桶 > 启用集成
启用后Athena表为空数据仍在传输中检查
LastIngestionTimes
(预置集群) /
lastIngestionTimes
(无服务器命名空间);等待后重新查询
命名空间中缺少
SYS_*
启用时未包含该系统表重新运行启用命令并包含该表,或使用
all
——预置集群用
--log-exports
,无服务器命名空间用
--s3-table-names
命名空间错误/为空手动构造了命名空间而非从API获取使用describe/get响应中的命名空间——
S3Tables.S3TableNamespace
(预置集群)或
s3TablePublishStatus.s3TableNamespace
(无服务器命名空间)
状态响应中完全没有
S3Tables
/
s3TablePublishStatus
字段,即使日志发布已开启
AWS CLI/SDK版本过旧。该字段会被静默省略而非报错,因此看起来与功能未启用的状态一致升级CLI/SDK后重新运行。在根据字段缺失采取行动前,请确认日志发布是否真的未开启——检查
aws-redshift
表存储桶中是否存在该命名空间,或
LogDestinationType
是否包含
s3table
无服务器环境中返回
Unknown options: --log-exports, --log-export-action
将预置集群的参数名称用于
update-namespace
命令
使用
--s3-table-names
--s3-table-action
——详情请参见启用部分的参数表
查询表时返回
AccessDenied
缺少
s3tables:GetTable
GetTableData
权限
参见
references/security.md
sys_connection_log
返回空结果
查询身份无可见性权限使用具有超级用户级别的身份查询
目录未出现在
pg_database
LF资源未注册,或SLR未加入ReadOnlyAdmins完成
references/permissions-setup.md
中的Lake Formation步骤,等待5分钟或重启集群
Glue返回“Unable to assume role”信任策略中缺少
sts:SetContext
/
sts:SetSourceIdentity
,或缺少
AWSLakeFormationDataAdmin
权限
修复信任策略并附加
AWSLakeFormationDataAdmin
权限
返回
VerificationStatus: NOT_VERIFIED
Lake Formation注册后的正常状态若查询可正常执行,则无需操作
Redshift中查询返回“does not exist”目录尚未自动挂载(轮询延迟)等待最多300秒或重启集群

Security Considerations

安全注意事项

Full policies, key policy, and detection setup:
${SKILL_DIR}/references/security.md
. Read it before granting access. The non-negotiables:
  • Scope IAM to the S3 Tables catalog, not wildcards. Glue database/table ARNs nest under
    s3tablescatalog/aws-redshift
    — the bare
    database/*
    form grants metadata read on the whole account.
    lakeformation:GetDataAccess
    is the one action that must use
    "Resource": "*"
    ; constrain it with an
    aws:ResourceAccount
    StringEquals
    condition.
  • The KMS key policy needs two principals, not one:
    systemtables.redshift.amazonaws.com
    (publisher) and
    maintenance.s3tables.amazonaws.com
    (compaction). Granting only the publisher lets writes succeed while compaction silently fails.
  • query_text
    can contain credentials
    , not just schema — interpolated SQL and
    CREATE USER ... PASSWORD
    land verbatim in
    sys_query_history
    . Treat broad access to that table as a secrets-exposure decision; restrict the column with Lake Formation.
  • Publishing is itself auditable and worth alarming on.
    s3tables.amazonaws.com
    AccessDenied
    spikes and
    sys_connection_log
    failed-auth counts are the two signals to alert on; encrypt the alarm topic with a customer-managed key.
完整的策略、密钥策略和检测设置请参见**
${SKILL_DIR}/references/security.md
**。授予访问权限前,请先阅读该文件。以下是必须遵守的规则:
  • 将IAM权限限定在S3 Tables目录范围内,而非使用通配符。Glue数据库/表ARN嵌套在
    s3tablescatalog/aws-redshift
    下——使用
    database/*
    会授予整个账户的元数据读取权限。
    lakeformation:GetDataAccess
    是唯一必须使用
    "Resource": "*"
    的操作;请通过
    aws:ResourceAccount
    StringEquals
    条件进行约束。
  • KMS密钥策略需要两个主体,而非一个:
    systemtables.redshift.amazonaws.com
    (发布者)和
    maintenance.s3tables.amazonaws.com
    (压缩)。若仅授予发布者权限,写入操作可成功,但压缩操作会静默失败。
  • query_text
    可能包含凭证
    ,而非仅包含架构信息——插值SQL和
    CREATE USER ... PASSWORD
    会原样存储在
    sys_query_history
    中。请将对该表的广泛访问视为可能泄露密钥的操作;通过Lake Formation限制该列的访问权限。
  • 日志发布本身可被审计且值得设置告警
    s3tables.amazonaws.com
    AccessDenied
    峰值和
    sys_connection_log
    中的失败认证次数是两个需要告警的信号;请使用客户托管密钥加密告警主题。

Reference Files

参考文件

${SKILL_DIR}
is the absolute path of the directory containing this SKILL.md. Load these on demand; do not read them all up front.
FileWhat it coversWhen to load
${SKILL_DIR}/references/permissions-setup.md
Athena prerequisites and workgroup encryption; the full Redshift auto-mount path — IAM role trust/inline policies, Lake Formation
register-resource
,
put-data-lake-settings
, the SLR
ReadOnlyAdmins
step, and the 300s auto-mount poll
Before running any IAM or Lake Formation setup
${SKILL_DIR}/references/example-queries.md
Worked SQL for longest-running queries, error analysis, connection auditing, queue-time trends, and joins across
sys_*
tables
When writing queries against the published tables
${SKILL_DIR}/references/security.md
Full least-privilege policy, KMS key policy with both service principals,
query_text
sensitivity, CloudTrail/metric-filter detection
Before granting access, or when hardening an existing setup
${SKILL_DIR}
是包含本SKILL.md的目录的绝对路径。请按需加载这些文件,无需提前全部阅读。
文件涵盖内容加载时机
${SKILL_DIR}/references/permissions-setup.md
Athena先决条件和工作组加密;完整的Redshift自动挂载路径——IAM角色信任/内联策略、Lake Formation的
register-resource
put-data-lake-settings
、SLR加入
ReadOnlyAdmins
步骤,以及300秒自动挂载轮询
执行任何IAM或Lake Formation设置前
${SKILL_DIR}/references/example-queries.md
最长运行时间查询、错误分析、连接审计、队列时间趋势以及跨
sys_*
表关联的SQL示例
编写针对已发布表的查询时
${SKILL_DIR}/references/security.md
完整的最小权限策略、包含两个服务主体的KMS密钥策略、
query_text
的敏感性、CloudTrail/指标过滤检测
授予访问权限前,或加固现有设置时

Additional Resources

额外资源