managing-amazon-kinesis-data-streams
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseAmazon Kinesis Data Streams
Amazon Kinesis Data Streams
Overview
概述
Domain expertise for Amazon Kinesis Data Streams. This skill covers streaming tables and Amazon S3 delivery, which provide serverless, fully managed delivery from a KDS stream directly to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets — no consumers, no Firehose, no Flink required for append-only delivery.
Delivery is configured as a channel on a stream via . Channels consume no shard capacity and no enhanced fan-out slots, so they do not compete with existing consumers.
CreateChannelThe AWS MCP server is recommended for executing AWS operations. When the MCP server is not available, use the AWS CLI or shell commands instead.
Amazon Kinesis Data Streams的领域专业技能。本技能涵盖流表和Amazon S3交付，提供从KDS流直接到S3 Tables上的Apache Iceberg表或通用S3存储桶的无服务器、全托管交付——对于仅追加式交付，无需消费者、Firehose或Flink。
交付通过在流上配置为一个通道。通道不占用分片容量和增强扇出插槽，因此不会与现有消费者竞争资源。
CreateChannel推荐使用AWS MCP服务器执行AWS操作。当MCP服务器不可用时，请改用AWS CLI或Shell命令。
Which Workflow Do You Need?
您需要哪种工作流？
| Customer Intent | Reference |
|---|---|
| Deliver stream data to Apache Iceberg tables on S3 Tables — setup, IAM, Glue schema, type mapping, partitioning, channel lifecycle | references/streaming-tables.md |
| Deliver stream data to a general-purpose S3 bucket as objects — setup, IAM, compression, storage class, output key templates, channel lifecycle | references/general-purpose-delivery-to-s3.md |
| Build a lakehouse / data lake from Kinesis; make streaming data queryable in Athena | references/streaming-tables.md |
| Zero-ops, serverless alternative to Firehose or Kinesis consumers for KDS → S3 delivery | references/general-purpose-delivery-to-s3.md |
| CloudWatch metrics and alarms, delivery logging, CloudTrail, encryption, channel states, quotas, naming rules, what can be updated | references/monitoring-security-and-limits.md |
| Channel is ACTIVE but no data arrives; records landing in the DLQ; rising data freshness | Start with references/monitoring-security-and-limits.md, then the destination-specific troubleshooting table |
| Deliver to both S3 and S3 Tables from one stream | Supported — create one channel per destination |
| 客户需求 | 参考文档 |
|---|---|
| 将流数据交付到S3 Tables上的Apache Iceberg表——设置、IAM、Glue模式、类型映射、分区、通道生命周期 | references/streaming-tables.md |
| 将流数据作为对象交付到通用S3存储桶——设置、IAM、压缩、存储类别、输出密钥模板、通道生命周期 | references/general-purpose-delivery-to-s3.md |
| 基于Kinesis构建湖仓/数据湖；使流数据可在Athena中查询 | references/streaming-tables.md |
| KDS→S3交付的零运维、无服务器替代方案，替代Firehose或Kinesis消费者 | references/general-purpose-delivery-to-s3.md |
| CloudWatch指标与告警、交付日志、CloudTrail、加密、通道状态、配额、命名规则、可更新内容 | references/monitoring-security-and-limits.md |
| 通道处于ACTIVE状态但无数据到达；记录进入DLQ；数据新鲜度上升 | 先查看references/monitoring-security-and-limits.md，然后查看目标特定的故障排查表 |
| 从一个流同时交付到S3和S3 Tables | 支持——为每个目标创建一个通道 |
Prerequisites
前提条件
Both destinations require an ON_DEMAND stream — provisioned-mode streams are not supported. Each channel also needs an IAM service execution role trusted by , and a destination in the same Region as the stream.
kinesis.amazonaws.com两个目标都需要ON_DEMAND流——不支持预配置模式的流。每个通道还需要一个受信任的IAM服务执行角色，以及一个与流位于同一区域的目标。
kinesis.amazonaws.comHard Limits to Check First
首先要检查的硬限制
These rule out the feature entirely, so confirm them before designing a solution:
- Append-only — no CDC, upserts, or deletes
- No schema evolution — a schema change means deleting and recreating the channel
- No backfill — only records produced after the channel is are delivered
ACTIVE - No transformations — records are delivered as-is
- New table per channel — cannot deliver into an existing Iceberg table
- Data freshness is 300–900 seconds — not sub-minute
- Same Region only — cross-account is supported, cross-Region is not
If any of these block the customer, recommend Managed Service for Apache Flink instead.
这些限制会直接排除使用该功能的可能，因此在设计解决方案前请确认：
- 仅追加式——不支持CDC、更新插入或删除操作
- 无模式演进——模式变更意味着需要删除并重新创建通道
- 无回填——仅交付通道处于状态后生成的记录
ACTIVE - 无转换——记录将按原样交付
- 每个通道对应新表——无法交付到现有Iceberg表
- 数据新鲜度为300–900秒——不支持亚分钟级
- 仅同一区域——支持跨账户，但不支持跨区域
如果上述任何限制阻碍了客户需求，请推荐Apache Flink托管服务作为替代方案。