AWS SQS

Amazon Simple Queue Service (SQS) is a fully managed message queuing service for decoupling and scaling microservices, distributed systems, and serverless applications.

Amazon Simple Queue Service（SQS）是一项全托管的消息队列服务，用于解耦和扩展微服务、分布式系统以及无服务器应用。

Core Concepts

核心概念

Queue Types

队列类型

Type	Description	Use Case
Standard	At-least-once, best-effort ordering	High throughput
FIFO	Exactly-once, strict ordering	Order-sensitive processing

类型	描述	适用场景
标准队列	至少一次交付，尽力而为排序	高吞吐量
FIFO队列	恰好一次交付，严格排序	对顺序敏感的处理场景

Key Settings

关键设置

Setting	Description	Default
Visibility Timeout	Time message is hidden after receive	30 seconds
Message Retention	How long messages are kept	4 days (max 14)
Delay Seconds	Delay before message is available	0
Max Message Size	Maximum message size	256 KB

设置项	描述	默认值
可见性超时	消息被接收后隐藏的时间	30秒
消息保留时长	消息的保留时间	4天（最长14天）
延迟秒数	消息可用前的延迟时间	0
最大消息大小	消息的最大大小	256 KB

Dead-Letter Queue (DLQ)

死信队列（DLQ）

Queue for messages that failed processing after maxReceiveCount attempts.

用于存储达到最大接收次数（maxReceiveCount）后处理失败的消息的队列。

Common Patterns

常见模式

Create a Standard Queue

创建标准队列

AWS CLI:

bash

aws sqs create-queue \
  --queue-name my-queue \
  --attributes '{
    "VisibilityTimeout": "60",
    "MessageRetentionPeriod": "604800",
    "ReceiveMessageWaitTimeSeconds": "20"
  }'

boto3:

python

import boto3

sqs = boto3.client('sqs')

response = sqs.create_queue(
    QueueName='my-queue',
    Attributes={
        'VisibilityTimeout': '60',
        'MessageRetentionPeriod': '604800',
        'ReceiveMessageWaitTimeSeconds': '20'  # Long polling
    }
)
queue_url = response['QueueUrl']

AWS CLI:

bash

aws sqs create-queue \
  --queue-name my-queue \
  --attributes '{
    "VisibilityTimeout": "60",
    "MessageRetentionPeriod": "604800",
    "ReceiveMessageWaitTimeSeconds": "20"
  }'

boto3:

python

import boto3

sqs = boto3.client('sqs')

response = sqs.create_queue(
    QueueName='my-queue',
    Attributes={
        'VisibilityTimeout': '60',
        'MessageRetentionPeriod': '604800',
        'ReceiveMessageWaitTimeSeconds': '20'  # Long polling
    }
)
queue_url = response['QueueUrl']

Create FIFO Queue

创建FIFO队列

bash

aws sqs create-queue \
  --queue-name my-queue.fifo \
  --attributes '{
    "FifoQueue": "true",
    "ContentBasedDeduplication": "true"
  }'

bash

aws sqs create-queue \
  --queue-name my-queue.fifo \
  --attributes '{
    "FifoQueue": "true",
    "ContentBasedDeduplication": "true"
  }'

Configure Dead-Letter Queue

配置死信队列

bash

undefined

bash

undefined

Create DLQ

aws sqs create-queue --queue-name my-queue-dlq

Get DLQ ARN

DLQ_ARN=$(aws sqs get-queue-attributes
--queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue-dlq
--attribute-names QueueArn
--query 'Attributes.QueueArn' --output text)

Set redrive policy on main queue

aws sqs set-queue-attributes
--queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue
--attributes "{ "RedrivePolicy": "{\"deadLetterTargetArn\":\"${DLQ_ARN}\",\"maxReceiveCount\":\"3\"}" }"

undefined

aws sqs set-queue-attributes
--queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue
--attributes "{ "RedrivePolicy": "{\"deadLetterTargetArn\":\"${DLQ_ARN}\",\"maxReceiveCount\":\"3\"}" }"

undefined

Send Messages

发送消息

python

import boto3
import json

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue'

python

import boto3
import json

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue'

Send single message

sqs.send_message( QueueUrl=queue_url, MessageBody=json.dumps({'order_id': '12345', 'action': 'process'}), MessageAttributes={ 'MessageType': { 'DataType': 'String', 'StringValue': 'Order' } } )

Send to FIFO queue

sqs.send_message( QueueUrl='https://sqs.us-east-1.amazonaws.com/123456789012/my-queue.fifo', MessageBody=json.dumps({'order_id': '12345'}), MessageGroupId='order-12345', MessageDeduplicationId='unique-id-12345' )

Batch send (up to 10 messages)

sqs.send_message_batch( QueueUrl=queue_url, Entries=[ {'Id': '1', 'MessageBody': json.dumps({'id': 1})}, {'Id': '2', 'MessageBody': json.dumps({'id': 2})}, {'Id': '3', 'MessageBody': json.dumps({'id': 3})} ] )

undefined

sqs.send_message_batch( QueueUrl=queue_url, Entries=[ {'Id': '1', 'MessageBody': json.dumps({'id': 1})}, {'Id': '2', 'MessageBody': json.dumps({'id': 2})}, {'Id': '3', 'MessageBody': json.dumps({'id': 3})} ] )

undefined

Receive and Process Messages

接收并处理消息

python

import boto3
import json

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue'

while True:
    # Long polling (wait up to 20 seconds)
    response = sqs.receive_message(
        QueueUrl=queue_url,
        MaxNumberOfMessages=10,
        WaitTimeSeconds=20,
        MessageAttributeNames=['All'],
        AttributeNames=['All']
    )

    messages = response.get('Messages', [])

    for message in messages:
        try:
            body = json.loads(message['Body'])
            print(f"Processing: {body}")

            # Process message...

            # Delete on success
            sqs.delete_message(
                QueueUrl=queue_url,
                ReceiptHandle=message['ReceiptHandle']
            )
        except Exception as e:
            print(f"Error processing message: {e}")
            # Message will become visible again after visibility timeout

python

import boto3
import json

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue'

while True:
    # Long polling (wait up to 20 seconds)
    response = sqs.receive_message(
        QueueUrl=queue_url,
        MaxNumberOfMessages=10,
        WaitTimeSeconds=20,
        MessageAttributeNames=['All'],
        AttributeNames=['All']
    )

    messages = response.get('Messages', [])

    for message in messages:
        try:
            body = json.loads(message['Body'])
            print(f"Processing: {body}")

            # Process message...

            # Delete on success
            sqs.delete_message(
                QueueUrl=queue_url,
                ReceiptHandle=message['ReceiptHandle']
            )
        except Exception as e:
            print(f"Error processing message: {e}")
            # Message will become visible again after visibility timeout

Lambda Integration

Lambda集成

bash

undefined

bash

undefined

Create event source mapping

aws lambda create-event-source-mapping
--function-name my-function
--event-source-arn arn:aws:sqs:us-east-1:123456789012:my-queue
--batch-size 10
--maximum-batching-window-in-seconds 5


Lambda handler:

```python
def handler(event, context):
    for record in event['Records']:
        body = json.loads(record['body'])
        message_id = record['messageId']

        try:
            process_message(body)
        except Exception as e:
            # Raise to put message back in queue
            raise

    return {'batchItemFailures': []}

aws lambda create-event-source-mapping
--function-name my-function
--event-source-arn arn:aws:sqs:us-east-1:123456789012:my-queue
--batch-size 10
--maximum-batching-window-in-seconds 5


Lambda处理器：

```python
def handler(event, context):
    for record in event['Records']:
        body = json.loads(record['body'])
        message_id = record['messageId']

        try:
            process_message(body)
        except Exception as e:
            # Raise to put message back in queue
            raise

    return {'batchItemFailures': []}

CLI Reference

CLI参考

Queue Management

队列管理

Command	Description
`aws sqs create-queue`	Create queue
`aws sqs delete-queue`	Delete queue
`aws sqs list-queues`	List queues
`aws sqs get-queue-url`	Get queue URL by name
`aws sqs get-queue-attributes`	Get queue settings
`aws sqs set-queue-attributes`	Update queue settings

命令	描述
`aws sqs create-queue`	创建队列
`aws sqs delete-queue`	删除队列
`aws sqs list-queues`	列出队列
`aws sqs get-queue-url`	通过名称获取队列URL
`aws sqs get-queue-attributes`	获取队列设置
`aws sqs set-queue-attributes`	更新队列设置

Messaging

消息操作

Command	Description
`aws sqs send-message`	Send single message
`aws sqs send-message-batch`	Send up to 10 messages
`aws sqs receive-message`	Receive messages
`aws sqs delete-message`	Delete message
`aws sqs delete-message-batch`	Delete up to 10 messages
`aws sqs purge-queue`	Delete all messages

命令	描述
`aws sqs send-message`	发送单条消息
`aws sqs send-message-batch`	批量发送消息（最多10条）
`aws sqs receive-message`	接收消息
`aws sqs delete-message`	删除消息
`aws sqs delete-message-batch`	批量删除消息（最多10条）
`aws sqs purge-queue`	清空队列所有消息

Visibility

可见性管理

Command	Description
`aws sqs change-message-visibility`	Change timeout
`aws sqs change-message-visibility-batch`	Batch change

命令	描述
`aws sqs change-message-visibility`	修改可见性超时
`aws sqs change-message-visibility-batch`	批量修改可见性超时

Best Practices

最佳实践

Message Processing

消息处理

Use long polling (WaitTimeSeconds=20) to reduce API calls
Delete messages promptly after successful processing
Configure appropriate visibility timeout (> processing time)
Implement idempotent consumers for at-least-once delivery

使用长轮询（WaitTimeSeconds=20）以减少API调用
处理成功后及时删除消息
配置合适的可见性超时（大于处理时间）
实现幂等消费者以支持至少一次交付

Dead-Letter Queues

死信队列

Always configure DLQ for production queues
Set appropriate maxReceiveCount (usually 3-5)
Monitor DLQ depth with CloudWatch alarms
Process DLQ messages manually or with automation

生产队列始终配置DLQ
设置合适的maxReceiveCount（通常为3-5）
通过CloudWatch告警监控DLQ深度
手动或通过自动化处理DLQ消息

FIFO Queues

FIFO队列

Use message group IDs to partition ordering
Enable content-based deduplication or provide dedup IDs
Throughput: 300 msgs/sec without batching, 3000 with

使用消息组ID划分排序分区
启用基于内容的去重或提供去重ID
吞吐量：不使用批处理时为300条/秒，使用批处理时为3000条/秒

Security

安全

Use queue policies to control access
Enable encryption with SSE-SQS or SSE-KMS
Use VPC endpoints for private access

使用队列策略控制访问
启用加密（使用SSE-SQS或SSE-KMS）
使用VPC终端节点实现私有访问

Troubleshooting

故障排查

Messages Not Being Received

无法接收消息

Causes:

Short polling returning empty
All messages in flight (visibility timeout)
Messages delayed (DelaySeconds)

Debug:

bash

undefined

原因：

短轮询返回空结果
所有消息均处于飞行状态（可见性超时中）
消息被延迟（DelaySeconds设置）

调试：

bash

undefined

Check queue attributes

aws sqs get-queue-attributes
--queue-url $QUEUE_URL
--attribute-names All

Check approximate message counts

aws sqs get-queue-attributes
--queue-url $QUEUE_URL
--attribute-names
ApproximateNumberOfMessages,
ApproximateNumberOfMessagesNotVisible,
ApproximateNumberOfMessagesDelayed

undefined

aws sqs get-queue-attributes
--queue-url $QUEUE_URL
--attribute-names
ApproximateNumberOfMessages,
ApproximateNumberOfMessagesNotVisible,
ApproximateNumberOfMessagesDelayed

undefined

Messages Going to DLQ

消息进入DLQ

Causes:

Processing errors
Visibility timeout too short
Consumer not deleting messages

Redrive from DLQ:

bash

undefined

原因：

处理错误
可见性超时过短
消费者未删除消息

从DLQ重新投递：

bash

undefined

Enable redrive allow policy on source queue

aws sqs set-queue-attributes
--queue-url $MAIN_QUEUE_URL
--attributes '{"RedriveAllowPolicy": "{"redrivePermission":"allowAll"}"}'

Start redrive

aws sqs start-message-move-task
--source-arn arn:aws:sqs:us-east-1:123456789012:my-queue-dlq
--destination-arn arn:aws:sqs:us-east-1:123456789012:my-queue

undefined

aws sqs start-message-move-task
--source-arn arn:aws:sqs:us-east-1:123456789012:my-queue-dlq
--destination-arn arn:aws:sqs:us-east-1:123456789012:my-queue

undefined

Duplicate Processing

重复处理

Solutions:

Use FIFO queues for exactly-once
Implement idempotency in consumer
Track processed message IDs in database

解决方案：

使用FIFO队列实现恰好一次交付
在消费者中实现幂等性
在数据库中跟踪已处理的消息ID

Lambda Not Processing

Lambda未处理消息

bash

undefined

bash

undefined

Check event source mapping

aws lambda list-event-source-mappings
--function-name my-function

Check for errors

aws lambda get-event-source-mapping
--uuid <mapping-uuid>

undefined

aws lambda get-event-source-mapping
--uuid <mapping-uuid>

undefined

sqs

Original

Translation

AWS SQS

AWS SQS

Table of Contents

目录

Core Concepts

核心概念

Queue Types

队列类型

Key Settings

关键设置

Dead-Letter Queue (DLQ)

死信队列（DLQ）

Common Patterns

常见模式

Create a Standard Queue

创建标准队列

Create FIFO Queue

创建FIFO队列

Configure Dead-Letter Queue

配置死信队列

Create DLQ

Create DLQ

Get DLQ ARN

Get DLQ ARN

Set redrive policy on main queue

Set redrive policy on main queue

Send Messages

发送消息

Send single message

Send single message

Send to FIFO queue

Send to FIFO queue

Batch send (up to 10 messages)

Batch send (up to 10 messages)

Receive and Process Messages

接收并处理消息

Lambda Integration

Lambda集成

Create event source mapping

Create event source mapping

CLI Reference

CLI参考

Queue Management

队列管理

Messaging

消息操作

Visibility

可见性管理

Best Practices

最佳实践

Message Processing

消息处理

Dead-Letter Queues

死信队列

FIFO Queues

FIFO队列

Security

安全

Troubleshooting

故障排查

Messages Not Being Received

无法接收消息

Check queue attributes

Check queue attributes

Check approximate message counts

Check approximate message counts

Messages Going to DLQ

消息进入DLQ

Enable redrive allow policy on source queue

Enable redrive allow policy on source queue

Start redrive

Start redrive

Duplicate Processing

重复处理

Lambda Not Processing

Lambda未处理消息

Check event source mapping