AWS S3 Management

AWS S3 管理

Overview

概述

Amazon S3 provides secure, durable, and highly scalable object storage. Manage buckets with encryption, versioning, access controls, lifecycle policies, and cross-region replication for reliable data storage and retrieval.

Amazon S3 提供安全、持久且高度可扩展的对象存储服务。可通过加密、版本控制、访问控制、生命周期策略和跨区域复制功能管理存储桶，实现可靠的数据存储与检索。

When to Use

适用场景

Static website hosting
Data backup and archival
Media library and CDN origin
Data lake and analytics
Log storage and analysis
Application asset storage
Disaster recovery
Data sharing and collaboration

静态网站托管
数据备份与归档
媒体库与CDN源站
数据湖与分析
日志存储与分析
应用资产存储
灾难恢复
数据共享与协作

Implementation Examples

实现示例

1. S3 Bucket Creation and Configuration with AWS CLI

1. 使用AWS CLI创建并配置S3存储桶

bash

undefined

bash

undefined

Create bucket

aws s3api create-bucket
--bucket my-app-bucket-$(date +%s)
--region us-east-1

Enable versioning

aws s3api put-bucket-versioning
--bucket my-app-bucket
--versioning-configuration Status=Enabled

Block public access

aws s3api put-public-access-block
--bucket my-app-bucket
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,
BlockPublicPolicy=true,RestrictPublicBuckets=true

Enable encryption

aws s3api put-bucket-encryption
--bucket my-app-bucket
--server-side-encryption-configuration '{ "Rules": [{ "ApplyServerSideEncryptionByDefault": { "SSEAlgorithm": "AES256" } }] }'

Upload file with metadata

aws s3 cp index.html s3://my-app-bucket/
--cache-control "max-age=3600"
--metadata "author=john,version=1"

Sync directory to S3

aws s3 sync ./dist s3://my-app-bucket/
--delete
--exclude "*.map"

List objects with metadata

aws s3api list-objects-v2
--bucket my-app-bucket
--query 'Contents[].{Key:Key,Size:Size,Modified:LastModified}'

undefined

aws s3api list-objects-v2
--bucket my-app-bucket
--query 'Contents[].{Key:Key,Size:Size,Modified:LastModified}'

undefined

2. S3 Lifecycle Policy Configuration

2. 配置S3生命周期策略

bash

undefined

bash

undefined

Create lifecycle policy

aws s3api put-bucket-lifecycle-configuration
--bucket my-app-bucket
--lifecycle-configuration '{ "Rules": [ { "Id": "archive-old-logs", "Status": "Enabled", "Prefix": "logs/", "Transitions": [ { "Days": 30, "StorageClass": "STANDARD_IA" }, { "Days": 90, "StorageClass": "GLACIER" } ], "Expiration": { "Days": 365 } }, { "Id": "cleanup-incomplete-uploads", "Status": "Enabled", "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 } } ] }'

Get bucket lifecycle

aws s3api get-bucket-lifecycle-configuration
--bucket my-app-bucket

undefined

aws s3api get-bucket-lifecycle-configuration
--bucket my-app-bucket

undefined

3. Terraform S3 Configuration

3. 使用Terraform配置S3

hcl

undefined

hcl

undefined

s3.tf

terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } }

provider "aws" { region = "us-east-1" }

terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } }

provider "aws" { region = "us-east-1" }

S3 bucket

resource "aws_s3_bucket" "app_data" { bucket = "my-app-data-${data.aws_caller_identity.current.account_id}" }

Block public access

resource "aws_s3_bucket_public_access_block" "app_data" { bucket = aws_s3_bucket.app_data.id

block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true }

resource "aws_s3_bucket_public_access_block" "app_data" { bucket = aws_s3_bucket.app_data.id

block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true }

Enable versioning

resource "aws_s3_bucket_versioning" "app_data" { bucket = aws_s3_bucket.app_data.id

versioning_configuration { status = "Enabled" } }

resource "aws_s3_bucket_versioning" "app_data" { bucket = aws_s3_bucket.app_data.id

versioning_configuration { status = "Enabled" } }

Server-side encryption

resource "aws_s3_bucket_server_side_encryption_configuration" "app_data" { bucket = aws_s3_bucket.app_data.id

rule { apply_server_side_encryption_by_default { sse_algorithm = "AES256" } } }

resource "aws_s3_bucket_server_side_encryption_configuration" "app_data" { bucket = aws_s3_bucket.app_data.id

rule { apply_server_side_encryption_by_default { sse_algorithm = "AES256" } } }

Lifecycle policy

resource "aws_s3_bucket_lifecycle_configuration" "app_data" { bucket = aws_s3_bucket.app_data.id

rule { id = "archive-logs" status = "Enabled"

filter {
  prefix = "logs/"
}

transition {
  days          = 30
  storage_class = "STANDARD_IA"
}

transition {
  days          = 90
  storage_class = "GLACIER"
}

expiration {
  days = 365
}

}

rule { id = "cleanup-incomplete-uploads" status = "Enabled"

abort_incomplete_multipart_upload {
  days_after_initiation = 7
}

} }

resource "aws_s3_bucket_lifecycle_configuration" "app_data" { bucket = aws_s3_bucket.app_data.id

rule { id = "archive-logs" status = "Enabled"

filter {
  prefix = "logs/"
}

transition {
  days          = 30
  storage_class = "STANDARD_IA"
}

transition {
  days          = 90
  storage_class = "GLACIER"
}

expiration {
  days = 365
}

}

rule { id = "cleanup-incomplete-uploads" status = "Enabled"

abort_incomplete_multipart_upload {
  days_after_initiation = 7
}

} }

CORS configuration

resource "aws_s3_bucket_cors_configuration" "app_data" { bucket = aws_s3_bucket.app_data.id

cors_rule { allowed_headers = ["*"] allowed_methods = ["GET", "PUT", "POST"] allowed_origins = ["https://example.com"] expose_headers = ["ETag"] max_age_seconds = 3000 } }

resource "aws_s3_bucket_cors_configuration" "app_data" { bucket = aws_s3_bucket.app_data.id

cors_rule { allowed_headers = ["*"] allowed_methods = ["GET", "PUT", "POST"] allowed_origins = ["https://example.com"] expose_headers = ["ETag"] max_age_seconds = 3000 } }

Bucket policy for CloudFront

resource "aws_s3_bucket_policy" "app_data" { bucket = aws_s3_bucket.app_data.id

policy = jsonencode({ Version = "2012-10-17" Statement = [ { Sid = "AllowCloudFront" Effect = "Allow" Principal = { Service = "cloudfront.amazonaws.com" } Action = "s3:GetObject" Resource = "${aws_s3_bucket.app_data.arn}/*" Condition = { StringEquals = { "AWS:SourceArn" = "arn:aws:cloudfront::${data.aws_caller_identity.current.account_id}:distribution/${aws_cloudfront_distribution.app.id}" } } } ] }) }

resource "aws_s3_bucket_policy" "app_data" { bucket = aws_s3_bucket.app_data.id

policy = jsonencode({ Version = "2012-10-17" Statement = [ { Sid = "AllowCloudFront" Effect = "Allow" Principal = { Service = "cloudfront.amazonaws.com" } Action = "s3:GetObject" Resource = "${aws_s3_bucket.app_data.arn}/*" Condition = { StringEquals = { "AWS:SourceArn" = "arn:aws:cloudfront::${data.aws_caller_identity.current.account_id}:distribution/${aws_cloudfront_distribution.app.id}" } } } ] }) }

Enable logging

resource "aws_s3_bucket_logging" "app_data" { bucket = aws_s3_bucket.app_data.id

target_bucket = aws_s3_bucket.logs.id target_prefix = "s3-logs/" }

resource "aws_s3_bucket_logging" "app_data" { bucket = aws_s3_bucket.app_data.id

target_bucket = aws_s3_bucket.logs.id target_prefix = "s3-logs/" }

Replication configuration

resource "aws_s3_bucket_replication_configuration" "app_data" { depends_on = [aws_s3_bucket_versioning.app_data] role = aws_iam_role.s3_replication.arn bucket = aws_s3_bucket.app_data.id

rule { status = "Enabled"

filter {}

destination {
  bucket       = aws_s3_bucket.replica.arn
  storage_class = "STANDARD_IA"

  replication_time {
    status = "Enabled"
    time {
      minutes = 15
    }
  }

  metrics {
    status = "Enabled"
    event_threshold {
      minutes = 15
    }
  }
}

} }

data "aws_caller_identity" "current" {}

resource "aws_s3_bucket_replication_configuration" "app_data" { depends_on = [aws_s3_bucket_versioning.app_data] role = aws_iam_role.s3_replication.arn bucket = aws_s3_bucket.app_data.id

rule { status = "Enabled"

filter {}

destination {
  bucket       = aws_s3_bucket.replica.arn
  storage_class = "STANDARD_IA"

  replication_time {
    status = "Enabled"
    time {
      minutes = 15
    }
  }

  metrics {
    status = "Enabled"
    event_threshold {
      minutes = 15
    }
  }
}

} }

data "aws_caller_identity" "current" {}

Replica bucket

resource "aws_s3_bucket" "replica" { bucket = "my-app-data-replica-${data.aws_caller_identity.current.account_id}" }

resource "aws_s3_bucket_versioning" "replica" { bucket = aws_s3_bucket.replica.id

versioning_configuration { status = "Enabled" } }

resource "aws_s3_bucket" "replica" { bucket = "my-app-data-replica-${data.aws_caller_identity.current.account_id}" }

resource "aws_s3_bucket_versioning" "replica" { bucket = aws_s3_bucket.replica.id

versioning_configuration { status = "Enabled" } }

Logs bucket

resource "aws_s3_bucket" "logs" { bucket = "my-app-logs-${data.aws_caller_identity.current.account_id}" }

IAM role for replication

resource "aws_iam_role" "s3_replication" { assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Action = "sts:AssumeRole" Effect = "Allow" Principal = { Service = "s3.amazonaws.com" } }] }) }

resource "aws_iam_role_policy" "s3_replication" { role = aws_iam_role.s3_replication.id

policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow" Action = [ "s3:GetReplicationConfiguration", "s3:ListBucket" ] Resource = aws_s3_bucket.app_data.arn }, { Effect = "Allow" Action = [ "s3:GetObjectVersionForReplication", "s3:GetObjectVersionAcl" ] Resource = "${aws_s3_bucket.app_data.arn}/" }, { Effect = "Allow" Action = [ "s3:ReplicateObject", "s3:ReplicateDelete" ] Resource = "${aws_s3_bucket.replica.arn}/" } ] }) }

undefined

resource "aws_iam_role" "s3_replication" { assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Action = "sts:AssumeRole" Effect = "Allow" Principal = { Service = "s3.amazonaws.com" } }] }) }

resource "aws_iam_role_policy" "s3_replication" { role = aws_iam_role.s3_replication.id

policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow" Action = [ "s3:GetReplicationConfiguration", "s3:ListBucket" ] Resource = aws_s3_bucket.app_data.arn }, { Effect = "Allow" Action = [ "s3:GetObjectVersionForReplication", "s3:GetObjectVersionAcl" ] Resource = "${aws_s3_bucket.app_data.arn}/" }, { Effect = "Allow" Action = [ "s3:ReplicateObject", "s3:ReplicateDelete" ] Resource = "${aws_s3_bucket.replica.arn}/" } ] }) }

undefined

4. S3 Access with Presigned URLs

4. 使用预签名URL访问S3

bash

undefined

bash

undefined

Generate presigned URL (1 hour expiration)

aws s3 presign s3://my-app-bucket/private/document.pdf
--expires-in 3600

Generate presigned URL for PUT (upload)

aws s3 presign s3://my-app-bucket/uploads/file.jpg
--expires-in 3600
--region us-east-1
--request-method PUT

undefined

aws s3 presign s3://my-app-bucket/uploads/file.jpg
--expires-in 3600
--region us-east-1
--request-method PUT

undefined

Best Practices

最佳实践

✅ DO

✅ 建议做法

Enable versioning for important data
Use server-side encryption
Block public access by default
Implement lifecycle policies
Enable logging and monitoring
Use bucket policies for access control
Enable MFA delete for critical buckets
Use IAM roles instead of access keys
Implement cross-region replication

为重要数据启用版本控制
使用服务器端加密
默认阻止公共访问
实施生命周期策略
启用日志记录与监控
使用存储桶策略进行访问控制
为关键存储桶启用MFA删除
使用IAM角色而非访问密钥
实施跨区域复制

❌ DON'T

❌ 避免做法

Make buckets publicly accessible
Store sensitive credentials
Ignore CloudTrail logging
Use overly permissive policies
Forget to set lifecycle rules
Ignore encryption requirements

将存储桶设置为公开可访问
存储敏感凭证
忽略CloudTrail日志记录
使用过于宽松的策略
忘记设置生命周期规则
忽略加密要求

Monitoring

监控

S3 CloudWatch metrics
CloudTrail for API logging
CloudWatch Alarms for threshold
S3 Inventory for object tracking
S3 Access Analyzer for permissions

S3 CloudWatch 指标
CloudTrail API日志记录
CloudWatch 阈值告警
S3 清单对象追踪
S3 访问分析器权限检查

aws-s3-management

Original

Translation

AWS S3 Management

AWS S3 管理

Overview

概述

When to Use

适用场景

Implementation Examples

实现示例

1. S3 Bucket Creation and Configuration with AWS CLI

1. 使用AWS CLI创建并配置S3存储桶

Create bucket

Create bucket

Enable versioning

Enable versioning

Block public access

Block public access

Enable encryption

Enable encryption

Upload file with metadata

Upload file with metadata

Sync directory to S3

Sync directory to S3

List objects with metadata

List objects with metadata

2. S3 Lifecycle Policy Configuration

2. 配置S3生命周期策略

Create lifecycle policy

Create lifecycle policy

Get bucket lifecycle

Get bucket lifecycle

3. Terraform S3 Configuration

3. 使用Terraform配置S3

s3.tf

s3.tf

S3 bucket

S3 bucket

Block public access

Block public access

Enable versioning

Enable versioning

Server-side encryption

Server-side encryption

Lifecycle policy

Lifecycle policy

CORS configuration

CORS configuration

Bucket policy for CloudFront

Bucket policy for CloudFront

Enable logging

Enable logging

Replication configuration

Replication configuration

Replica bucket

Replica bucket

Logs bucket

Logs bucket

IAM role for replication

IAM role for replication

4. S3 Access with Presigned URLs

4. 使用预签名URL访问S3

Generate presigned URL (1 hour expiration)

Generate presigned URL (1 hour expiration)

Generate presigned URL for PUT (upload)

Generate presigned URL for PUT (upload)

Best Practices

最佳实践

✅ DO

✅ 建议做法

❌ DON'T

❌ 避免做法

Monitoring

监控

Resources

资源