s3

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

AWS S3

AWS S3

Amazon Simple Storage Service (S3) provides scalable object storage with industry-leading durability (99.999999999%). S3 is fundamental to AWS—used for data lakes, backups, static websites, and as storage for many other AWS services.
Amazon Simple Storage Service(S3)提供可扩展的对象存储服务,具备行业领先的持久性(99.999999999%)。S3是AWS的核心服务之一,可用于数据湖、备份、静态网站托管,同时也是众多其他AWS服务的存储支撑。

Table of Contents

目录

Core Concepts

核心概念

Buckets

存储桶

Containers for objects. Bucket names are globally unique across all AWS accounts.
用于存储对象的容器。存储桶名称在所有AWS账户中具有全局唯一性。

Objects

对象

Files stored in S3, consisting of data, metadata, and a unique key (path). Maximum size: 5 TB.
存储在S3中的文件,由数据、元数据和唯一键(路径)组成。最大支持5TB的文件大小。

Storage Classes

存储类别

ClassUse CaseDurabilityAvailability
StandardFrequently accessed99.999999999%99.99%
Intelligent-TieringUnknown access patterns99.999999999%99.9%
Standard-IAInfrequent access99.999999999%99.9%
Glacier InstantArchive with instant retrieval99.999999999%99.9%
Glacier FlexibleArchive (minutes to hours)99.999999999%99.99%
Glacier Deep ArchiveLong-term archive99.999999999%99.99%
类别适用场景持久性可用性
Standard频繁访问99.999999999%99.9%
Intelligent-Tiering访问模式未知99.999999999%99.9%
Standard-IA不频繁访问99.999999999%99.9%
Glacier Instant可即时检索的归档存储99.999999999%99.9%
Glacier Flexible归档存储(检索需数分钟至数小时)99.999999999%99.99%
Glacier Deep Archive长期归档存储99.999999999%99.99%

Versioning

版本控制

Keeps multiple versions of an object. Essential for data protection and recovery.
保留对象的多个版本。是数据保护与恢复的关键功能。

Common Patterns

常见使用模式

Create a Bucket with Best Practices

遵循最佳实践创建存储桶

AWS CLI:
bash
undefined
AWS CLI:
bash
undefined

Create bucket (us-east-1 doesn't need LocationConstraint)

创建存储桶(us-east-1区域无需指定LocationConstraint)

aws s3api create-bucket
--bucket my-secure-bucket-12345
--region us-west-2
--create-bucket-configuration LocationConstraint=us-west-2
aws s3api create-bucket
--bucket my-secure-bucket-12345
--region us-west-2
--create-bucket-configuration LocationConstraint=us-west-2

Enable versioning

启用版本控制

aws s3api put-bucket-versioning
--bucket my-secure-bucket-12345
--versioning-configuration Status=Enabled
aws s3api put-bucket-versioning
--bucket my-secure-bucket-12345
--versioning-configuration Status=Enabled

Block public access

阻止公共访问

aws s3api put-public-access-block
--bucket my-secure-bucket-12345
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-public-access-block
--bucket my-secure-bucket-12345
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Enable encryption

启用加密

aws s3api put-bucket-encryption
--bucket my-secure-bucket-12345
--server-side-encryption-configuration '{ "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}] }'

**boto3:**

```python
import boto3

s3 = boto3.client('s3', region_name='us-west-2')
aws s3api put-bucket-encryption
--bucket my-secure-bucket-12345
--server-side-encryption-configuration '{ "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}] }'

**boto3:**

```python
import boto3

s3 = boto3.client('s3', region_name='us-west-2')

Create bucket

创建存储桶

s3.create_bucket( Bucket='my-secure-bucket-12345', CreateBucketConfiguration={'LocationConstraint': 'us-west-2'} )
s3.create_bucket( Bucket='my-secure-bucket-12345', CreateBucketConfiguration={'LocationConstraint': 'us-west-2'} )

Enable versioning

启用版本控制

s3.put_bucket_versioning( Bucket='my-secure-bucket-12345', VersioningConfiguration={'Status': 'Enabled'} )
s3.put_bucket_versioning( Bucket='my-secure-bucket-12345', VersioningConfiguration={'Status': 'Enabled'} )

Block public access

阻止公共访问

s3.put_public_access_block( Bucket='my-secure-bucket-12345', PublicAccessBlockConfiguration={ 'BlockPublicAcls': True, 'IgnorePublicAcls': True, 'BlockPublicPolicy': True, 'RestrictPublicBuckets': True } )
undefined
s3.put_public_access_block( Bucket='my-secure-bucket-12345', PublicAccessBlockConfiguration={ 'BlockPublicAcls': True, 'IgnorePublicAcls': True, 'BlockPublicPolicy': True, 'RestrictPublicBuckets': True } )
undefined

Upload and Download Objects

上传与下载对象

bash
undefined
bash
undefined

Upload a single file

上传单个文件

aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt

Upload with metadata

携带元数据上传

aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
--metadata "environment=production,version=1.0"
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
--metadata "environment=production,version=1.0"

Download a file

下载文件

aws s3 cp s3://my-bucket/path/myfile.txt ./myfile.txt
aws s3 cp s3://my-bucket/path/myfile.txt ./myfile.txt

Sync a directory

同步目录

aws s3 sync ./local-folder s3://my-bucket/prefix/ --delete
aws s3 sync ./local-folder s3://my-bucket/prefix/ --delete

Copy between buckets

在存储桶之间复制文件

aws s3 cp s3://source-bucket/file.txt s3://dest-bucket/file.txt
undefined
aws s3 cp s3://source-bucket/file.txt s3://dest-bucket/file.txt
undefined

Generate Presigned URL

生成预签名URL

python
import boto3
from botocore.config import Config

s3 = boto3.client('s3', config=Config(signature_version='s3v4'))
python
import boto3
from botocore.config import Config

s3 = boto3.client('s3', config=Config(signature_version='s3v4'))

Generate presigned URL for download (GET)

生成用于下载的预签名URL(GET请求)

url = s3.generate_presigned_url( 'get_object', Params={'Bucket': 'my-bucket', 'Key': 'path/to/file.txt'}, ExpiresIn=3600 # URL valid for 1 hour )
url = s3.generate_presigned_url( 'get_object', Params={'Bucket': 'my-bucket', 'Key': 'path/to/file.txt'}, ExpiresIn=3600 # URL有效期1小时 )

Generate presigned URL for upload (PUT)

生成用于上传的预签名URL(PUT请求)

upload_url = s3.generate_presigned_url( 'put_object', Params={ 'Bucket': 'my-bucket', 'Key': 'uploads/newfile.txt', 'ContentType': 'text/plain' }, ExpiresIn=3600 )
undefined
upload_url = s3.generate_presigned_url( 'put_object', Params={ 'Bucket': 'my-bucket', 'Key': 'uploads/newfile.txt', 'ContentType': 'text/plain' }, ExpiresIn=3600 )
undefined

Configure Lifecycle Policy

配置生命周期策略

bash
cat > lifecycle.json << 'EOF'
{
  "Rules": [
    {
      "ID": "MoveToGlacierAfter90Days",
      "Status": "Enabled",
      "Filter": {"Prefix": "logs/"},
      "Transitions": [
        {"Days": 90, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 365}
    },
    {
      "ID": "DeleteOldVersions",
      "Status": "Enabled",
      "Filter": {},
      "NoncurrentVersionExpiration": {"NoncurrentDays": 30}
    }
  ]
}
EOF

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json
bash
cat > lifecycle.json << 'EOF'
{
  "Rules": [
    {
      "ID": "MoveToGlacierAfter90Days",
      "Status": "Enabled",
      "Filter": {"Prefix": "logs/"},
      "Transitions": [
        {"Days": 90, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 365}
    },
    {
      "ID": "DeleteOldVersions",
      "Status": "Enabled",
      "Filter": {},
      "NoncurrentVersionExpiration": {"NoncurrentDays": 30}
    }
  ]
}
EOF

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json

Event Notifications to Lambda

配置Lambda事件通知

bash
aws s3api put-bucket-notification-configuration \
  --bucket my-bucket \
  --notification-configuration '{
    "LambdaFunctionConfigurations": [
      {
        "LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:ProcessS3Upload",
        "Events": ["s3:ObjectCreated:*"],
        "Filter": {
          "Key": {
            "FilterRules": [
              {"Name": "prefix", "Value": "uploads/"},
              {"Name": "suffix", "Value": ".jpg"}
            ]
          }
        }
      }
    ]
  }'
bash
aws s3api put-bucket-notification-configuration \
  --bucket my-bucket \
  --notification-configuration '{
    "LambdaFunctionConfigurations": [
      {
        "LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:ProcessS3Upload",
        "Events": ["s3:ObjectCreated:*"],
        "Filter": {
          "Key": {
            "FilterRules": [
              {"Name": "prefix", "Value": "uploads/"},
              {"Name": "suffix", "Value": ".jpg"}
            ]
          }
        }
      }
    ]
  }'

CLI Reference

CLI 参考

High-Level Commands (aws s3)

高级命令(aws s3)

CommandDescription
aws s3 ls
List buckets or objects
aws s3 cp
Copy files
aws s3 mv
Move files
aws s3 rm
Delete files
aws s3 sync
Sync directories
aws s3 mb
Make bucket
aws s3 rb
Remove bucket
命令描述
aws s3 ls
列出存储桶或对象
aws s3 cp
复制文件
aws s3 mv
移动文件
aws s3 rm
删除文件
aws s3 sync
同步目录
aws s3 mb
创建存储桶
aws s3 rb
删除存储桶

Low-Level Commands (aws s3api)

低级命令(aws s3api)

CommandDescription
aws s3api create-bucket
Create bucket with options
aws s3api put-object
Upload with full control
aws s3api get-object
Download with options
aws s3api delete-object
Delete single object
aws s3api put-bucket-policy
Set bucket policy
aws s3api put-bucket-versioning
Enable versioning
aws s3api list-object-versions
List all versions
命令描述
aws s3api create-bucket
通过参数配置创建存储桶
aws s3api put-object
完全可控的文件上传
aws s3api get-object
带参数配置的文件下载
aws s3api delete-object
删除单个对象
aws s3api put-bucket-policy
设置存储桶策略
aws s3api put-bucket-versioning
启用版本控制
aws s3api list-object-versions
列出所有对象版本

Useful Flags

实用参数

  • --recursive
    : Process all objects in prefix
  • --exclude/--include
    : Filter objects
  • --dryrun
    : Preview changes
  • --storage-class
    : Set storage class
  • --acl
    : Set access control (prefer policies instead)
  • --recursive
    : 处理前缀下的所有对象
  • --exclude/--include
    : 过滤对象
  • --dryrun
    : 预览变更
  • --storage-class
    : 设置存储类别
  • --acl
    : 设置访问控制(推荐使用策略替代)

Best Practices

最佳实践

Security

安全

  • Block public access at account and bucket level
  • Enable versioning for data protection
  • Use bucket policies over ACLs
  • Enable encryption (SSE-S3 or SSE-KMS)
  • Enable access logging for audit
  • Use VPC endpoints for private access
  • Enable MFA Delete for critical buckets
  • 在账户和存储桶层面阻止公共访问
  • 启用版本控制以保护数据
  • 优先使用存储桶策略而非ACL
  • 启用加密(SSE-S3或SSE-KMS)
  • 启用访问日志用于审计
  • 使用VPC终端节点实现私有访问
  • 为关键存储桶启用MFA删除

Performance

性能

  • Use Transfer Acceleration for distant uploads
  • Use multipart upload for files > 100 MB
  • Randomize key prefixes for high-throughput (less relevant with 2024 improvements)
  • Use byte-range fetches for large file downloads
  • 使用传输加速实现远距离上传
  • 对大于100MB的文件使用分段上传
  • 随机化键前缀以支持高吞吐量(2024年优化后重要性降低)
  • 使用字节范围获取下载大文件

Cost Optimization

成本优化

  • Use lifecycle policies to transition to cheaper storage
  • Enable Intelligent-Tiering for unpredictable access
  • Delete incomplete multipart uploads:
    json
    {
      "Rules": [{
        "ID": "AbortIncompleteMultipartUpload",
        "Status": "Enabled",
        "Filter": {},
        "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7}
      }]
    }
  • Use S3 Storage Lens to analyze storage patterns
  • 使用生命周期策略将数据过渡到更经济的存储类别
  • 为不可预测访问模式的数据启用Intelligent-Tiering
  • 删除未完成的分段上传:
    json
    {
      "Rules": [{
        "ID": "AbortIncompleteMultipartUpload",
        "Status": "Enabled",
        "Filter": {},
        "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7}
      }]
    }
  • 使用S3 Storage Lens分析存储模式

Troubleshooting

故障排查

Access Denied Errors

访问被拒绝错误

Causes:
  1. Bucket policy denies access
  2. IAM policy missing permissions
  3. Public access block preventing access
  4. Object owned by different account
  5. VPC endpoint policy blocking
Debug steps:
bash
undefined
原因:
  1. 存储桶策略拒绝访问
  2. IAM策略缺少权限
  3. 公共访问阻止规则限制访问
  4. 对象归属于其他账户
  5. VPC终端节点策略阻止访问
调试步骤:
bash
undefined

Check your identity

检查当前身份

aws sts get-caller-identity
aws sts get-caller-identity

Check bucket policy

检查存储桶策略

aws s3api get-bucket-policy --bucket my-bucket
aws s3api get-bucket-policy --bucket my-bucket

Check public access block

检查公共访问阻止规则

aws s3api get-public-access-block --bucket my-bucket
aws s3api get-public-access-block --bucket my-bucket

Check object ownership

检查对象所有权

aws s3api get-object-attributes
--bucket my-bucket
--key myfile.txt
--object-attributes ObjectOwner
undefined
aws s3api get-object-attributes
--bucket my-bucket
--key myfile.txt
--object-attributes ObjectOwner
undefined

CORS Errors

CORS错误

Symptom: Browser blocks cross-origin request
Fix:
bash
aws s3api put-bucket-cors --bucket my-bucket --cors-configuration '{
  "CORSRules": [{
    "AllowedOrigins": ["https://myapp.com"],
    "AllowedMethods": ["GET", "PUT", "POST"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3600
  }]
}'
症状: 浏览器阻止跨域请求
修复方案:
bash
aws s3api put-bucket-cors --bucket my-bucket --cors-configuration '{
  "CORSRules": [{
    "AllowedOrigins": ["https://myapp.com"],
    "AllowedMethods": ["GET", "PUT", "POST"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3600
  }]
}'

Slow Uploads

上传速度慢

Solutions:
  • Use multipart upload for large files
  • Enable Transfer Acceleration
  • Use
    aws s3 cp
    with
    --expected-size
    for large files
  • Check network throughput to the region
解决方案:
  • 对大文件使用分段上传
  • 启用传输加速
  • 对大文件使用
    aws s3 cp
    并添加
    --expected-size
    参数
  • 检查到目标区域的网络吞吐量

403 on Presigned URL

预签名URL返回403错误

Causes:
  • URL expired
  • Signer lacks permissions
  • Bucket policy blocks access
  • Region mismatch (v4 signatures are region-specific)
Fix: Ensure signer has permissions and use correct region.
原因:
  • URL已过期
  • 签名者缺少权限
  • 存储桶策略阻止访问
  • 区域不匹配(v4签名与区域绑定)
修复方案: 确保签名者拥有对应权限,并使用正确的区域。

References

参考资料