s3
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseAWS S3
AWS S3
Amazon Simple Storage Service (S3) provides scalable object storage with industry-leading durability (99.999999999%). S3 is fundamental to AWS—used for data lakes, backups, static websites, and as storage for many other AWS services.
Amazon Simple Storage Service(S3)提供可扩展的对象存储服务,具备行业领先的持久性(99.999999999%)。S3是AWS的核心服务之一,可用于数据湖、备份、静态网站托管,同时也是众多其他AWS服务的存储支撑。
Table of Contents
目录
Core Concepts
核心概念
Buckets
存储桶
Containers for objects. Bucket names are globally unique across all AWS accounts.
用于存储对象的容器。存储桶名称在所有AWS账户中具有全局唯一性。
Objects
对象
Files stored in S3, consisting of data, metadata, and a unique key (path). Maximum size: 5 TB.
存储在S3中的文件,由数据、元数据和唯一键(路径)组成。最大支持5TB的文件大小。
Storage Classes
存储类别
| Class | Use Case | Durability | Availability |
|---|---|---|---|
| Standard | Frequently accessed | 99.999999999% | 99.99% |
| Intelligent-Tiering | Unknown access patterns | 99.999999999% | 99.9% |
| Standard-IA | Infrequent access | 99.999999999% | 99.9% |
| Glacier Instant | Archive with instant retrieval | 99.999999999% | 99.9% |
| Glacier Flexible | Archive (minutes to hours) | 99.999999999% | 99.99% |
| Glacier Deep Archive | Long-term archive | 99.999999999% | 99.99% |
| 类别 | 适用场景 | 持久性 | 可用性 |
|---|---|---|---|
| Standard | 频繁访问 | 99.999999999% | 99.9% |
| Intelligent-Tiering | 访问模式未知 | 99.999999999% | 99.9% |
| Standard-IA | 不频繁访问 | 99.999999999% | 99.9% |
| Glacier Instant | 可即时检索的归档存储 | 99.999999999% | 99.9% |
| Glacier Flexible | 归档存储(检索需数分钟至数小时) | 99.999999999% | 99.99% |
| Glacier Deep Archive | 长期归档存储 | 99.999999999% | 99.99% |
Versioning
版本控制
Keeps multiple versions of an object. Essential for data protection and recovery.
保留对象的多个版本。是数据保护与恢复的关键功能。
Common Patterns
常见使用模式
Create a Bucket with Best Practices
遵循最佳实践创建存储桶
AWS CLI:
bash
undefinedAWS CLI:
bash
undefinedCreate bucket (us-east-1 doesn't need LocationConstraint)
创建存储桶(us-east-1区域无需指定LocationConstraint)
aws s3api create-bucket
--bucket my-secure-bucket-12345
--region us-west-2
--create-bucket-configuration LocationConstraint=us-west-2
--bucket my-secure-bucket-12345
--region us-west-2
--create-bucket-configuration LocationConstraint=us-west-2
aws s3api create-bucket
--bucket my-secure-bucket-12345
--region us-west-2
--create-bucket-configuration LocationConstraint=us-west-2
--bucket my-secure-bucket-12345
--region us-west-2
--create-bucket-configuration LocationConstraint=us-west-2
Enable versioning
启用版本控制
aws s3api put-bucket-versioning
--bucket my-secure-bucket-12345
--versioning-configuration Status=Enabled
--bucket my-secure-bucket-12345
--versioning-configuration Status=Enabled
aws s3api put-bucket-versioning
--bucket my-secure-bucket-12345
--versioning-configuration Status=Enabled
--bucket my-secure-bucket-12345
--versioning-configuration Status=Enabled
Block public access
阻止公共访问
aws s3api put-public-access-block
--bucket my-secure-bucket-12345
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
--bucket my-secure-bucket-12345
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-public-access-block
--bucket my-secure-bucket-12345
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
--bucket my-secure-bucket-12345
--public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
Enable encryption
启用加密
aws s3api put-bucket-encryption
--bucket my-secure-bucket-12345
--server-side-encryption-configuration '{ "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}] }'
--bucket my-secure-bucket-12345
--server-side-encryption-configuration '{ "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}] }'
**boto3:**
```python
import boto3
s3 = boto3.client('s3', region_name='us-west-2')aws s3api put-bucket-encryption
--bucket my-secure-bucket-12345
--server-side-encryption-configuration '{ "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}] }'
--bucket my-secure-bucket-12345
--server-side-encryption-configuration '{ "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}] }'
**boto3:**
```python
import boto3
s3 = boto3.client('s3', region_name='us-west-2')Create bucket
创建存储桶
s3.create_bucket(
Bucket='my-secure-bucket-12345',
CreateBucketConfiguration={'LocationConstraint': 'us-west-2'}
)
s3.create_bucket(
Bucket='my-secure-bucket-12345',
CreateBucketConfiguration={'LocationConstraint': 'us-west-2'}
)
Enable versioning
启用版本控制
s3.put_bucket_versioning(
Bucket='my-secure-bucket-12345',
VersioningConfiguration={'Status': 'Enabled'}
)
s3.put_bucket_versioning(
Bucket='my-secure-bucket-12345',
VersioningConfiguration={'Status': 'Enabled'}
)
Block public access
阻止公共访问
s3.put_public_access_block(
Bucket='my-secure-bucket-12345',
PublicAccessBlockConfiguration={
'BlockPublicAcls': True,
'IgnorePublicAcls': True,
'BlockPublicPolicy': True,
'RestrictPublicBuckets': True
}
)
undefineds3.put_public_access_block(
Bucket='my-secure-bucket-12345',
PublicAccessBlockConfiguration={
'BlockPublicAcls': True,
'IgnorePublicAcls': True,
'BlockPublicPolicy': True,
'RestrictPublicBuckets': True
}
)
undefinedUpload and Download Objects
上传与下载对象
bash
undefinedbash
undefinedUpload a single file
上传单个文件
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
Upload with metadata
携带元数据上传
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
--metadata "environment=production,version=1.0"
--metadata "environment=production,version=1.0"
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
--metadata "environment=production,version=1.0"
--metadata "environment=production,version=1.0"
Download a file
下载文件
aws s3 cp s3://my-bucket/path/myfile.txt ./myfile.txt
aws s3 cp s3://my-bucket/path/myfile.txt ./myfile.txt
Sync a directory
同步目录
aws s3 sync ./local-folder s3://my-bucket/prefix/ --delete
aws s3 sync ./local-folder s3://my-bucket/prefix/ --delete
Copy between buckets
在存储桶之间复制文件
aws s3 cp s3://source-bucket/file.txt s3://dest-bucket/file.txt
undefinedaws s3 cp s3://source-bucket/file.txt s3://dest-bucket/file.txt
undefinedGenerate Presigned URL
生成预签名URL
python
import boto3
from botocore.config import Config
s3 = boto3.client('s3', config=Config(signature_version='s3v4'))python
import boto3
from botocore.config import Config
s3 = boto3.client('s3', config=Config(signature_version='s3v4'))Generate presigned URL for download (GET)
生成用于下载的预签名URL(GET请求)
url = s3.generate_presigned_url(
'get_object',
Params={'Bucket': 'my-bucket', 'Key': 'path/to/file.txt'},
ExpiresIn=3600 # URL valid for 1 hour
)
url = s3.generate_presigned_url(
'get_object',
Params={'Bucket': 'my-bucket', 'Key': 'path/to/file.txt'},
ExpiresIn=3600 # URL有效期1小时
)
Generate presigned URL for upload (PUT)
生成用于上传的预签名URL(PUT请求)
upload_url = s3.generate_presigned_url(
'put_object',
Params={
'Bucket': 'my-bucket',
'Key': 'uploads/newfile.txt',
'ContentType': 'text/plain'
},
ExpiresIn=3600
)
undefinedupload_url = s3.generate_presigned_url(
'put_object',
Params={
'Bucket': 'my-bucket',
'Key': 'uploads/newfile.txt',
'ContentType': 'text/plain'
},
ExpiresIn=3600
)
undefinedConfigure Lifecycle Policy
配置生命周期策略
bash
cat > lifecycle.json << 'EOF'
{
"Rules": [
{
"ID": "MoveToGlacierAfter90Days",
"Status": "Enabled",
"Filter": {"Prefix": "logs/"},
"Transitions": [
{"Days": 90, "StorageClass": "GLACIER"}
],
"Expiration": {"Days": 365}
},
{
"ID": "DeleteOldVersions",
"Status": "Enabled",
"Filter": {},
"NoncurrentVersionExpiration": {"NoncurrentDays": 30}
}
]
}
EOF
aws s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration file://lifecycle.jsonbash
cat > lifecycle.json << 'EOF'
{
"Rules": [
{
"ID": "MoveToGlacierAfter90Days",
"Status": "Enabled",
"Filter": {"Prefix": "logs/"},
"Transitions": [
{"Days": 90, "StorageClass": "GLACIER"}
],
"Expiration": {"Days": 365}
},
{
"ID": "DeleteOldVersions",
"Status": "Enabled",
"Filter": {},
"NoncurrentVersionExpiration": {"NoncurrentDays": 30}
}
]
}
EOF
aws s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration file://lifecycle.jsonEvent Notifications to Lambda
配置Lambda事件通知
bash
aws s3api put-bucket-notification-configuration \
--bucket my-bucket \
--notification-configuration '{
"LambdaFunctionConfigurations": [
{
"LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:ProcessS3Upload",
"Events": ["s3:ObjectCreated:*"],
"Filter": {
"Key": {
"FilterRules": [
{"Name": "prefix", "Value": "uploads/"},
{"Name": "suffix", "Value": ".jpg"}
]
}
}
}
]
}'bash
aws s3api put-bucket-notification-configuration \
--bucket my-bucket \
--notification-configuration '{
"LambdaFunctionConfigurations": [
{
"LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:ProcessS3Upload",
"Events": ["s3:ObjectCreated:*"],
"Filter": {
"Key": {
"FilterRules": [
{"Name": "prefix", "Value": "uploads/"},
{"Name": "suffix", "Value": ".jpg"}
]
}
}
}
]
}'CLI Reference
CLI 参考
High-Level Commands (aws s3)
高级命令(aws s3)
| Command | Description |
|---|---|
| List buckets or objects |
| Copy files |
| Move files |
| Delete files |
| Sync directories |
| Make bucket |
| Remove bucket |
| 命令 | 描述 |
|---|---|
| 列出存储桶或对象 |
| 复制文件 |
| 移动文件 |
| 删除文件 |
| 同步目录 |
| 创建存储桶 |
| 删除存储桶 |
Low-Level Commands (aws s3api)
低级命令(aws s3api)
| Command | Description |
|---|---|
| Create bucket with options |
| Upload with full control |
| Download with options |
| Delete single object |
| Set bucket policy |
| Enable versioning |
| List all versions |
| 命令 | 描述 |
|---|---|
| 通过参数配置创建存储桶 |
| 完全可控的文件上传 |
| 带参数配置的文件下载 |
| 删除单个对象 |
| 设置存储桶策略 |
| 启用版本控制 |
| 列出所有对象版本 |
Useful Flags
实用参数
- : Process all objects in prefix
--recursive - : Filter objects
--exclude/--include - : Preview changes
--dryrun - : Set storage class
--storage-class - : Set access control (prefer policies instead)
--acl
- : 处理前缀下的所有对象
--recursive - : 过滤对象
--exclude/--include - : 预览变更
--dryrun - : 设置存储类别
--storage-class - : 设置访问控制(推荐使用策略替代)
--acl
Best Practices
最佳实践
Security
安全
- Block public access at account and bucket level
- Enable versioning for data protection
- Use bucket policies over ACLs
- Enable encryption (SSE-S3 or SSE-KMS)
- Enable access logging for audit
- Use VPC endpoints for private access
- Enable MFA Delete for critical buckets
- 在账户和存储桶层面阻止公共访问
- 启用版本控制以保护数据
- 优先使用存储桶策略而非ACL
- 启用加密(SSE-S3或SSE-KMS)
- 启用访问日志用于审计
- 使用VPC终端节点实现私有访问
- 为关键存储桶启用MFA删除
Performance
性能
- Use Transfer Acceleration for distant uploads
- Use multipart upload for files > 100 MB
- Randomize key prefixes for high-throughput (less relevant with 2024 improvements)
- Use byte-range fetches for large file downloads
- 使用传输加速实现远距离上传
- 对大于100MB的文件使用分段上传
- 随机化键前缀以支持高吞吐量(2024年优化后重要性降低)
- 使用字节范围获取下载大文件
Cost Optimization
成本优化
- Use lifecycle policies to transition to cheaper storage
- Enable Intelligent-Tiering for unpredictable access
- Delete incomplete multipart uploads:
json
{ "Rules": [{ "ID": "AbortIncompleteMultipartUpload", "Status": "Enabled", "Filter": {}, "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7} }] } - Use S3 Storage Lens to analyze storage patterns
- 使用生命周期策略将数据过渡到更经济的存储类别
- 为不可预测访问模式的数据启用Intelligent-Tiering
- 删除未完成的分段上传:
json
{ "Rules": [{ "ID": "AbortIncompleteMultipartUpload", "Status": "Enabled", "Filter": {}, "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7} }] } - 使用S3 Storage Lens分析存储模式
Troubleshooting
故障排查
Access Denied Errors
访问被拒绝错误
Causes:
- Bucket policy denies access
- IAM policy missing permissions
- Public access block preventing access
- Object owned by different account
- VPC endpoint policy blocking
Debug steps:
bash
undefined原因:
- 存储桶策略拒绝访问
- IAM策略缺少权限
- 公共访问阻止规则限制访问
- 对象归属于其他账户
- VPC终端节点策略阻止访问
调试步骤:
bash
undefinedCheck your identity
检查当前身份
aws sts get-caller-identity
aws sts get-caller-identity
Check bucket policy
检查存储桶策略
aws s3api get-bucket-policy --bucket my-bucket
aws s3api get-bucket-policy --bucket my-bucket
Check public access block
检查公共访问阻止规则
aws s3api get-public-access-block --bucket my-bucket
aws s3api get-public-access-block --bucket my-bucket
Check object ownership
检查对象所有权
aws s3api get-object-attributes
--bucket my-bucket
--key myfile.txt
--object-attributes ObjectOwner
--bucket my-bucket
--key myfile.txt
--object-attributes ObjectOwner
undefinedaws s3api get-object-attributes
--bucket my-bucket
--key myfile.txt
--object-attributes ObjectOwner
--bucket my-bucket
--key myfile.txt
--object-attributes ObjectOwner
undefinedCORS Errors
CORS错误
Symptom: Browser blocks cross-origin request
Fix:
bash
aws s3api put-bucket-cors --bucket my-bucket --cors-configuration '{
"CORSRules": [{
"AllowedOrigins": ["https://myapp.com"],
"AllowedMethods": ["GET", "PUT", "POST"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600
}]
}'症状: 浏览器阻止跨域请求
修复方案:
bash
aws s3api put-bucket-cors --bucket my-bucket --cors-configuration '{
"CORSRules": [{
"AllowedOrigins": ["https://myapp.com"],
"AllowedMethods": ["GET", "PUT", "POST"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600
}]
}'Slow Uploads
上传速度慢
Solutions:
- Use multipart upload for large files
- Enable Transfer Acceleration
- Use with
aws s3 cpfor large files--expected-size - Check network throughput to the region
解决方案:
- 对大文件使用分段上传
- 启用传输加速
- 对大文件使用并添加
aws s3 cp参数--expected-size - 检查到目标区域的网络吞吐量
403 on Presigned URL
预签名URL返回403错误
Causes:
- URL expired
- Signer lacks permissions
- Bucket policy blocks access
- Region mismatch (v4 signatures are region-specific)
Fix: Ensure signer has permissions and use correct region.
原因:
- URL已过期
- 签名者缺少权限
- 存储桶策略阻止访问
- 区域不匹配(v4签名与区域绑定)
修复方案: 确保签名者拥有对应权限,并使用正确的区域。