Back to Details

cloud-storage

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Cloud Object Storage

云对象存储

AWS S3 (Recommended Default)

AWS S3（推荐默认方案）

Setup (SDK v3)

设置（SDK v3）

typescript
import { S3Client, PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({ region: process.env.AWS_REGION });
typescript
import { S3Client, PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({ region: process.env.AWS_REGION });

Upload

上传

typescript
// Direct upload
await s3.send(new PutObjectCommand({
  Bucket: process.env.S3_BUCKET,
  Key: `uploads/${userId}/${filename}`,
  Body: buffer,
  ContentType: mimeType,
  Metadata: { 'uploaded-by': userId },
}));

// Presigned URL (client uploads directly to S3)
const url = await getSignedUrl(s3, new PutObjectCommand({
  Bucket: process.env.S3_BUCKET,
  Key: `uploads/${key}`,
  ContentType: mimeType,
}), { expiresIn: 3600 });
typescript
// 直接上传
await s3.send(new PutObjectCommand({
  Bucket: process.env.S3_BUCKET,
  Key: `uploads/${userId}/${filename}`,
  Body: buffer,
  ContentType: mimeType,
  Metadata: { 'uploaded-by': userId },
}));

// 预签名URL（客户端直接上传至S3）
const url = await getSignedUrl(s3, new PutObjectCommand({
  Bucket: process.env.S3_BUCKET,
  Key: `uploads/${key}`,
  ContentType: mimeType,
}), { expiresIn: 3600 });

Presigned Download

预签名下载

typescript
const url = await getSignedUrl(s3, new GetObjectCommand({
  Bucket: process.env.S3_BUCKET,
  Key: fileKey,
}), { expiresIn: 3600 });
typescript
const url = await getSignedUrl(s3, new GetObjectCommand({
  Bucket: process.env.S3_BUCKET,
  Key: fileKey,
}), { expiresIn: 3600 });

Multipart Upload (large files)

分块上传（大文件）

typescript
import { CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand } from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';

// Simplified with lib-storage
const upload = new Upload({
  client: s3,
  params: { Bucket: bucket, Key: key, Body: readableStream },
  partSize: 10 * 1024 * 1024, // 10MB parts
  leavePartsOnError: false,
});
upload.on('httpUploadProgress', (progress) => console.log(progress));
await upload.done();
typescript
import { CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand } from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';

// 使用lib-storage简化实现
const upload = new Upload({
  client: s3,
  params: { Bucket: bucket, Key: key, Body: readableStream },
  partSize: 10 * 1024 * 1024, // 10MB分块
  leavePartsOnError: false,
});
upload.on('httpUploadProgress', (progress) => console.log(progress));
await upload.done();

Python (boto3)

Python（boto3）

python
import boto3
from botocore.config import Config

s3 = boto3.client('s3', config=Config(signature_version='s3v4'))
python
import boto3
from botocore.config import Config

s3 = boto3.client('s3', config=Config(signature_version='s3v4'))

Upload

上传

s3.upload_fileobj(file_obj, bucket, key, ExtraArgs={'ContentType': mime_type})
s3.upload_fileobj(file_obj, bucket, key, ExtraArgs={'ContentType': mime_type})

Presigned URL

预签名URL

url = s3.generate_presigned_url('put_object', Params={'Bucket': bucket, 'Key': key, 'ContentType': mime_type}, ExpiresIn=3600)
undefined
url = s3.generate_presigned_url('put_object', Params={'Bucket': bucket, 'Key': key, 'ContentType': mime_type}, ExpiresIn=3600)
undefined

Java (Spring)

Java（Spring）

java
@Service
public class S3Service {
    private final S3Client s3;

    public String generatePresignedUploadUrl(String key, String contentType) {
        var presigner = S3Presigner.builder().region(Region.of(region)).build();
        var request = PutObjectPresignRequest.builder()
            .signatureDuration(Duration.ofHours(1))
            .putObjectRequest(b -> b.bucket(bucket).key(key).contentType(contentType))
            .build();
        return presigner.presignPutObject(request).url().toString();
    }
}
java
@Service
public class S3Service {
    private final S3Client s3;

    public String generatePresignedUploadUrl(String key, String contentType) {
        var presigner = S3Presigner.builder().region(Region.of(region)).build();
        var request = PutObjectPresignRequest.builder()
            .signatureDuration(Duration.ofHours(1))
            .putObjectRequest(b -> b.bucket(bucket).key(key).contentType(contentType))
            .build();
        return presigner.presignPutObject(request).url().toString();
    }
}

Bucket Policy Patterns

存储桶策略模式

Public read for assets

资源公开可读

json
{
  "Statement": [{
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::my-bucket/public/*"
  }]
}
json
{
  "Statement": [{
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::my-bucket/public/*"
  }]
}

CORS for direct uploads

直接上传的CORS配置

json
{
  "CORSRules": [{
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT", "POST"],
    "AllowedOrigins": ["https://myapp.com"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3600
  }]
}
json
{
  "CORSRules": [{
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT", "POST"],
    "AllowedOrigins": ["https://myapp.com"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3600
  }]
}

Anti-Patterns

反模式

Anti-PatternFix
Proxying large files through serverUse presigned URLs for direct client upload/download
No Content-Type on uploadAlways set ContentType to enable correct browser behavior
Predictable file keysUse UUIDs or hashed paths to prevent enumeration
No lifecycle rulesSet expiration for temp uploads, transition old files to Glacier
Long-lived presigned URLsKeep expiry short (1h for uploads, 15m for downloads)
No multipart for large filesUse multipart upload for files > 100MB
反模式修复方案
通过服务器代理大文件使用预签名URL实现客户端直接上传/下载
上传时未设置Content-Type始终设置ContentType以确保浏览器行为正确
文件键可预测使用UUID或哈希路径防止枚举
未配置生命周期规则为临时上传文件设置过期时间，将旧文件迁移至Glacier
预签名URL有效期过长缩短有效期（上传设为1小时，下载设为15分钟）
大文件未使用分块上传对大于100MB的文件使用分块上传

Quick Troubleshooting

快速故障排查

IssueCauseFix
403 on presigned URLExpired, wrong region, or CORSCheck expiry, region, CORS config
SignatureDoesNotMatchContentType mismatchEnsure client sends same ContentType used in signing
Slow uploadsSingle-part for large filesUse multipart upload with parallel parts
CORS errorsMissing CORS configurationAdd CORSRules to bucket
AccessDenied on public filesMissing bucket policyAdd public read policy for the prefix
问题原因修复方案
预签名URL返回403URL过期、区域错误或CORS配置问题检查有效期、区域和CORS配置
SignatureDoesNotMatch错误ContentType不匹配确保客户端发送的ContentType与签名时使用的一致
上传速度慢大文件使用单块上传使用分块上传并启用并行分块
CORS错误缺少CORS配置为存储桶添加CORSRules
公开文件返回AccessDenied缺少存储桶策略为指定前缀添加公开可读策略

Production Checklist

生产环境检查清单

  • Presigned URLs for all client uploads (never proxy large files)
  • ContentType set on every upload
  • UUID-based keys (prevent enumeration)
  • CORS configured for direct uploads
  • Lifecycle rules for temp/old files
  • Encryption at rest enabled (SSE-S3 or SSE-KMS)
  • Access logging enabled
  • Versioning enabled for critical buckets
  • CloudFront CDN for public assets
  • 所有客户端上传使用预签名URL（绝不代理大文件）
  • 每次上传都设置ContentType
  • 使用UUID作为文件键（防止枚举）
  • 配置CORS以支持直接上传
  • 为临时/旧文件配置生命周期规则
  • 启用静态加密（SSE-S3或SSE-KMS）
  • 启用访问日志
  • 为关键存储桶启用版本控制
  • 为公开资源配置CloudFront CDN