Back to Details

object-storage

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Object Storage

对象存储

Configure and manage object storage solutions including AWS S3, MinIO (self-hosted), and compatible providers. Covers CLI operations, bucket policies, lifecycle rules, versioning, encryption, and the MinIO client (mc).
配置并管理对象存储解决方案，包括AWS S3、MinIO（自建）及兼容服务商。内容涵盖CLI操作、存储桶策略、生命周期规则、版本控制、加密以及MinIO客户端（mc）。

When to Use

适用场景

  • Storing application assets, backups, logs, or media files
  • Setting up an S3-compatible object store on-premises with MinIO
  • Configuring lifecycle rules to transition or expire objects automatically
  • Implementing access control with bucket policies and IAM
  • Syncing data between local filesystems and object storage
  • Serving static content from S3 or MinIO
  • 存储应用资产、备份文件、日志或媒体文件
  • 通过MinIO在本地部署兼容S3的对象存储
  • 配置生命周期规则，自动转换或过期对象
  • 通过存储桶策略与IAM实现访问控制
  • 在本地文件系统与对象存储之间同步数据
  • 从S3或MinIO提供静态内容服务

Prerequisites

前置条件

  • AWS CLI v2 installed and configured (
    aws configure
    ) for S3 operations
  • Docker installed for MinIO self-hosted setup
  • MinIO client (
    mc
    ) installed for MinIO management
  • IAM credentials with appropriate S3 permissions
  • Network access to the object storage endpoint
  • 已安装并配置AWS CLI v2（执行
    aws configure
    ），用于S3操作
  • 已安装Docker，用于MinIO自建部署
  • 已安装MinIO客户端（
    mc
    ），用于MinIO管理
  • 拥有具备相应S3权限的IAM凭证
  • 能够访问对象存储端点的网络权限

AWS S3 CLI Operations

AWS S3 CLI操作

Bucket Management

存储桶管理

bash
undefined
bash
undefined

Create a new bucket

创建新存储桶

aws s3 mb s3://my-app-assets-prod
aws s3 mb s3://my-app-assets-prod

Create a bucket in a specific region

在指定区域创建存储桶

aws s3 mb s3://my-app-assets-eu --region eu-west-1
aws s3 mb s3://my-app-assets-eu --region eu-west-1

List all buckets

列出所有存储桶

aws s3 ls
aws s3 ls

List objects in a bucket (with sizes)

列出存储桶中的对象（包含大小）

aws s3 ls s3://my-app-assets-prod --recursive --human-readable --summarize
aws s3 ls s3://my-app-assets-prod --recursive --human-readable --summarize

Delete an empty bucket

删除空存储桶

aws s3 rb s3://my-old-bucket
aws s3 rb s3://my-old-bucket

Delete a bucket and ALL its contents (destructive)

删除存储桶及其所有内容（破坏性操作）

aws s3 rb s3://my-old-bucket --force
undefined
aws s3 rb s3://my-old-bucket --force
undefined

Upload and Download

上传与下载

bash
undefined
bash
undefined

Upload a single file

上传单个文件

aws s3 cp ./report.pdf s3://my-app-assets-prod/reports/
aws s3 cp ./report.pdf s3://my-app-assets-prod/reports/

Upload with a specific storage class

使用指定存储类别上传

aws s3 cp ./archive.tar.gz s3://my-app-assets-prod/archives/ --storage-class GLACIER
aws s3 cp ./archive.tar.gz s3://my-app-assets-prod/archives/ --storage-class GLACIER

Upload with server-side encryption (AES-256)

使用服务器端加密（AES-256）上传

aws s3 cp ./sensitive.dat s3://my-app-assets-prod/data/ --sse AES256
aws s3 cp ./sensitive.dat s3://my-app-assets-prod/data/ --sse AES256

Download a file

下载文件

aws s3 cp s3://my-app-assets-prod/reports/report.pdf ./downloads/
aws s3 cp s3://my-app-assets-prod/reports/report.pdf ./downloads/

Sync a local directory to S3 (upload only changed files)

将本地目录同步到S3（仅上传变更文件）

aws s3 sync ./build/ s3://my-app-assets-prod/static/ --delete
aws s3 sync ./build/ s3://my-app-assets-prod/static/ --delete

Sync from S3 to local

从S3同步到本地

aws s3 sync s3://my-app-assets-prod/static/ ./local-copy/
aws s3 sync s3://my-app-assets-prod/static/ ./local-copy/

Sync with exclusion patterns

使用排除规则同步

aws s3 sync ./logs/ s3://my-app-logs/ --exclude ".tmp" --exclude ".git/"
aws s3 sync ./logs/ s3://my-app-logs/ --exclude ".tmp" --exclude ".git/"

Copy between buckets

在存储桶之间复制

aws s3 sync s3://source-bucket/ s3://destination-bucket/ --source-region us-east-1 --region eu-west-1
aws s3 sync s3://source-bucket/ s3://destination-bucket/ --source-region us-east-1 --region eu-west-1

Generate a pre-signed URL (temporary access, 1 hour)

生成预签名URL（临时访问，有效期1小时）

aws s3 presign s3://my-app-assets-prod/reports/report.pdf --expires-in 3600
aws s3 presign s3://my-app-assets-prod/reports/report.pdf --expires-in 3600

Recursive delete of a prefix

递归删除指定前缀下的内容

aws s3 rm s3://my-app-assets-prod/old-data/ --recursive
undefined
aws s3 rm s3://my-app-assets-prod/old-data/ --recursive
undefined

Versioning

版本控制

bash
undefined
bash
undefined

Enable versioning on a bucket

为存储桶启用版本控制

aws s3api put-bucket-versioning
--bucket my-app-assets-prod
--versioning-configuration Status=Enabled
aws s3api put-bucket-versioning
--bucket my-app-assets-prod
--versioning-configuration Status=Enabled

Check versioning status

检查版本控制状态

aws s3api get-bucket-versioning --bucket my-app-assets-prod
aws s3api get-bucket-versioning --bucket my-app-assets-prod

List object versions

列出对象版本

aws s3api list-object-versions --bucket my-app-assets-prod --prefix reports/
aws s3api list-object-versions --bucket my-app-assets-prod --prefix reports/

Restore a previous version (copy old version to current)

恢复历史版本（将旧版本复制为当前版本）

aws s3api copy-object
--bucket my-app-assets-prod
--copy-source "my-app-assets-prod/reports/report.pdf?versionId=abc123"
--key reports/report.pdf
aws s3api copy-object
--bucket my-app-assets-prod
--copy-source "my-app-assets-prod/reports/report.pdf?versionId=abc123"
--key reports/report.pdf

Delete a specific version permanently

永久删除指定版本

aws s3api delete-object
--bucket my-app-assets-prod
--key reports/old-report.pdf
--version-id abc123
undefined
aws s3api delete-object
--bucket my-app-assets-prod
--key reports/old-report.pdf
--version-id abc123
undefined

Bucket Policies

存储桶策略

bash
undefined
bash
undefined

Apply a bucket policy from a JSON file

从JSON文件应用存储桶策略

aws s3api put-bucket-policy --bucket my-app-assets-prod --policy file://policy.json

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadForStaticSite",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/static/*"
    },
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "AES256"
        }
      }
    },
    {
      "Sid": "RestrictToVPC",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-app-assets-prod",
        "arn:aws:s3:::my-app-assets-prod/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-abc123"
        }
      }
    }
  ]
}
aws s3api put-bucket-policy --bucket my-app-assets-prod --policy file://policy.json

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadForStaticSite",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/static/*"
    },
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "AES256"
        }
      }
    },
    {
      "Sid": "RestrictToVPC",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-app-assets-prod",
        "arn:aws:s3:::my-app-assets-prod/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-abc123"
        }
      }
    }
  ]
}

Lifecycle Rules

生命周期规则

bash
undefined
bash
undefined

Apply lifecycle configuration

应用生命周期配置

aws s3api put-bucket-lifecycle-configuration
--bucket my-app-assets-prod
--lifecycle-configuration file://lifecycle.json

```json
{
  "Rules": [
    {
      "ID": "TransitionLogsToIA",
      "Filter": { "Prefix": "logs/" },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    },
    {
      "ID": "CleanupIncompleteUploads",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    },
    {
      "ID": "ExpireOldVersions",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 30
      }
    }
  ]
}
bash
undefined
aws s3api put-bucket-lifecycle-configuration
--bucket my-app-assets-prod
--lifecycle-configuration file://lifecycle.json

```json
{
  "Rules": [
    {
      "ID": "TransitionLogsToIA",
      "Filter": { "Prefix": "logs/" },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    },
    {
      "ID": "CleanupIncompleteUploads",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    },
    {
      "ID": "ExpireOldVersions",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 30
      }
    }
  ]
}
bash
undefined

View current lifecycle rules

查看当前生命周期规则

aws s3api get-bucket-lifecycle-configuration --bucket my-app-assets-prod
aws s3api get-bucket-lifecycle-configuration --bucket my-app-assets-prod

Enable S3 access logging

启用S3访问日志

aws s3api put-bucket-logging --bucket my-app-assets-prod --bucket-logging-status '{ "LoggingEnabled": { "TargetBucket": "my-app-logs", "TargetPrefix": "s3-access-logs/" } }'
undefined
aws s3api put-bucket-logging --bucket my-app-assets-prod --bucket-logging-status '{ "LoggingEnabled": { "TargetBucket": "my-app-logs", "TargetPrefix": "s3-access-logs/" } }'
undefined

MinIO Self-Hosted Setup

MinIO自建部署

Docker Deployment

Docker部署

bash
undefined
bash
undefined

Single-node MinIO with persistent storage

单节点MinIO（带持久化存储）

docker run -d
--name minio
--restart unless-stopped
-p 9000:9000
-p 9001:9001
-e MINIO_ROOT_USER=minioadmin
-e MINIO_ROOT_PASSWORD=minio-secret-key-change-me
-v /data/minio:/data
minio/minio server /data --console-address ":9001"
undefined
docker run -d
--name minio
--restart unless-stopped
-p 9000:9000
-p 9001:9001
-e MINIO_ROOT_USER=minioadmin
-e MINIO_ROOT_PASSWORD=minio-secret-key-change-me
-v /data/minio:/data
minio/minio server /data --console-address ":9001"
undefined

Docker Compose (Multi-Drive)

Docker Compose（多磁盘）

yaml
undefined
yaml
undefined

docker-compose.yml

docker-compose.yml

version: "3.8" services: minio: image: minio/minio:latest command: server /data{1...4} --console-address ":9001" environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: minio-secret-key-change-me MINIO_BROWSER_REDIRECT_URL: https://minio-console.example.com ports: - "9000:9000" - "9001:9001" volumes: - minio-data1:/data1 - minio-data2:/data2 - minio-data3:/data3 - minio-data4:/data4 healthcheck: test: ["CMD", "mc", "ready", "local"] interval: 30s timeout: 10s retries: 3 restart: unless-stopped
volumes: minio-data1: minio-data2: minio-data3: minio-data4:

```bash
version: "3.8" services: minio: image: minio/minio:latest command: server /data{1...4} --console-address ":9001" environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: minio-secret-key-change-me MINIO_BROWSER_REDIRECT_URL: https://minio-console.example.com ports: - "9000:9000" - "9001:9001" volumes: - minio-data1:/data1 - minio-data2:/data2 - minio-data3:/data3 - minio-data4:/data4 healthcheck: test: ["CMD", "mc", "ready", "local"] interval: 30s timeout: 10s retries: 3 restart: unless-stopped
volumes: minio-data1: minio-data2: minio-data3: minio-data4:

```bash

Start the stack

启动栈

docker compose up -d
docker compose up -d

Check health

检查健康状态

docker compose ps curl -s http://localhost:9000/minio/health/live
undefined
docker compose ps curl -s http://localhost:9000/minio/health/live
undefined

MinIO Client (mc) Commands

MinIO客户端（mc）命令

bash
undefined
bash
undefined

Install mc

安装mc

curl -O https://dl.min.io/client/mc/release/linux-amd64/mc chmod +x mc && mv mc /usr/local/bin/
curl -O https://dl.min.io/client/mc/release/linux-amd64/mc chmod +x mc && mv mc /usr/local/bin/

Configure an alias for the MinIO server

配置MinIO服务器别名

mc alias set myminio http://localhost:9000 minioadmin minio-secret-key-change-me
mc alias set myminio http://localhost:9000 minioadmin minio-secret-key-change-me

Configure an alias for AWS S3

配置AWS S3别名

mc alias set aws https://s3.amazonaws.com AKIAEXAMPLE SECRETKEYEXAMPLE
mc alias set aws https://s3.amazonaws.com AKIAEXAMPLE SECRETKEYEXAMPLE

Bucket operations

存储桶操作

mc mb myminio/app-data mc mb myminio/backups mc ls myminio/
mc mb myminio/app-data mc mb myminio/backups mc ls myminio/

Upload and download

上传与下载

mc cp ./backup.tar.gz myminio/backups/ mc cp myminio/backups/backup.tar.gz ./restore/
mc cp ./backup.tar.gz myminio/backups/ mc cp myminio/backups/backup.tar.gz ./restore/

Sync a directory (mirror)

同步目录（镜像）

mc mirror ./static/ myminio/app-data/static/ mc mirror --watch ./static/ myminio/app-data/static/ # Continuous sync
mc mirror ./static/ myminio/app-data/static/ mc mirror --watch ./static/ myminio/app-data/static/ # 持续同步

Set bucket policy (download = public read)

设置存储桶策略（download = 公开读取）

mc anonymous set download myminio/app-data/static
mc anonymous set download myminio/app-data/static

Set a specific policy from JSON

从JSON文件设置特定策略

mc anonymous set-json policy.json myminio/app-data
mc anonymous set-json policy.json myminio/app-data

Enable versioning

启用版本控制

mc version enable myminio/app-data
mc version enable myminio/app-data

Set lifecycle rule: expire objects in tmp/ after 7 days

设置生命周期规则：tmp/目录下的对象7天后过期

mc ilm rule add --expiry-days 7 --prefix "tmp/" myminio/app-data
mc ilm rule add --expiry-days 7 --prefix "tmp/" myminio/app-data

List lifecycle rules

列出生命周期规则

mc ilm rule ls myminio/app-data
mc ilm rule ls myminio/app-data

Create a service account (for applications)

创建服务账号（供应用使用）

mc admin user svcacct add myminio minioadmin --access-key myapp-key --secret-key myapp-secret
mc admin user svcacct add myminio minioadmin --access-key myapp-key --secret-key myapp-secret

View server info and disk usage

查看服务器信息与磁盘使用情况

mc admin info myminio
mc admin info myminio

Check bucket disk usage

检查存储桶磁盘使用情况

mc du myminio/app-data
mc du myminio/app-data

Set a notification target (webhook on object creation)

设置通知目标（对象创建时触发webhook）

mc event add myminio/app-data arn:minio:sqs::myqueue:webhook --event put mc event ls myminio/app-data
undefined
mc event add myminio/app-data arn:minio:sqs::myqueue:webhook --event put mc event ls myminio/app-data
undefined

Troubleshooting

故障排查

SymptomDiagnostic CommandCommon Fix
Access Denied on S3
aws s3api get-bucket-policy --bucket name
Check IAM policy, bucket policy, and block public access settings
Slow uploads
aws s3 cp --debug
Use multipart: 
aws configure set s3.multipart_threshold 64MB
403 on pre-signed URLCheck clock skew, URL expirySync system clock with NTP; regenerate URL
MinIO unhealthy
mc admin info myminio
Check disk space, container logs, port availability
Lifecycle rules not applying
aws s3api get-bucket-lifecycle-configuration
Rules run once per day; check Filter prefix matches
Objects not versioned
aws s3api get-bucket-versioning
Enable versioning; it does not apply retroactively
mc: connection refused
mc alias ls
Verify endpoint URL, port, and credentials
Large sync is slowMonitor with 
mc mirror --watch
Use 
--multi-thread
flag, increase bandwidth
症状诊断命令常见修复方法
S3访问被拒绝
aws s3api get-bucket-policy --bucket name
检查IAM策略、存储桶策略及阻止公开访问设置
上传速度缓慢
aws s3 cp --debug
使用分块上传：
aws configure set s3.multipart_threshold 64MB
预签名URL返回403检查时钟偏差、URL有效期同步系统时钟至NTP；重新生成URL
MinIO状态异常
mc admin info myminio
检查磁盘空间、容器日志、端口可用性
生命周期规则未生效
aws s3api get-bucket-lifecycle-configuration
规则每日运行一次；检查Filter前缀是否匹配
对象未被版本控制
aws s3api get-bucket-versioning
启用版本控制；版本控制不会回溯生效
mc: 连接被拒绝
mc alias ls
验证端点URL、端口及凭证
大文件同步缓慢使用
mc mirror --watch
监控		使用
--multi-thread
参数，提升带宽

Related Skills

相关技能

  • block-storage
    -- Underlying disk storage for MinIO data volumes
  • backup-recovery
    -- Using S3/MinIO as a backup destination with restic
  • nfs-storage
    -- Alternative shared storage for file-level access
  • linux-administration
    -- Server setup and maintenance for MinIO hosts
  • block-storage
    -- MinIO数据卷的底层磁盘存储
  • backup-recovery
    -- 使用S3/MinIO作为restic的备份目标
  • nfs-storage
    -- 用于文件级访问的替代共享存储
  • linux-administration
    -- MinIO主机的服务器搭建与维护