Back to Details

cloud-backup

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

云备份方案

云备份方案

概述

概述

S3/OSS 备份、跨区域复制、生命周期管理技能。
S3/OSS 备份、跨区域复制、生命周期管理技能。

AWS S3 备份

AWS S3 备份

基础操作

基础操作

bash
undefined
bash
undefined

上传文件

上传文件

aws s3 cp backup.tar.gz s3://my-bucket/backups/
aws s3 cp backup.tar.gz s3://my-bucket/backups/

上传目录

上传目录

aws s3 sync /backup s3://my-bucket/backups/
aws s3 sync /backup s3://my-bucket/backups/

下载

下载

aws s3 cp s3://my-bucket/backups/backup.tar.gz ./ aws s3 sync s3://my-bucket/backups/ /restore/
aws s3 cp s3://my-bucket/backups/backup.tar.gz ./ aws s3 sync s3://my-bucket/backups/ /restore/

列出文件

列出文件

aws s3 ls s3://my-bucket/backups/ aws s3 ls s3://my-bucket/backups/ --recursive
undefined
aws s3 ls s3://my-bucket/backups/ aws s3 ls s3://my-bucket/backups/ --recursive
undefined

高级选项

高级选项

bash
undefined
bash
undefined

排除文件

排除文件

aws s3 sync /backup s3://my-bucket/ --exclude "*.log"
aws s3 sync /backup s3://my-bucket/ --exclude "*.log"

存储类型

存储类型

aws s3 cp backup.tar.gz s3://my-bucket/ --storage-class STANDARD_IA aws s3 cp backup.tar.gz s3://my-bucket/ --storage-class GLACIER
aws s3 cp backup.tar.gz s3://my-bucket/ --storage-class STANDARD_IA aws s3 cp backup.tar.gz s3://my-bucket/ --storage-class GLACIER

服务端加密

服务端加密

aws s3 cp backup.tar.gz s3://my-bucket/ --sse AES256 aws s3 cp backup.tar.gz s3://my-bucket/ --sse aws:kms --sse-kms-key-id alias/my-key
aws s3 cp backup.tar.gz s3://my-bucket/ --sse AES256 aws s3 cp backup.tar.gz s3://my-bucket/ --sse aws:kms --sse-kms-key-id alias/my-key

多部分上传

多部分上传

aws s3 cp large-file.tar.gz s3://my-bucket/ --expected-size 10737418240
undefined
aws s3 cp large-file.tar.gz s3://my-bucket/ --expected-size 10737418240
undefined

生命周期策略

生命周期策略

json
{
  "Rules": [
    {
      "ID": "BackupLifecycle",
      "Status": "Enabled",
      "Filter": {"Prefix": "backups/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "STANDARD_IA"},
        {"Days": 90, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 365}
    }
  ]
}
bash
undefined
json
{
  "Rules": [
    {
      "ID": "BackupLifecycle",
      "Status": "Enabled",
      "Filter": {"Prefix": "backups/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "STANDARD_IA"},
        {"Days": 90, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 365}
    }
  ]
}
bash
undefined

应用策略

应用策略

aws s3api put-bucket-lifecycle-configuration
--bucket my-bucket
--lifecycle-configuration file://lifecycle.json
undefined
aws s3api put-bucket-lifecycle-configuration
--bucket my-bucket
--lifecycle-configuration file://lifecycle.json
undefined

跨区域复制

跨区域复制

bash
undefined
bash
undefined

启用版本控制

启用版本控制

aws s3api put-bucket-versioning
--bucket source-bucket
--versioning-configuration Status=Enabled
aws s3api put-bucket-versioning
--bucket source-bucket
--versioning-configuration Status=Enabled

配置复制规则

配置复制规则

aws s3api put-bucket-replication
--bucket source-bucket
--replication-configuration file://replication.json
undefined
aws s3api put-bucket-replication
--bucket source-bucket
--replication-configuration file://replication.json
undefined

阿里云 OSS 备份

阿里云 OSS 备份

基础操作

基础操作

bash
undefined
bash
undefined

上传

上传

ossutil cp backup.tar.gz oss://my-bucket/backups/ ossutil cp -r /backup oss://my-bucket/backups/
ossutil cp backup.tar.gz oss://my-bucket/backups/ ossutil cp -r /backup oss://my-bucket/backups/

下载

下载

ossutil cp oss://my-bucket/backups/backup.tar.gz ./
ossutil cp oss://my-bucket/backups/backup.tar.gz ./

同步

同步

ossutil sync /backup oss://my-bucket/backups/
ossutil sync /backup oss://my-bucket/backups/

列出

列出

ossutil ls oss://my-bucket/backups/
undefined
ossutil ls oss://my-bucket/backups/
undefined

存储类型

存储类型

bash
undefined
bash
undefined

低频访问

低频访问

ossutil cp backup.tar.gz oss://my-bucket/ --meta x-oss-storage-class:IA
ossutil cp backup.tar.gz oss://my-bucket/ --meta x-oss-storage-class:IA

归档

归档

ossutil cp backup.tar.gz oss://my-bucket/ --meta x-oss-storage-class:Archive
undefined
ossutil cp backup.tar.gz oss://my-bucket/ --meta x-oss-storage-class:Archive
undefined

Azure Blob 备份

Azure Blob 备份

基础操作

基础操作

bash
undefined
bash
undefined

上传

上传

az storage blob upload
--account-name myaccount
--container-name backups
--name backup.tar.gz
--file backup.tar.gz
az storage blob upload
--account-name myaccount
--container-name backups
--name backup.tar.gz
--file backup.tar.gz

下载

下载

az storage blob download
--account-name myaccount
--container-name backups
--name backup.tar.gz
--file backup.tar.gz
az storage blob download
--account-name myaccount
--container-name backups
--name backup.tar.gz
--file backup.tar.gz

同步

同步

azcopy sync /backup "https://myaccount.blob.core.windows.net/backups"
undefined
azcopy sync /backup "https://myaccount.blob.core.windows.net/backups"
undefined

rclone 通用工具

rclone 通用工具

配置

配置

bash
undefined
bash
undefined

交互式配置

交互式配置

rclone config
rclone config

配置文件 ~/.config/rclone/rclone.conf

配置文件 ~/.config/rclone/rclone.conf

[s3] type = s3 provider = AWS access_key_id = xxx secret_access_key = xxx region = us-east-1
[oss] type = s3 provider = Alibaba access_key_id = xxx secret_access_key = xxx endpoint = oss-cn-hangzhou.aliyuncs.com
undefined
[s3] type = s3 provider = AWS access_key_id = xxx secret_access_key = xxx region = us-east-1
[oss] type = s3 provider = Alibaba access_key_id = xxx secret_access_key = xxx endpoint = oss-cn-hangzhou.aliyuncs.com
undefined

操作

操作

bash
undefined
bash
undefined

同步

同步

rclone sync /backup s3:my-bucket/backups
rclone sync /backup s3:my-bucket/backups

复制

复制

rclone copy /backup s3:my-bucket/backups
rclone copy /backup s3:my-bucket/backups

加密备份

加密备份

rclone sync /backup crypt:backups
undefined
rclone sync /backup crypt:backups
undefined

常见场景

常见场景

场景 1：自动备份脚本

场景 1：自动备份脚本

bash
#!/bin/bash
DATE=$(date +%Y%m%d)
BACKUP_DIR="/backup"
S3_BUCKET="s3://my-bucket/backups"
bash
#!/bin/bash
DATE=$(date +%Y%m%d)
BACKUP_DIR="/backup"
S3_BUCKET="s3://my-bucket/backups"

创建备份

创建备份

tar -czvf ${BACKUP_DIR}/backup_${DATE}.tar.gz /data
tar -czvf ${BACKUP_DIR}/backup_${DATE}.tar.gz /data

上传到 S3

上传到 S3

aws s3 cp ${BACKUP_DIR}/backup_${DATE}.tar.gz ${S3_BUCKET}/
aws s3 cp ${BACKUP_DIR}/backup_${DATE}.tar.gz ${S3_BUCKET}/

清理本地

清理本地

find ${BACKUP_DIR} -name "backup_*.tar.gz" -mtime +7 -delete
find ${BACKUP_DIR} -name "backup_*.tar.gz" -mtime +7 -delete

验证

验证

aws s3 ls ${S3_BUCKET}/backup_${DATE}.tar.gz
undefined
aws s3 ls ${S3_BUCKET}/backup_${DATE}.tar.gz
undefined

场景 2：数据库云备份

场景 2：数据库云备份

bash
#!/bin/bash
bash
#!/bin/bash

MySQL 备份到 S3

MySQL 备份到 S3

mysqldump -u root -p database | gzip |
aws s3 cp - s3://my-bucket/mysql/backup_$(date +%Y%m%d).sql.gz
mysqldump -u root -p database | gzip |
aws s3 cp - s3://my-bucket/mysql/backup_$(date +%Y%m%d).sql.gz

PostgreSQL 备份到 S3

PostgreSQL 备份到 S3

pg_dump database | gzip |
aws s3 cp - s3://my-bucket/postgres/backup_$(date +%Y%m%d).sql.gz
undefined
pg_dump database | gzip |
aws s3 cp - s3://my-bucket/postgres/backup_$(date +%Y%m%d).sql.gz
undefined

场景 3：增量同步

场景 3：增量同步

bash
#!/bin/bash
bash
#!/bin/bash

使用 aws s3 sync 增量同步

使用 aws s3 sync 增量同步

aws s3 sync /data s3://my-bucket/data/
--exclude "*.tmp"
--delete
aws s3 sync /data s3://my-bucket/data/
--exclude "*.tmp"
--delete

使用 rclone

使用 rclone

rclone sync /data remote:bucket/data --progress
undefined
rclone sync /data remote:bucket/data --progress
undefined

场景 4：灾备复制

场景 4：灾备复制

bash
undefined
bash
undefined

跨区域复制配置

跨区域复制配置

{ "Role": "arn:aws:iam::account:role/replication-role", "Rules": [{ "Status": "Enabled", "Priority": 1, "Filter": {}, "Destination": { "Bucket": "arn:aws:s3:::dest-bucket", "StorageClass": "STANDARD_IA" } }] }
undefined
{ "Role": "arn:aws:iam::account:role/replication-role", "Rules": [{ "Status": "Enabled", "Priority": 1, "Filter": {}, "Destination": { "Bucket": "arn:aws:s3:::dest-bucket", "StorageClass": "STANDARD_IA" } }] }
undefined

成本优化

成本优化

存储类型适用场景成本
Standard频繁访问高
IA月访问中
Glacier年访问低
Deep Archive归档最低
存储类型适用场景成本
Standard频繁访问高
IA月访问中
Glacier年访问低
Deep Archive归档最低

故障排查

故障排查

问题排查方法
上传失败检查权限、网络、大小限制
同步慢使用多线程、分片上传
成本高检查存储类型、生命周期
恢复慢Glacier 需要先解冻
bash
undefined
问题排查方法
上传失败检查权限、网络、大小限制
同步慢使用多线程、分片上传
成本高检查存储类型、生命周期
恢复慢Glacier 需要先解冻
bash
undefined

检查 S3 权限

检查 S3 权限

aws s3api get-bucket-policy --bucket my-bucket
aws s3api get-bucket-policy --bucket my-bucket

检查上传状态

检查上传状态

aws s3api list-multipart-uploads --bucket my-bucket
aws s3api list-multipart-uploads --bucket my-bucket

Glacier 解冻

Glacier 解冻

aws s3api restore-object
--bucket my-bucket
--key backups/archive.tar.gz
--restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Standard"}}'
undefined
aws s3api restore-object
--bucket my-bucket
--key backups/archive.tar.gz
--restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Standard"}}'
undefined