celery-patterns

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Celery Patterns for Django

Django 中的 Celery 任务模式

Core Philosophy

核心原则

Idempotent tasks: Running a task twice must produce the same result
Pass IDs, not objects: Arguments must be JSON-serializable
Always handle failures: Log errors, never swallow exceptions silently
Proper retry strategies: Use exponential backoff for external services

幂等任务：任务执行两次必须产生相同结果
传递ID而非对象：参数必须可JSON序列化
始终处理失败场景：记录错误，绝不能静默忽略异常
合理的重试策略：针对外部服务使用指数退避算法

Task Design

任务设计

Structure

结构

Place tasks in
```
apps/<app>/tasks.py
```
Use
```
@shared_task
```
for reusable tasks across projects
Use
```
bind=True
```
when needing access to task instance (retries, task ID, state updates)
Add type hints to task signatures
Log task start, completion, and failures

将任务放置在
```
apps/<app>/tasks.py
```
中
跨项目可复用的任务使用
```
@shared_task
```
装饰器
当需要访问任务实例（重试、任务ID、状态更新）时，设置
```
bind=True
```
为任务签名添加类型提示
记录任务的启动、完成和失败情况

Arguments

参数

Pass model primary keys, not model instances
Keep arguments simple and serializable (str, int, dict, list)
Validate that referenced objects exist at task start

传递模型主键而非模型实例
保持参数简洁且可序列化（字符串、整数、字典、列表）
在任务启动时验证引用的对象是否存在

Retry Strategies

重试策略

When to Use Each Approach

各策略适用场景

Fixed delay: Internal operations with predictable recovery (database locks)
Exponential backoff: External APIs that may rate-limit or have variable recovery
No retry: Validation errors, business logic failures, permanent errors

固定延迟：适用于恢复可预测的内部操作（如数据库锁）
指数退避：适用于可能限流或恢复时间不确定的外部API
不重试：适用于验证错误、业务逻辑失败、永久性错误

Configuration

配置

Set
```
max_retries
```
based on acceptable total wait time
Use
```
retry_jitter=True
```
to prevent thundering herd
Set
```
retry_backoff_max
```
to cap maximum wait time
Use
```
autoretry_for
```
tuple for automatic retry on specific exceptions

根据可接受的总等待时间设置
```
max_retries
```
设置
```
retry_jitter=True
```
以避免惊群效应
设置
```
retry_backoff_max
```
来限制最长等待时间
使用
```
autoretry_for
```
元组针对特定异常自动重试

Idempotency Patterns

幂等性模式

Check-Before-Process

先检查再处理

Query current state before processing; skip if already complete

处理前查询当前状态；若已完成则跳过

Status Field Tracking

状态字段追踪

Use status transitions (pending → processing → complete/failed) with

select_for_update()

for race condition safety

使用状态流转（pending → processing → complete/failed）并结合

select_for_update()

避免竞态条件

Unique Constraints

唯一约束

Use database constraints to prevent duplicate processing

使用数据库约束防止重复处理

Periodic Tasks (Beat)

周期性任务（Beat）

Configure schedules in
```
config/celery.py
```
using
```
beat_schedule
```
Use
```
crontab()
```
for time-based schedules
Use float values for interval-based schedules (seconds)
Keep periodic tasks lightweight; spawn subtasks for heavy work

在
```
config/celery.py
```
中使用
```
beat_schedule
```
配置任务调度
基于时间的调度使用
```
crontab()
```
基于间隔的调度使用浮点数（单位：秒）
保持周期性任务轻量化；繁重工作通过生成子任务处理

Anti-Patterns to Avoid

需避免的反模式

Passing model instances instead of IDs
Non-idempotent operations (incrementing without checks)
Silent exception handling (bare
```
except: pass
```
)
Missing logging for task lifecycle
Long-running tasks without progress updates
Retry on permanent failures (validation, business logic errors)

传递模型实例而非ID
非幂等操作（如无检查的递增操作）
静默异常处理（如空的
```
except: pass
```
）
缺少任务生命周期的日志记录
无进度更新的长时间运行任务
对永久性错误（验证、业务逻辑错误）进行重试

Commands

命令

Worker:
```
uv run celery -A config worker -l info
```
Beat:
```
uv run celery -A config beat -l info
```
Monitoring:
```
uv run celery -A config flower
```

启动Worker：
```
uv run celery -A config worker -l info
```
启动Beat：
```
uv run celery -A config beat -l info
```
监控命令：
```
uv run celery -A config flower
```