migration-helper

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Convex Migration Helper

Convex 迁移助手

Safely migrate Convex schemas and data when making breaking changes.

安全地在进行破坏性变更时迁移Convex schema和数据。

When to Use

适用场景

Adding new required fields to existing tables
Changing field types or structure
Splitting or merging tables
Renaming fields
Migrating from nested to relational data

为现有表添加新的必填字段
更改字段类型或结构
拆分或合并表
重命名字段
从嵌套数据迁移到关系型数据

Migration Principles

迁移原则

No Automatic Migrations: Convex doesn't automatically migrate data
Additive Changes are Safe: Adding optional fields or new tables is safe
Breaking Changes Need Code: Required fields, type changes need migration code
Zero-Downtime: Write migrations to keep app running during migration

无自动迁移：Convex不会自动迁移数据
增量变更安全：添加可选字段或新表是安全的
破坏性变更需要代码：必填字段、类型变更需要迁移代码
零停机：编写迁移代码以确保迁移期间应用正常运行

Safe Changes (No Migration Needed)

无需迁移的安全变更

Adding Optional Field

添加可选字段

typescript

// Before
users: defineTable({
  name: v.string(),
})

// After - Safe! New field is optional
users: defineTable({
  name: v.string(),
  bio: v.optional(v.string()),
})

typescript

// 之前
users: defineTable({
  name: v.string(),
})

// 之后 - 安全！新增字段为可选
users: defineTable({
  name: v.string(),
  bio: v.optional(v.string()),
})

Adding New Table

添加新表

typescript

// Safe to add completely new tables
posts: defineTable({
  userId: v.id("users"),
  title: v.string(),
}).index("by_user", ["userId"])

typescript

// 完全可以安全添加新表
posts: defineTable({
  userId: v.id("users"),
  title: v.string(),
}).index("by_user", ["userId"])

Adding Index

添加索引

typescript

// Safe to add indexes at any time
users: defineTable({
  name: v.string(),
  email: v.string(),
})
  .index("by_email", ["email"]) // New index

typescript

// 任何时候都可以安全添加索引
users: defineTable({
  name: v.string(),
  email: v.string(),
})
  .index("by_email", ["email"]) // 新增索引

Breaking Changes (Migration Required)

需要迁移的破坏性变更

Adding Required Field

添加必填字段

Problem: Existing documents won't have the new field.

Solution: Add as optional first, backfill data, then make required.

typescript

// Step 1: Add as optional
users: defineTable({
  name: v.string(),
  email: v.optional(v.string()), // Start optional
})

// Step 2: Create migration
import { internalMutation } from "./_generated/server";
import { v } from "convex/values";

export const backfillEmails = internalMutation({
  args: {},
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();

    for (const user of users) {
      if (!user.email) {
        await ctx.db.patch(user._id, {
          email: `user-${user._id}@example.com`, // Default value
        });
      }
    }
  },
});

// Step 3: Run migration via dashboard or CLI
// npx convex run migrations:backfillEmails

// Step 4: Make field required (after all data migrated)
users: defineTable({
  name: v.string(),
  email: v.string(), // Now required
})

问题：现有文档没有这个新字段。

解决方案：先添加为可选字段，回填数据，然后设置为必填。

typescript

// 步骤1：添加为可选字段
users: defineTable({
  name: v.string(),
  email: v.optional(v.string()), // 初始为可选
})

// 步骤2：创建迁移
import { internalMutation } from "./_generated/server";
import { v } from "convex/values";

export const backfillEmails = internalMutation({
  args: {},
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();

    for (const user of users) {
      if (!user.email) {
        await ctx.db.patch(user._id, {
          email: `user-${user._id}@example.com`, // 默认值
        });
      }
    }
  },
});

// 步骤3：通过仪表盘或CLI运行迁移
// npx convex run migrations:backfillEmails

// 步骤4：设置字段为必填（所有数据迁移完成后）
users: defineTable({
  name: v.string(),
  email: v.string(), // 现在为必填
})

Changing Field Type

更改字段类型

Example: Change

tags: v.array(v.string())

to separate table

typescript

// Step 1: Create new structure (additive)
tags: defineTable({
  name: v.string(),
}).index("by_name", ["name"]),

postTags: defineTable({
  postId: v.id("posts"),
  tagId: v.id("tags"),
})
  .index("by_post", ["postId"])
  .index("by_tag", ["tagId"]),

// Keep old field as optional during migration
posts: defineTable({
  title: v.string(),
  tags: v.optional(v.array(v.string())), // Keep temporarily
})

// Step 2: Write migration
export const migrateTags = internalMutation({
  args: { batchSize: v.optional(v.number()) },
  handler: async (ctx, args) => {
    const batchSize = args.batchSize ?? 100;

    const posts = await ctx.db
      .query("posts")
      .filter(q => q.neq(q.field("tags"), undefined))
      .take(batchSize);

    for (const post of posts) {
      if (!post.tags || post.tags.length === 0) {
        await ctx.db.patch(post._id, { tags: undefined });
        continue;
      }

      // Create tags and relationships
      for (const tagName of post.tags) {
        // Get or create tag
        let tag = await ctx.db
          .query("tags")
          .withIndex("by_name", q => q.eq("name", tagName))
          .unique();

        if (!tag) {
          const tagId = await ctx.db.insert("tags", { name: tagName });
          tag = { _id: tagId, name: tagName };
        }

        // Create relationship
        const existing = await ctx.db
          .query("postTags")
          .withIndex("by_post", q => q.eq("postId", post._id))
          .filter(q => q.eq(q.field("tagId"), tag._id))
          .unique();

        if (!existing) {
          await ctx.db.insert("postTags", {
            postId: post._id,
            tagId: tag._id,
          });
        }
      }

      // Remove old field
      await ctx.db.patch(post._id, { tags: undefined });
    }

    return { migrated: posts.length };
  },
});

// Step 3: Run in batches via cron or manually
// Run multiple times until all migrated

// Step 4: Remove old field from schema
posts: defineTable({
  title: v.string(),
  // tags field removed
})

示例：将

tags: v.array(v.string())

改为独立表

typescript

// 步骤1：创建新结构（增量变更）
tags: defineTable({
  name: v.string(),
}).index("by_name", ["name"]),

postTags: defineTable({
  postId: v.id("posts"),
  tagId: v.id("tags"),
})
  .index("by_post", ["postId"])
  .index("by_tag", ["tagId"]),

// 迁移期间保留旧字段为可选
posts: defineTable({
  title: v.string(),
  tags: v.optional(v.array(v.string())), // 临时保留
})

// 步骤2：编写迁移
export const migrateTags = internalMutation({
  args: { batchSize: v.optional(v.number()) },
  handler: async (ctx, args) => {
    const batchSize = args.batchSize ?? 100;

    const posts = await ctx.db
      .query("posts")
      .filter(q => q.neq(q.field("tags"), undefined))
      .take(batchSize);

    for (const post of posts) {
      if (!post.tags || post.tags.length === 0) {
        await ctx.db.patch(post._id, { tags: undefined });
        continue;
      }

      // 创建标签和关联关系
      for (const tagName of post.tags) {
        // 获取或创建标签
        let tag = await ctx.db
          .query("tags")
          .withIndex("by_name", q => q.eq("name", tagName))
          .unique();

        if (!tag) {
          const tagId = await ctx.db.insert("tags", { name: tagName });
          tag = { _id: tagId, name: tagName };
        }

        // 创建关联
        const existing = await ctx.db
          .query("postTags")
          .withIndex("by_post", q => q.eq("postId", post._id))
          .filter(q => q.eq(q.field("tagId"), tag._id))
          .unique();

        if (!existing) {
          await ctx.db.insert("postTags", {
            postId: post._id,
            tagId: tag._id,
          });
        }
      }

      // 删除旧字段
      await ctx.db.patch(post._id, { tags: undefined });
    }

    return { migrated: posts.length };
  },
});

// 步骤3：通过定时任务或手动分批运行
// 多次运行直到所有数据迁移完成

// 步骤4：从schema中移除旧字段
posts: defineTable({
  title: v.string(),
  // 移除tags字段
})

Renaming Field

重命名字段

typescript

// Step 1: Add new field (optional)
users: defineTable({
  name: v.string(),
  displayName: v.optional(v.string()), // New name
})

// Step 2: Copy data
export const renameField = internalMutation({
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();

    for (const user of users) {
      await ctx.db.patch(user._id, {
        displayName: user.name,
      });
    }
  },
});

// Step 3: Update schema (remove old field)
users: defineTable({
  displayName: v.string(),
})

// Step 4: Update all code to use new field name

typescript

// 步骤1：添加新字段（可选）
users: defineTable({
  name: v.string(),
  displayName: v.optional(v.string()), // 新名称
})

// 步骤2：复制数据
export const renameField = internalMutation({
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();

    for (const user of users) {
      await ctx.db.patch(user._id, {
        displayName: user.name,
      });
    }
  },
});

// 步骤3：更新schema（移除旧字段）
users: defineTable({
  displayName: v.string(),
})

// 步骤4：更新所有使用旧字段名的代码

Migration Patterns

迁移模式

Batch Processing

批量处理

For large tables, process in batches:

typescript

export const migrateBatch = internalMutation({
  args: {
    cursor: v.optional(v.string()),
    batchSize: v.number(),
  },
  handler: async (ctx, args) => {
    const batchSize = args.batchSize;
    let query = ctx.db.query("largeTable");

    // Use cursor for pagination if needed
    const items = await query.take(batchSize);

    for (const item of items) {
      await ctx.db.patch(item._id, {
        // migration logic
      });
    }

    return {
      processed: items.length,
      hasMore: items.length === batchSize,
    };
  },
});

对于大型表，分批处理：

typescript

export const migrateBatch = internalMutation({
  args: {
    cursor: v.optional(v.string()),
    batchSize: v.number(),
  },
  handler: async (ctx, args) => {
    const batchSize = args.batchSize;
    let query = ctx.db.query("largeTable");

    // 如有需要，使用游标进行分页
    const items = await query.take(batchSize);

    for (const item of items) {
      await ctx.db.patch(item._id, {
        // 迁移逻辑
      });
    }

    return {
      processed: items.length,
      hasMore: items.length === batchSize,
    };
  },
});

Scheduled Migration

定时迁移

Use cron jobs for gradual migration:

typescript

// convex/crons.ts
import { cronJobs } from "convex/server";
import { internal } from "./_generated/api";

const crons = cronJobs();

crons.interval(
  "migrate-batch",
  { minutes: 5 }, // Every 5 minutes
  internal.migrations.migrateBatch,
  { batchSize: 100 }
);

export default crons;

使用定时任务进行渐进式迁移：

typescript

// convex/crons.ts
import { cronJobs } from "convex/server";
import { internal } from "./_generated/api";

const crons = cronJobs();

crons.interval(
  "migrate-batch",
  { minutes: 5 }, // 每5分钟一次
  internal.migrations.migrateBatch,
  { batchSize: 100 }
);

export default crons;

Dual-Write Pattern

双写模式

For zero-downtime migrations:

typescript

// Write to both old and new structure during transition
export const createPost = mutation({
  args: { title: v.string(), tags: v.array(v.string()) },
  handler: async (ctx, args) => {
    const user = await getCurrentUser(ctx);

    // Create post
    const postId = await ctx.db.insert("posts", {
      userId: user._id,
      title: args.title,
      // Keep writing old field during migration
      tags: args.tags,
    });

    // ALSO write to new structure
    for (const tagName of args.tags) {
      let tag = await ctx.db
        .query("tags")
        .withIndex("by_name", q => q.eq("name", tagName))
        .unique();

      if (!tag) {
        const tagId = await ctx.db.insert("tags", { name: tagName });
        tag = { _id: tagId };
      }

      await ctx.db.insert("postTags", {
        postId,
        tagId: tag._id,
      });
    }

    return postId;
  },
});

// After migration complete, remove old writes

实现零停机迁移：

typescript

// 过渡期间同时写入旧结构和新结构
export const createPost = mutation({
  args: { title: v.string(), tags: v.array(v.string()) },
  handler: async (ctx, args) => {
    const user = await getCurrentUser(ctx);

    // 创建文章
    const postId = await ctx.db.insert("posts", {
      userId: user._id,
      title: args.title,
      // 迁移期间继续写入旧字段
      tags: args.tags,
    });

    // 同时写入新结构
    for (const tagName of args.tags) {
      let tag = await ctx.db
        .query("tags")
        .withIndex("by_name", q => q.eq("name", tagName))
        .unique();

      if (!tag) {
        const tagId = await ctx.db.insert("tags", { name: tagName });
        tag = { _id: tagId };
      }

      await ctx.db.insert("postTags", {
        postId,
        tagId: tag._id,
      });
    }

    return postId;
  },
});

// 迁移完成后，移除旧的写入逻辑

Testing Migrations

测试迁移

Verify Migration Success

验证迁移成功

typescript

export const verifyMigration = query({
  args: {},
  handler: async (ctx) => {
    const total = (await ctx.db.query("users").collect()).length;
    const migrated = (await ctx.db
      .query("users")
      .filter(q => q.neq(q.field("newField"), undefined))
      .collect()
    ).length;

    return {
      total,
      migrated,
      remaining: total - migrated,
      percentComplete: (migrated / total) * 100,
    };
  },
});

typescript

export const verifyMigration = query({
  args: {},
  handler: async (ctx) => {
    const total = (await ctx.db.query("users").collect()).length;
    const migrated = (await ctx.db
      .query("users")
      .filter(q => q.neq(q.field("newField"), undefined))
      .collect()
    ).length;

    return {
      total,
      migrated,
      remaining: total - migrated,
      percentComplete: (migrated / total) * 100,
    };
  },
});

Migration Checklist

迁移检查清单

Common Pitfalls

常见陷阱

Don't make field required immediately: Always add as optional first
Don't migrate in a single transaction: Batch large migrations
Don't forget to update queries: Update all code using old field
Don't delete old field too soon: Wait until all data migrated
Test thoroughly: Verify migration on dev environment first

不要立即将字段设为必填：始终先添加为可选字段
不要在单个事务中迁移：大型迁移需分批处理
不要忘记更新查询：更新所有使用旧字段的代码
不要过早删除旧字段：等待所有数据迁移完成后再删除
彻底测试：先在开发环境验证迁移

Example: Complete Migration Flow

示例：完整迁移流程

typescript

// 1. Current schema
export default defineSchema({
  users: defineTable({
    name: v.string(),
  }),
});

// 2. Add optional field
export default defineSchema({
  users: defineTable({
    name: v.string(),
    role: v.optional(v.union(
      v.literal("user"),
      v.literal("admin")
    )),
  }),
});

// 3. Migration function
export const addDefaultRoles = internalMutation({
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();
    for (const user of users) {
      if (!user.role) {
        await ctx.db.patch(user._id, { role: "user" });
      }
    }
  },
});

// 4. Run migration: npx convex run migrations:addDefaultRoles

// 5. Verify: Check all users have role

// 6. Make required
export default defineSchema({
  users: defineTable({
    name: v.string(),
    role: v.union(
      v.literal("user"),
      v.literal("admin")
    ),
  }),
});

typescript

// 1. 当前schema
export default defineSchema({
  users: defineTable({
    name: v.string(),
  }),
});

// 2. 添加可选字段
export default defineSchema({
  users: defineTable({
    name: v.string(),
    role: v.optional(v.union(
      v.literal("user"),
      v.literal("admin")
    )),
  }),
});

// 3. 迁移函数
export const addDefaultRoles = internalMutation({
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();
    for (const user of users) {
      if (!user.role) {
        await ctx.db.patch(user._id, { role: "user" });
      }
    }
  },
});

// 4. 运行迁移：npx convex run migrations:addDefaultRoles

// 5. 验证：检查所有用户都已设置role

// 6. 设置为必填
export default defineSchema({
  users: defineTable({
    name: v.string(),
    role: v.union(
      v.literal("user"),
      v.literal("admin")
    ),
  }),
});