cloudflare-vectorize
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseCloudflare Vectorize
Cloudflare Vectorize
Complete implementation guide for Cloudflare Vectorize - a globally distributed vector database for building semantic search, RAG (Retrieval Augmented Generation), and AI-powered applications with Cloudflare Workers.
Status: Production Ready ✅
Last Updated: 2025-10-21
Dependencies: cloudflare-worker-base (for Worker setup), cloudflare-workers-ai (for embeddings)
Latest Versions: wrangler@4.43.0, @cloudflare/workers-types@4.20251014.0
Token Savings: ~65%
Errors Prevented: 8
Dev Time Saved: ~3 hours
这是Cloudflare Vectorize的完整实现指南,它是一个全球分布式向量数据库,可用于基于Cloudflare Workers构建语义搜索、RAG(检索增强生成)和AI驱动的应用。
状态:已就绪可用于生产 ✅
最后更新:2025-10-21
依赖项:cloudflare-worker-base(用于Worker搭建)、cloudflare-workers-ai(用于生成嵌入向量)
最新版本:wrangler@4.43.0, @cloudflare/workers-types@4.20251014.0
Token节省率:约65%
避免的错误数:8
节省的开发时间:约3小时
What This Skill Provides
本技能提供的功能
Core Capabilities
核心能力
- ✅ Index Management: Create, configure, and manage vector indexes
- ✅ Vector Operations: Insert, upsert, query, delete, and list vectors
- ✅ Metadata Filtering: Advanced filtering with 10 metadata indexes per index
- ✅ Semantic Search: Find similar vectors using cosine, euclidean, or dot-product metrics
- ✅ RAG Patterns: Complete retrieval-augmented generation workflows
- ✅ Workers AI Integration: Native embedding generation with @cf/baai/bge-base-en-v1.5
- ✅ OpenAI Integration: Support for text-embedding-3-small/large models
- ✅ Document Processing: Text chunking and batch ingestion pipelines
- ✅ 索引管理:创建、配置和管理向量索引
- ✅ 向量操作:插入、更新插入、查询、删除和列出向量
- ✅ 元数据过滤:每个索引支持10个元数据索引的高级过滤
- ✅ 语义搜索:使用余弦、欧几里得或点积相似度指标查找相似向量
- ✅ RAG模式:完整的检索增强生成工作流
- ✅ Workers AI集成:与@cf/baai/bge-base-en-v1.5原生集成实现嵌入向量生成
- ✅ OpenAI集成:支持text-embedding-3-small/large模型
- ✅ 文档处理:文本分块和批量导入流水线
Templates Included
包含的模板
- basic-search.ts - Simple vector search with Workers AI
- rag-chat.ts - Full RAG chatbot with context retrieval
- document-ingestion.ts - Document chunking and embedding pipeline
- metadata-filtering.ts - Advanced filtering patterns
- basic-search.ts - 基于Workers AI的简单向量搜索
- rag-chat.ts - 带上下文检索的完整RAG聊天机器人
- document-ingestion.ts - 文档分块与嵌入向量流水线
- metadata-filtering.ts - 高级过滤模式
⚠️ Vectorize V2 Breaking Changes (September 2024)
⚠️ Vectorize V2重大变更(2024年9月)
IMPORTANT: Vectorize V2 became GA in September 2024 with significant breaking changes.
重要提示:Vectorize V2已于2024年9月正式发布,包含多项重大破坏性变更。
What Changed in V2
V2中的变更
Performance Improvements:
- Index capacity: 200,000 → 5 million vectors per index
- Query latency: 549ms → 31ms median (18× faster)
- TopK limit: 20 → 100 results per query
- Scale limits: 100 → 50,000 indexes per account
- Namespace limits: 100 → 50,000 namespaces per index
Breaking API Changes:
-
Async Mutations - All mutations now asynchronous:typescript
// V2: Returns mutationId const result = await env.VECTORIZE_INDEX.insert(vectors); console.log(result.mutationId); // "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" // Vector inserts/deletes may take a few seconds to be reflected -
returnMetadata Parameter - Boolean → String enum:typescript
// ❌ V1 (deprecated) { returnMetadata: true } // ✅ V2 (required) { returnMetadata: 'all' | 'indexed' | 'none' } -
Metadata Indexes Required Before Insert:
- V2 requires metadata indexes created BEFORE vectors inserted
- Vectors added before metadata index won't be indexed
- Must re-upsert vectors after creating metadata index
V1 Deprecation Timeline:
- December 2024: Can no longer create V1 indexes
- Existing V1 indexes: Continue to work (other operations unaffected)
- Migration: Use flag for V1 operations
wrangler vectorize --deprecated-v1
Wrangler Version Required:
- Minimum: wrangler@3.71.0 for V2 commands
- Recommended: wrangler@4.43.0+ (latest)
性能提升:
- 索引容量:从20万 → 每个索引500万向量
- 查询延迟:从549ms → 中位数31ms(快18倍)
- TopK限制:从20 → 每次查询100条结果
- 规模限制:从100 → 每个账户5万个索引
- 命名空间限制:从100 → 每个索引5万个命名空间
破坏性API变更:
-
异步突变 - 所有突变操作现在均为异步:typescript
// V2:返回mutationId const result = await env.VECTORIZE_INDEX.insert(vectors); console.log(result.mutationId); // "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" // 向量插入/删除可能需要几秒才能生效 -
returnMetadata参数 - 从布尔值改为字符串枚举:typescript
// ❌ V1(已弃用) { returnMetadata: true } // ✅ V2(必填) { returnMetadata: 'all' | 'indexed' | 'none' } -
插入向量前需创建元数据索引:
- V2要求在插入向量之前创建元数据索引
- 在元数据索引创建前添加的向量不会被索引
- 创建元数据索引后必须重新更新插入向量
V1弃用时间线:
- 2024年12月:无法再创建V1索引
- 现有V1索引:可继续使用(其他操作不受影响)
- 迁移:使用标志执行V1操作
wrangler vectorize --deprecated-v1
所需Wrangler版本:
- 最低要求:wrangler@3.71.0(支持V2命令)
- 推荐版本:wrangler@4.43.0+(最新版本)
Check Mutation Status
检查突变状态
typescript
// Get index info to check last mutation processed
const info = await env.VECTORIZE_INDEX.describe();
console.log(info.mutationId); // Last mutation ID
console.log(info.processedUpToMutation); // Last processed timestamptypescript
// 获取索引信息以检查最后处理的突变
const info = await env.VECTORIZE_INDEX.describe();
console.log(info.mutationId); // 最后一个突变ID
console.log(info.processedUpToMutation); // 最后处理的时间戳Critical Setup Rules
关键设置规则
⚠️ MUST DO BEFORE INSERTING VECTORS
⚠️ 插入向量前必须完成以下操作
bash
undefinedbash
undefined1. Create the index with FIXED dimensions and metric
1. 创建具有固定维度和相似度指标的索引
npx wrangler vectorize create my-index
--dimensions=768
--metric=cosine
--dimensions=768
--metric=cosine
npx wrangler vectorize create my-index
--dimensions=768
--metric=cosine
--dimensions=768
--metric=cosine
2. Create metadata indexes IMMEDIATELY (before inserting vectors!)
2. 立即创建元数据索引(在插入向量之前!)
npx wrangler vectorize create-metadata-index my-index
--property-name=category
--type=string
--property-name=category
--type=string
npx wrangler vectorize create-metadata-index my-index
--property-name=timestamp
--type=number
--property-name=timestamp
--type=number
**Why**: Metadata indexes MUST exist before vectors are inserted. Vectors added before a metadata index was created won't be filterable on that property.npx wrangler vectorize create-metadata-index my-index
--property-name=category
--type=string
--property-name=category
--type=string
npx wrangler vectorize create-metadata-index my-index
--property-name=timestamp
--type=number
--property-name=timestamp
--type=number
**原因**:元数据索引必须在插入向量之前存在。在元数据索引创建前添加的向量无法通过该属性进行过滤。Index Configuration (Cannot Be Changed Later)
索引配置(创建后无法修改)
bash
undefinedbash
undefinedDimensions MUST match your embedding model output:
维度必须与你的嵌入向量模型输出匹配:
- Workers AI @cf/baai/bge-base-en-v1.5: 768 dimensions
- Workers AI @cf/baai/bge-base-en-v1.5:768维度
- OpenAI text-embedding-3-small: 1536 dimensions
- OpenAI text-embedding-3-small:1536维度
- OpenAI text-embedding-3-large: 3072 dimensions
- OpenAI text-embedding-3-large:3072维度
Metrics determine similarity calculation:
相似度指标决定相似度计算方式:
- cosine: Best for normalized embeddings (most common)
- cosine:最适合归一化嵌入向量(最常用)
- euclidean: Absolute distance between vectors
- euclidean:向量间的绝对距离
- dot-product: For non-normalized vectors
- dot-product:用于非归一化向量
undefinedundefinedWrangler Configuration
Wrangler配置
wrangler.jsonc:
jsonc
{
"name": "my-vectorize-worker",
"main": "src/index.ts",
"compatibility_date": "2025-10-21",
"vectorize": [
{
"binding": "VECTORIZE_INDEX",
"index_name": "my-index"
}
],
"ai": {
"binding": "AI"
}
}wrangler.jsonc:
jsonc
{
"name": "my-vectorize-worker",
"main": "src/index.ts",
"compatibility_date": "2025-10-21",
"vectorize": [
{
"binding": "VECTORIZE_INDEX",
"index_name": "my-index"
}
],
"ai": {
"binding": "AI"
}
}TypeScript Types
TypeScript类型定义
typescript
export interface Env {
VECTORIZE_INDEX: VectorizeIndex;
AI: Ai;
}
interface VectorizeVector {
id: string;
values: number[] | Float32Array | Float64Array;
namespace?: string;
metadata?: Record<string, string | number | boolean | string[]>;
}
interface VectorizeMatches {
matches: Array<{
id: string;
score: number;
values?: number[];
metadata?: Record<string, any>;
namespace?: string;
}>;
count: number;
}typescript
export interface Env {
VECTORIZE_INDEX: VectorizeIndex;
AI: Ai;
}
interface VectorizeVector {
id: string;
values: number[] | Float32Array | Float64Array;
namespace?: string;
metadata?: Record<string, string | number | boolean | string[]>;
}
interface VectorizeMatches {
matches: Array<{
id: string;
score: number;
values?: number[];
metadata?: Record<string, any>;
namespace?: string;
}>;
count: number;
}Metadata Filter Operators (V2)
元数据过滤操作符(V2)
Vectorize V2 supports advanced metadata filtering with range queries:
typescript
// Equality (implicit $eq)
{ category: "docs" }
// Not equals
{ status: { $ne: "archived" } }
// In/Not in arrays
{ category: { $in: ["docs", "tutorials"] } }
{ category: { $nin: ["deprecated", "draft"] } }
// Range queries (numbers) - NEW in V2
{ timestamp: { $gte: 1704067200, $lt: 1735689600 } }
// Range queries (strings) - prefix searching
{ url: { $gte: "/docs/workers", $lt: "/docs/workersz" } }
// Nested metadata with dot notation
{ "author.id": "user123" }
// Multiple conditions (implicit AND)
{ category: "docs", language: "en", "metadata.published": true }Vectorize V2支持带范围查询的高级元数据过滤:
typescript
// 相等匹配(隐式$eq)
{ category: "docs" }
// 不相等
{ status: { $ne: "archived" } }
// 在/不在数组中
{ category: { $in: ["docs", "tutorials"] } }
{ category: { $nin: ["deprecated", "draft"] } }
// 范围查询(数字)- V2新增
{ timestamp: { $gte: 1704067200, $lt: 1735689600 } }
// 范围查询(字符串)- 前缀搜索
{ url: { $gte: "/docs/workers", $lt: "/docs/workersz" } }
// 使用点符号访问嵌套元数据
{ "author.id": "user123" }
// 多条件(隐式AND)
{ category: "docs", language: "en", "metadata.published": true }Metadata Best Practices
元数据最佳实践
1. Cardinality Considerations
1. 基数考量
Low Cardinality (Good for $eq filters):
typescript
// Few unique values - efficient filtering
metadata: {
category: "docs", // ~10 categories
language: "en", // ~5 languages
published: true // 2 values (boolean)
}High Cardinality (Avoid in range queries):
typescript
// Many unique values - avoid large range scans
metadata: {
user_id: "uuid-v4...", // Millions of unique values
timestamp_ms: 1704067200123 // Use seconds instead
}低基数(适合$eq过滤):
typescript
// 唯一值数量少 - 过滤效率高
metadata: {
category: "docs", // 约10个分类
language: "en", // 约5种语言
published: true // 2种取值(布尔值)
}高基数(避免用于范围查询):
typescript
// 唯一值数量多 - 避免大范围扫描
metadata: {
user_id: "uuid-v4...", // 数百万个唯一值
timestamp_ms: 1704067200123 // 改用秒级时间戳
}2. Metadata Limits
2. 元数据限制
- Max 10 metadata indexes per Vectorize index
- Max 10 KiB metadata per vector
- String indexes: First 64 bytes (UTF-8)
- Number indexes: Float64 precision
- Filter size: Max 2048 bytes (compact JSON)
- 每个Vectorize索引最多10个元数据索引
- 每个向量的元数据最大10 KiB
- 字符串索引:前64字节(UTF-8编码)
- 数字索引:Float64精度
- 过滤条件大小:最大2048字节(压缩JSON)
3. Key Restrictions
3. 键名限制
typescript
// ❌ INVALID metadata keys
metadata: {
"": "value", // Empty key
"user.name": "John", // Contains dot (reserved for nesting)
"$admin": true, // Starts with $
"key\"with\"quotes": 1 // Contains quotes
}
// ✅ VALID metadata keys
metadata: {
"user_name": "John",
"isAdmin": true,
"nested": { "allowed": true } // Access as "nested.allowed" in filters
}typescript
// ❌ 无效的元数据键名
metadata: {
"": "value", // 空键名
"user.name": "John", // 包含点号(保留用于嵌套结构)
"$admin": true, // 以$开头
"key\"with\"quotes": 1 // 包含引号
}
// ✅ 有效的元数据键名
metadata: {
"user_name": "John",
"isAdmin": true,
"nested": { "allowed": true } // 在过滤中使用"nested.allowed"访问
}Common Errors & Solutions
常见错误与解决方案
Error 1: Metadata Index Created After Vectors Inserted
错误1:元数据索引在插入向量后创建
Problem: Filtering doesn't work on existing vectors
Solution: Delete and re-insert vectors OR create metadata indexes BEFORE inserting问题:现有向量无法被过滤
解决方案:删除并重新插入向量,或在插入向量前创建元数据索引Error 2: Dimension Mismatch
错误2:维度不匹配
Problem: "Vector dimensions do not match index configuration"
Solution: Ensure embedding model output matches index dimensions:
- Workers AI bge-base: 768
- OpenAI small: 1536
- OpenAI large: 3072问题:"向量维度与索引配置不匹配"
解决方案:确保嵌入向量模型的输出与索引维度匹配:
- Workers AI bge-base:768维度
- OpenAI small:1536维度
- OpenAI large:3072维度Error 3: Invalid Metadata Keys
错误3:无效的元数据键名
Problem: "Invalid metadata key"
Solution: Keys cannot:
- Be empty
- Contain . (dot)
- Contain " (quote)
- Start with $ (dollar sign)问题:"无效的元数据键名"
解决方案:键名不能:
- 为空
- 包含.(点号)
- 包含"(引号)
- 以$(美元符号)开头Error 4: Filter Too Large
错误4:过滤条件过大
Problem: "Filter exceeds 2048 bytes"
Solution: Simplify filter or split into multiple queries问题:"过滤条件超过2048字节"
解决方案:简化过滤条件或拆分为多个查询Error 5: Range Query on High Cardinality
错误5:高基数字段的范围查询
Problem: Slow queries or reduced accuracy
Solution: Use lower cardinality fields for range queries, or use seconds instead of milliseconds for timestamps问题:查询缓慢或准确性降低
解决方案:使用低基数字段进行范围查询,或对时间戳使用秒级而非毫秒级Error 6: Insert vs Upsert Confusion
错误6:Insert与Upsert混淆
Problem: Updates not reflecting in index
Solution: Use upsert() to overwrite existing vectors, not insert()问题:更新未在索引中生效
解决方案:使用upsert()覆盖现有向量,而非insert()Error 7: Missing Bindings
错误7:缺少绑定配置
Problem: "VECTORIZE_INDEX is not defined"
Solution: Add [[vectorize]] binding to wrangler.jsonc问题:"VECTORIZE_INDEX未定义"
解决方案:在wrangler.jsonc中添加[[vectorize]]绑定Error 8: Namespace vs Metadata Confusion
错误8:命名空间与元数据混淆
Problem: Unclear when to use namespace vs metadata filtering
Solution:
- Namespace: Partition key, applied BEFORE metadata filters
- Metadata: Flexible key-value filtering within namespace问题:不清楚何时使用命名空间或元数据过滤
解决方案:
- 命名空间:分区键,在元数据过滤前生效
- 元数据:命名空间内的灵活键值对过滤Error 9: V2 Async Mutation Timing (NEW in V2)
错误9:V2异步突变时序(V2新增)
Problem: Inserted vectors not immediately queryable
Solution: V2 mutations are asynchronous - vectors may take a few seconds to be reflected
- Use mutationId to track mutation status
- Check env.VECTORIZE_INDEX.describe() for processedUpToMutation timestamp问题:插入的向量无法立即查询到
解决方案:V2的突变操作是异步的 - 向量可能需要几秒才能生效
- 使用mutationId跟踪突变状态
- 查看env.VECTORIZE_INDEX.describe()中的processedUpToMutation时间戳Error 10: V1 returnMetadata Boolean (BREAKING in V2)
错误10:V1中的returnMetadata布尔值(V2中为破坏性变更)
Problem: "returnMetadata must be 'all', 'indexed', or 'none'"
Solution: V2 changed returnMetadata from boolean to string enum:
- ❌ V1: { returnMetadata: true }
- ✅ V2: { returnMetadata: 'all' }问题:"returnMetadata必须为'all'、'indexed'或'none'"
解决方案:V2将returnMetadata从布尔值改为字符串枚举:
- ❌ V1:{ returnMetadata: true }
- ✅ V2:{ returnMetadata: 'all' }V2 Migration Checklist
V2迁移检查清单
If migrating from V1 to V2:
- ✅ Update wrangler to 3.71.0+ ()
npm install -g wrangler@latest - ✅ Create new V2 index (can't upgrade V1 → V2)
- ✅ Create metadata indexes BEFORE inserting vectors
- ✅ Update boolean → string enum ('all', 'indexed', 'none')
returnMetadata - ✅ Handle async mutations (expect in responses)
mutationId - ✅ Test with V2 limits (topK up to 100, 5M vectors per index)
- ✅ Update error handling for async behavior
V1 Deprecation:
- After December 2024: Cannot create new V1 indexes
- Existing V1 indexes: Continue to work
- Use for V1 operations
wrangler vectorize --deprecated-v1
如果从V1迁移至V2:
- ✅ 将wrangler更新至3.71.0+()
npm install -g wrangler@latest - ✅ 创建新的V2索引(无法直接将V1升级为V2)
- ✅ 在插入向量前创建元数据索引
- ✅ 将从布尔值改为字符串枚举('all'、'indexed'、'none')
returnMetadata - ✅ 处理异步突变(响应中会返回)
mutationId - ✅ 测试V2的限制(topK最多100、每个索引500万向量)
- ✅ 更新错误处理逻辑以适配异步行为
V1弃用说明:
- 2024年12月后:无法创建新的V1索引
- 现有V1索引:可继续使用
- 执行V1操作需使用标志
wrangler vectorize --deprecated-v1
Official Documentation
官方文档
- Vectorize V2 Docs: https://developers.cloudflare.com/vectorize/
- V2 Changelog: https://developers.cloudflare.com/vectorize/platform/changelog/
- V1 to V2 Migration: https://developers.cloudflare.com/vectorize/reference/transition-vectorize-legacy/
- Metadata Filtering: https://developers.cloudflare.com/vectorize/reference/metadata-filtering/
- Workers AI Models: https://developers.cloudflare.com/workers-ai/models/
Status: Production Ready ✅ (Vectorize V2 GA - September 2024)
Last Updated: 2025-11-22
Token Savings: ~70%
Errors Prevented: 10 (includes V2 breaking changes)
- Vectorize V2文档:https://developers.cloudflare.com/vectorize/
- V2更新日志:https://developers.cloudflare.com/vectorize/platform/changelog/
- V1至V2迁移指南:https://developers.cloudflare.com/vectorize/reference/transition-vectorize-legacy/
- 元数据过滤:https://developers.cloudflare.com/vectorize/reference/metadata-filtering/
- Workers AI模型:https://developers.cloudflare.com/workers-ai/models/
状态:已就绪可用于生产 ✅(Vectorize V2正式发布 - 2024年9月)
最后更新:2025-11-22
Token节省率:约70%
避免的错误数:10(包含V2重大变更相关错误)