Back to Details

context-optimizer

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Context Pruner

Context Pruner

Advanced context management optimized for DeepSeek's 64k context window. Provides intelligent pruning, compression, and token optimization to prevent context overflow while preserving important information.
针对DeepSeek 64k上下文窗口优化的高级上下文管理工具。提供智能修剪、压缩与Token优化功能，在防止上下文溢出的同时保留重要信息。

Key Features

核心功能

  • DeepSeek-optimized: Specifically tuned for 64k context window
  • Adaptive pruning: Multiple strategies based on context usage
  • Semantic deduplication: Removes redundant information
  • Priority-aware: Preserves high-value messages
  • Token-efficient: Minimizes token overhead
  • Real-time monitoring: Continuous context health tracking
  • DeepSeek专属优化: 专为64k上下文窗口调校
  • 自适应修剪: 基于上下文使用情况的多种策略
  • 语义去重: 移除冗余信息
  • 优先级感知: 保留高价值消息
  • Token高效: 最大限度降低Token开销
  • 实时监控: 持续跟踪上下文健康状态

Quick Start

快速开始

Auto-compaction with dynamic context:

带动态上下文的自动压缩：

javascript
import { createContextPruner } from './lib/index.js';

const pruner = createContextPruner({
  contextLimit: 64000, // DeepSeek's limit
  autoCompact: true,    // Enable automatic compaction
  dynamicContext: true, // Enable dynamic relevance-based context
  strategies: ['semantic', 'temporal', 'extractive', 'adaptive'],
  queryAwareCompaction: true, // Compact based on current query relevance
});

await pruner.initialize();

// Process messages with auto-compaction and dynamic context
const processed = await pruner.processMessages(messages, currentQuery);

// Get context health status
const status = pruner.getStatus();
console.log(`Context health: ${status.health}, Relevance scores: ${status.relevanceScores}`);

// Manual compaction when needed
const compacted = await pruner.autoCompact(messages, currentQuery);
javascript
import { createContextPruner } from './lib/index.js';

const pruner = createContextPruner({
  contextLimit: 64000, // DeepSeek的上下文窗口
  autoCompact: true,    // 启用自动压缩
  dynamicContext: true, // 启用动态上下文管理
  strategies: ['semantic', 'temporal', 'extractive', 'adaptive'],
  queryAwareCompaction: true, // 根据当前查询相关性进行压缩
});

await pruner.initialize();

// 使用自动压缩与动态上下文处理消息
const processed = await pruner.processMessages(messages, currentQuery);

// 获取上下文健康状态
const status = pruner.getStatus();
console.log(`Context health: ${status.health}, Relevance scores: ${status.relevanceScores}`);

// 必要时手动执行压缩
const compacted = await pruner.autoCompact(messages, currentQuery);

Archive Retrieval (Hierarchical Memory):

归档检索（分层记忆）：

javascript
// When something isn't in current context, search archive
const archiveResult = await pruner.retrieveFromArchive('query about previous conversation', {
  maxContextTokens: 1000,
  minRelevance: 0.4,
});

if (archiveResult.found) {
  // Add relevant snippets to current context
  const archiveContext = archiveResult.snippets.join('\n\n');
  // Use archiveContext in your prompt
  console.log(`Found ${archiveResult.sources.length} relevant sources`);
  console.log(`Retrieved ${archiveResult.totalTokens} tokens from archive`);
}
javascript
// 当当前上下文中不存在所需信息时，搜索归档
const archiveResult = await pruner.retrieveFromArchive('query about previous conversation', {
  maxContextTokens: 1000,
  minRelevance: 0.4,
});

if (archiveResult.found) {
  // 将相关片段添加到当前上下文
  const archiveContext = archiveResult.snippets.join('\n\n');
  // 在提示词中使用archiveContext
  console.log(`Found ${archiveResult.sources.length} relevant sources`);
  console.log(`Retrieved ${archiveResult.totalTokens} tokens from archive`);
}

Auto-Compaction Strategies

自动压缩策略

  1. Semantic Compaction: Merges similar messages instead of removing them
  2. Temporal Compaction: Summarizes older conversations by time windows
  3. Extractive Compaction: Extracts key information from verbose messages
  4. Adaptive Compaction: Chooses best strategy based on message characteristics
  5. Dynamic Context: Filters messages based on relevance to current query
  1. 语义压缩: 合并相似消息而非直接移除
  2. 时间压缩: 按时间窗口总结旧对话
  3. 提取式压缩: 从冗长消息中提取关键信息
  4. 自适应压缩: 根据消息特征选择最优策略
  5. 动态上下文: 根据与当前查询的相关性过滤消息

Dynamic Context Management

动态上下文管理

  • Query-aware Relevance: Scores messages based on similarity to current query
  • Relevance Decay: Relevance scores decay over time for older conversations
  • Adaptive Filtering: Automatically filters low-relevance messages
  • Priority Integration: Combines message priority with semantic relevance
  • 查询感知相关性: 根据与当前查询的相似度为消息打分
  • 相关性衰减: 旧对话的相关性分数随时间衰减
  • 自适应过滤: 自动过滤低相关性消息
  • 优先级整合: 将消息优先级与语义相关性结合

Hierarchical Memory System

分层记忆系统

The context archive provides a RAM vs Storage approach:
  • Current Context (RAM): Limited (64k tokens), fast access, auto-compacted
  • Archive (Storage): Larger (100MB), slower but searchable
  • Smart Retrieval: When information isn't in current context, efficiently search archive
  • Selective Loading: Extract only relevant snippets, not entire documents
  • Automatic Storage: Compacted content automatically stored in archive
上下文归档采用类似内存（RAM）与存储（Storage）的分层方案：
  • 当前上下文（RAM）: 容量有限（64k Token），访问速度快，自动压缩
  • 归档（Storage）: 容量更大（100MB），访问速度较慢但可搜索
  • 智能检索: 当当前上下文中不存在所需信息时，高效搜索归档
  • 选择性加载: 仅提取相关片段，而非完整文档
  • 自动存储: 压缩后的内容自动存储到归档

Configuration

配置

javascript
{
  contextLimit: 64000, // DeepSeek's context window
  autoCompact: true, // Enable automatic compaction
  compactThreshold: 0.75, // Start compacting at 75% usage
  aggressiveCompactThreshold: 0.9, // Aggressive compaction at 90%
  
  dynamicContext: true, // Enable dynamic context management
  relevanceDecay: 0.95, // Relevance decays 5% per time step
  minRelevanceScore: 0.3, // Minimum relevance to keep
  queryAwareCompaction: true, // Compact based on current query relevance
  
  strategies: ['semantic', 'temporal', 'extractive', 'adaptive'],
  preserveRecent: 10, // Always keep last N messages
  preserveSystem: true, // Always keep system messages
  minSimilarity: 0.85, // Semantic similarity threshold
  
  // Archive settings
  enableArchive: true, // Enable hierarchical memory system
  archivePath: './context-archive',
  archiveSearchLimit: 10,
  archiveMaxSize: 100 * 1024 * 1024, // 100MB
  archiveIndexing: true,
  
  // Chat logging
  logToChat: true, // Log optimization events to chat
  chatLogLevel: 'brief', // 'brief', 'detailed', or 'none'
  chatLogFormat: '📊 {action}: {details}', // Format for chat messages
  
  // Performance
  batchSize: 5, // Messages to process in batch
  maxCompactionRatio: 0.5, // Maximum 50% compaction in one pass
}
javascript
{
  contextLimit: 64000, // DeepSeek的上下文窗口
  autoCompact: true, // 启用自动压缩
  compactThreshold: 0.75, // 当使用率达到75%时开始压缩
  aggressiveCompactThreshold: 0.9, // 当使用率达到90%时进行激进压缩
  
  dynamicContext: true, // 启用动态上下文管理
  relevanceDecay: 0.95, // 每时间步相关性衰减5%
  minRelevanceScore: 0.3, // 保留消息的最低相关性分数
  queryAwareCompaction: true, // 根据当前查询相关性进行压缩
  
  strategies: ['semantic', 'temporal', 'extractive', 'adaptive'],
  preserveRecent: 10, // 始终保留最后N条消息
  preserveSystem: true, // 始终保留系统消息
  minSimilarity: 0.85, // 语义相似度阈值
  
  // 归档设置
  enableArchive: true, // 启用分层记忆系统
  archivePath: './context-archive',
  archiveSearchLimit: 10,
  archiveMaxSize: 100 * 1024 * 1024, // 100MB
  archiveIndexing: true,
  
  // 聊天日志设置
  logToChat: true, // 将优化事件记录到聊天
  chatLogLevel: 'brief', // 可选值：'brief'（简要）、'detailed'（详细）或'none'（无）
  chatLogFormat: '📊 {action}: {details}', // 聊天消息的格式
  
  // 性能设置
  batchSize: 5, // 批量处理的消息数量
  maxCompactionRatio: 0.5, // 单次压缩的最大比例为50%
}

Chat Logging

聊天日志记录

The context optimizer can log events directly to chat:
javascript
// Example chat log messages:
// 📊 Context optimized: Compacted 15 messages → 8 (47% reduction)
// 📊 Archive search: Found 3 relevant snippets (42% similarity)
// 📊 Dynamic context: Filtered 12 low-relevance messages

// Configure logging:
const pruner = createContextPruner({
  logToChat: true,
  chatLogLevel: 'brief', // Options: 'brief', 'detailed', 'none'
  chatLogFormat: '📊 {action}: {details}',
  
  // Custom log handler (optional)
  onLog: (level, message, data) => {
    if (level === 'info' && data.action === 'compaction') {
      // Send to chat
      console.log(`🧠 Context optimized: ${message}`);
    }
  }
});
上下文优化器可直接将事件记录到聊天中：
javascript
// 聊天日志消息示例：
// 📊 Context optimized: Compacted 15 messages → 8 (47% reduction)
// 📊 Archive search: Found 3 relevant snippets (42% similarity)
// 📊 Dynamic context: Filtered 12 low-relevance messages

// 配置日志：
const pruner = createContextPruner({
  logToChat: true,
  chatLogLevel: 'brief', // 可选值：'brief'（简要）、'detailed'（详细）或'none'（无）
  chatLogFormat: '📊 {action}: {details}',
  
  // 自定义日志处理器（可选）
  onLog: (level, message, data) => {
    if (level === 'info' && data.action === 'compaction') {
      // 发送到聊天
      console.log(`🧠 Context optimized: ${message}`);
    }
  }
});

Integration with Clawdbot

与Clawdbot集成

Add to your Clawdbot config:
yaml
skills:
  context-pruner:
    enabled: true
    config:
      contextLimit: 64000
      autoPrune: true
The pruner will automatically monitor context usage and apply appropriate pruning strategies to stay within DeepSeek's 64k limit.
添加到你的Clawdbot配置中：
yaml
skills:
  context-pruner:
    enabled: true
    config:
      contextLimit: 64000
      autoPrune: true
修剪器将自动监控上下文使用情况，并应用合适的修剪策略以维持在DeepSeek的64k限制内。