Back to Details

prompt-caching-strategy

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Prompt Caching Strategy

Prompt 缓存策略

Arrange reusable prompt content to improve cache reuse, then measure input cost and latency. Savings depend on model, platform, cache eligibility, writes, retention, reuse and output volume. Runtime cache hits and savings for this skill are unmeasured; the examples below validate arithmetic without provider calls.
合理排布可复用的 prompt 内容以提升缓存复用率，随后衡量输入成本与延迟。节省幅度取决于模型、平台、缓存适配条件、写入操作、保留时长、复用次数及输出体量。本 skill 的运行时缓存命中率与节省量未经过实测；以下示例仅用于验证算术逻辑，未实际调用服务商接口。

When to Activate

何时启用

Use for repeated API requests that share authorized instructions, tools or reference material. First identify the exact provider endpoint, model, service tier and caching mode. Consult its current documentation before selecting rates, minimum prefix length or retention. Caching does not reduce context-window use.
适用于共享授权指令、工具或参考资料的重复 API 请求。首先需确定具体的服务商端点、模型、服务层级与缓存模式。在选择费率、最小前缀长度或保留时长前，请查阅其最新文档。缓存不会减少上下文窗口的占用。

How It Works

工作原理

Preserve a stable prefix

保持稳定前缀

Keep reusable instructions, tool schemas and examples stable; place changing questions, timestamps and retrieved results after that content where the API allows. Respect the provider's serialization order: Claude uses tools, system, then messages. Matching prefixes need identical content, including tool schemas and relevant request settings. A changed prefix can prevent reuse from that point. A dynamic suffix may itself be cached later if reused unchanged.
text
Reusable prefix: approved tools, instructions, stable examples/reference
Changing suffix: current question, fresh retrieval results, request metadata
Choose only relevant context. Padding prompts or broadening access just to meet an eligibility threshold needs its own cost and quality justification.
保持可复用指令、工具 schema 与示例的稳定性；在 API 允许的情况下，将变化的问题、时间戳和检索结果放在这些内容之后。遵循服务商的序列化顺序：Claude 采用 tools、system、messages 的顺序。匹配的前缀需要内容完全一致，包括工具 schema 和相关请求设置。前缀发生变化可能会导致从该位置起无法复用缓存。动态后缀如果后续被原样复用，自身也可能被缓存。
text
Reusable prefix: approved tools, instructions, stable examples/reference
Changing suffix: current question, fresh retrieval results, request metadata
仅选择相关的上下文内容。仅仅为了满足适配阈值而填充 prompt 或扩大访问范围，需要单独评估其成本与质量合理性。

Preserve user and workspace boundaries

保留用户与工作空间边界

Authorize content access before cache lookup or reuse. Scope application-managed cache handles and accounting keys to the approved user, workspace, provider account, model and content revision. Share only material explicitly approved for that scope. A cache key is a routing/accounting hint, not an authorization gate or a sandbox. Provider cache isolation does not replace application access checks. Invalidate application references when permissions or content change; follow provider retention/deletion controls. Avoid logging private prompt contents. Context selection, capabilities, sandbox enforcement and evidence remain independent of cache optimization.
在缓存查找或复用前完成内容访问授权。将应用管理的缓存句柄与计费密钥的范围限定为已授权的用户、工作空间、服务商账户、模型与内容版本。仅共享该范围内明确获批的资料。缓存键是路由/计费的提示项，而非授权门禁或沙箱。服务商的缓存隔离不能替代应用的访问检查。当权限或内容发生变化时，使应用引用失效；遵循服务商的保留/删除控制规则。避免记录私密 prompt 内容。上下文选择、功能权限、沙箱执行与证据留存均独立于缓存优化。

Provider Requirements and Usage Evidence

服务商要求与使用凭证

The linked documentation is authoritative; endpoint and SDK versions can expose different usage field names. Record that version alongside counters.
ProviderRequirements to verifyCounters to record
OpenAIModel-specific automatic/explicit modes, minimum length, retention and write/read prices; earlier and newer models differResponses 
usage.input_tokens_details.cached_tokens
and, when present, 
cache_write_tokens
; input and output totals. Chat Completions uses 
prompt_tokens_details.cached_tokens
ClaudeSupported platform, model minimum, 
cache_control
support and TTL; cache writes may cost more than ordinary input
input_tokens
, 
cache_creation_input_tokens
, 
cache_read_input_tokens
, 
output_tokens
; split writes by TTL when rates differ
GeminiModel minimum and API: Interactions supports implicit caching; explicit cache objects require GenerateContentInteractions 
usage.total_cached_tokens
; GenerateContent 
usage_metadata
cache/input/output counters plus explicit-cache token count and retained duration
Use OpenAI's prompt caching guide and pricing for the selected model. For Chat Completions, check the usage schema. Derive ordinary input from the total minus cached reads and any separately reported writes; do not count the same tokens twice.
Use Claude's prompt caching guide for current model/platform limits and TTL rates. Its 
input_tokens
counts ordinary input separately from cache reads and writes. A short prefix can produce zero cache counters even when marked for caching. Do not assume every Claude model or hosted platform uses the same minimum or read price.
Use Gemini's caching guide, GenerateContent caching guide and pricing. Explicit-cache storage is charged by token count and retained duration, separately from reads, ordinary input and output. Implicit hits are not guaranteed. Verify billing categories for the chosen endpoint rather than copying another provider's rates.
所链接的文档具有权威性；不同的端点与 SDK 版本可能会暴露不同的使用字段名。请同时记录版本信息与计数项。
服务商需验证的要求需记录的计数项
OpenAI特定模型的自动/显式模式、最小长度、保留时长与读写价格；新旧模型存在差异响应中的 
usage.input_tokens_details.cached_tokens
，以及存在时的 
cache_write_tokens
；输入与输出总量。Chat Completions 使用 
prompt_tokens_details.cached_tokens
Claude支持的平台、模型最小要求、
cache_control
支持与 TTL；缓存写入成本可能高于普通输入
input_tokens
、
cache_creation_input_tokens
、
cache_read_input_tokens
、
output_tokens
；当费率不同时，按 TTL 拆分写入量
Gemini模型最小要求与 API：Interactions 支持隐式缓存；显式缓存对象需使用 GenerateContentInteractions 的 
usage.total_cached_tokens
；GenerateContent 的 
usage_metadata
缓存/输入/输出计数，以及显式缓存的 token 数与保留时长
请使用所选模型对应的 OpenAI prompt 缓存指南 与定价页面。对于 Chat Completions，请查阅使用量 schema。普通输入量等于总输入量减去缓存读取量和任何单独报告的写入量；请勿重复计算同一 token。
请使用 Claude prompt 缓存指南 了解当前模型/平台限制与 TTL 费率。其 
input_tokens
单独统计普通输入，与缓存读取和写入分开。即使标记了缓存，短前缀也可能产生零缓存计数。请勿假定所有 Claude 模型或托管平台都使用相同的最小长度或读取价格。
请使用 Gemini 缓存指南、GenerateContent 缓存指南 与定价页面。显式缓存存储按 token 数与保留时长计费，与读取、普通输入和输出分开。隐式缓存命中不做保证。请核实所选端点的计费类别，请勿直接套用其他服务商的费率。

Cost Comparison Calculator

成本对比计算器

Normalize actual usage into nonoverlapping categories and use prices in the same currency per million tokens. Split write categories when TTL, modality, tier or model prices differ. Storage uses million-token-hours; other applicable charges such as tools and grounding need their own lines. Include all warmup, miss, expiration, refresh and retry requests in the measured period.
text
baseline = (ordinary + written + read) * ordinary_input_rate / 1e6
           + output * output_rate / 1e6
actual   = ordinary * ordinary_input_rate / 1e6
           + written * cache_write_rate / 1e6
           + read * cache_read_rate / 1e6
           + output * output_rate / 1e6
           + storage_token_hours * storage_rate / 1e6
savings  = baseline - actual
This counterfactual holds prompt and output volumes fixed. For explicit-cache creation, include any separately billed creation input in the write category; use the endpoint's documented rate. Set nonexistent categories to zero. Compare actual before/after invoices separately if warmup requests or output differ. Never apply a cache discount to the total bill or to output tokens.
The executable calculator takes normalized counters, not raw provider responses:
javascript
function compareCacheCost(usage, rates) {
  const counts = ['uncached', 'written', 'read', 'output', 'storageTokenHours'];
  const prices = ['input', 'write', 'read', 'output', 'storage'];
  for (const [values, keys] of [[usage, counts], [rates, prices]]) {
    for (const key of keys) {
      if (!Number.isFinite(values[key]) || values[key] < 0) {
        throw new Error(`${key} must be a finite nonnegative number`);
      }
    }
  }
  const outputCost = usage.output * rates.output / 1e6;
  const storageCost = usage.storageTokenHours * rates.storage / 1e6;
  const baseline = (usage.uncached + usage.written + usage.read)
    * rates.input / 1e6 + outputCost;
  const actual = (usage.uncached * rates.input + usage.written * rates.write
    + usage.read * rates.read) / 1e6 + outputCost + storageCost;
  return { baseline, actual, savings: baseline - actual, outputCost, storageCost };
}
将实际使用量归一化为互不重叠的类别，并使用统一货币的每百万 token 价格。当 TTL、模态、层级或模型价格不同时，拆分写入类别。存储费用按百万 token-小时计算；其他适用费用（如工具与 grounding）需单独列项。计算周期内需包含所有预热、未命中、过期、刷新与重试请求。
text
baseline = (ordinary + written + read) * ordinary_input_rate / 1e6
           + output * output_rate / 1e6
actual   = ordinary * ordinary_input_rate / 1e6
           + written * cache_write_rate / 1e6
           + read * cache_read_rate / 1e6
           + output * output_rate / 1e6
           + storage_token_hours * storage_rate / 1e6
savings  = baseline - actual
此反事实分析假设 prompt 与输出量固定。对于显式缓存创建，将任何单独计费的创建输入计入写入类别；使用端点文档中规定的费率。不存在的类别设为零。如果预热请求或输出量存在差异，请单独对比实际的前后账单。切勿将缓存折扣应用于总账单或输出 token。
可执行计算器接收归一化后的计数项，而非服务商原始响应：
javascript
function compareCacheCost(usage, rates) {
  const counts = ['uncached', 'written', 'read', 'output', 'storageTokenHours'];
  const prices = ['input', 'write', 'read', 'output', 'storage'];
  for (const [values, keys] of [[usage, counts], [rates, prices]]) {
    for (const key of keys) {
      if (!Number.isFinite(values[key]) || values[key] < 0) {
        throw new Error(`${key} must be a finite nonnegative number`);
      }
    }
  }
  const outputCost = usage.output * rates.output / 1e6;
  const storageCost = usage.storageTokenHours * rates.storage / 1e6;
  const baseline = (usage.uncached + usage.written + usage.read)
    * rates.input / 1e6 + outputCost;
  const actual = (usage.uncached * rates.input + usage.written * rates.write
    + usage.read * rates.read) / 1e6 + outputCost + storageCost;
  return { baseline, actual, savings: baseline - actual, outputCost, storageCost };
}

Examples

示例

These are hypothetical prices and counts, not provider quotes or measured hits. Ten calls each contain a 10,000-token stable prefix, 2,000-token changing suffix and 2,000 output tokens. Assume one prefix write and nine full prefix reads.
CategoryTokensExample rate per millionCost
Ordinary input20,000$3.00$0.0600
Cache write10,000$3.75$0.0375
Cache read90,000$0.30$0.0270
Output20,000$15.00$0.3000
Storage0 token-hours$0.00$0.0000
Baseline: 120,000 input tokens at $3 plus the same output = $0.6600. With caching: $0.4245, savings $0.2355. The write premium is included; the $0.3000 output bill remains unchanged. An output-heavy workload has the same absolute input savings under these assumptions, a smaller fraction of total cost.
If an explicit cache retains 10,000 tokens for two hours at a hypothetical $1 per million-token-hour, add $0.0200 storage. Actual cost becomes $0.4445. Use the selected Gemini endpoint's prices for a real estimate.
For a one-off call, that prefix write costs $0.0375 rather than $0.0300 ordinary input: $0.0075 more, with no read savings. For a 500-token prefix below a selected model's documented minimum, treat the request as uncached; claiming future hits without counters would overstate savings. Repeated expiry or changing prefixes can also turn expected savings into higher cost.
以下为假设价格与计数，并非服务商报价或实测命中数据。10 次调用每次包含 10,000 token 的稳定前缀、2,000 token 的变化后缀与 2,000 输出 token。假设执行 1 次前缀写入与 9 次完整前缀读取。
类别Token 数每百万示例费率成本
普通输入20,000$3.00$0.0600
缓存写入10,000$3.75$0.0375
缓存读取90,000$0.30$0.0270
输出20,000$15.00$0.3000
存储0 token-小时$0.00$0.0000
基准成本：120,000 输入 token 按 $3 计费，加上相同输出量 = $0.6600。 使用缓存后：$0.4245，节省 $0.2355。已包含写入溢价；$0.3000 的输出费用保持不变。在这些假设下，输出密集型工作负载的绝对输入节省量相同，但占总成本的比例更低。
如果显式缓存将 10,000 token 保留 2 小时，假设费率为 $1 每百万 token-小时，则需增加 $0.0200 存储费用。实际成本变为 $0.4445。实际估算请使用所选 Gemini 端点的定价。
对于单次调用，前缀写入成本为 $0.0375，而非普通输入的 $0.0300：多花费 $0.0075，且无读取节省。对于 500 token 的前缀（低于所选模型的文档规定最小值），请将该请求视为未缓存；在没有计数的情况下声称未来命中会高估节省量。频繁过期或前缀变化也可能导致预期节省反而变成更高成本。

Verification Workflow

验证工作流

  1. Capture eligible prefix size and model/platform requirements from current docs.
  2. Run the offline example and edge cases with 
    node tests/skills/prompt-caching-strategy.test.js
    in the ECC repository.
  3. If provider traffic is authorized, compare matched requests and aggregate actual cached-token counts, all input categories, output, storage, latency and realized cost by approved user/workspace. Record misses and rewrites too.
  4. Report cached reads divided by total input tokens as a token hit rate; report request hit rate separately. Keep denominators and sampling period explicit.
  5. If no traffic was measured, label hit rate, latency improvement and realized savings unmeasured. Offline arithmetic establishes no runtime cache behavior.
  1. 从当前文档中获取符合条件的前缀大小与模型/平台要求。
  2. 在 ECC 仓库中运行 
    node tests/skills/prompt-caching-strategy.test.js
    执行离线示例与边界用例。
  3. 如果已授权服务商流量访问，对比匹配的请求，按已授权用户/工作空间聚合实际缓存 token 计数、所有输入类别、输出、存储、延迟与实际成本。同时记录未命中与重写情况。
  4. 将缓存读取量除以总输入 token 数作为 token 命中率；单独统计请求命中率。明确标注分母与采样周期。
  5. 如果未测量流量，请将命中率、延迟改善与实际节省量标记为未实测。离线算术计算无法证明运行时缓存行为。

Relationship to Other Skills

与其他技能的关联

  • prompt-optimizer improves prompt quality before optimizing reuse.
  • context-budget identifies unnecessary context; caching still consumes it.
  • cost-aware-llm-pipeline compares caching with routing, retries and other costs.
  • prompt-optimizer：在优化复用前提升 prompt 质量。
  • context-budget：识别不必要的上下文；缓存仍会占用上下文。
  • cost-aware-llm-pipeline：对比缓存与路由、重试等其他成本。