ce-retune

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Retune a Corpus for a New Model

针对新模型重新调整Corpus语料库

A corpus that degrades on a new model is a measurement problem before it is a writing problem. Reading the prose and rewriting what looks wrong produces a plausible fix list and no way to know whether any item mattered.
Outcome: a corpus whose measured behavior on the target model clears a bar registered before any change, with the regression classes removed and each removal attributable.
Done: the bar is cleared, or the run reports the specific claim it could not support. A green test suite is not done: it proves nothing broke, not that behavior improved.
Non-goal: word reduction. Leanness and performance are separate programs that happen to share a corpus, and only one of them is the result. Report completion, not word count.
语料库在新模型上表现退化,首先是测量问题,而非文案撰写问题。仅通过阅读文案并改写看起来有问题的内容,只能得到一份看似合理的修改清单,却无法得知其中任何一项修改是否真正有效。
成果: 语料库在目标模型上的可测量行为达到修改前预先设定的指标,回归问题已被消除,且每一项修改的影响均可追溯。
完成标准: 达到设定指标,或运行报告明确指出无法达成的具体目标。测试套件全绿不代表完成:这只能证明没有功能损坏,无法证明行为有所改善。
非目标: 精简文字。简洁性与性能是两个独立的优化方向,只是共享同一语料库,且只有其中一个是本次任务的成果。报告任务完成情况,而非字数统计。

Setup

准备工作

Run this once at the start of this invocation, before any subagent dispatch, and follow the directives it prints — except where one conflicts with this skill's own rules on asking the user questions, whether those rules are scoped to a non-interactive mode or apply in every mode, in which case this skill's rules win and no blocking question is asked. Run the fence exactly as written, as its own command: do not pipe or filter it (no
head
,
tail
, or
grep
), do not truncate its output, and do not bundle it into a batch with other commands. Its output opens with a
=== skill context
header and ends with
CE_CONTEXT_END
; if you received one of those lines without the other, the output was truncated — rerun the fence verbatim once. That recovery is the only rerun: otherwise do not rerun it within the same invocation; a later invocation of this or any other skill runs its own. If no Node runtime is available the skill proceeds unchanged.
bash
SKILL_DIR="<absolute path of the directory containing the SKILL.md you just read>";
NODE="$(for c in node nodejs; do command -v "$c" >/dev/null 2>&1 && "$c" -e '' >/dev/null 2>&1 && { echo "$c"; break; }; done)";
if [ -n "$NODE" ]; then
"$NODE" "$SKILL_DIR/scripts/context.mjs" || echo "context script failed; continue with the skill's normal behavior";
else
echo "no Node runtime; continue with the skill's normal behavior";
fi
在调用任何子Agent之前,先执行一次以下操作,并遵循其输出指令——除非指令与本Skill中关于询问用户问题的规则冲突(无论这些规则适用于非交互模式还是所有模式),此时以本Skill的规则为准,不得提出阻塞性问题。严格按照原样执行以下代码块,作为独立命令:不要进行管道或过滤操作(如
head
tail
grep
),不要截断输出,不要与其他命令批量执行。其输出以
=== skill context
开头,以
CE_CONTEXT_END
结尾;如果只收到其中一行,说明输出被截断——请重新完整执行一次。仅在这种情况下重新执行;同一调用中不得再次执行;后续调用本Skill或其他Skill时,将自行执行该操作。若没有Node运行环境,Skill将正常继续执行。
bash
SKILL_DIR="<absolute path of the directory containing the SKILL.md you just read>";
NODE="$(for c in node nodejs; do command -v "$c" >/dev/null 2>&1 && "$c" -e '' >/dev/null 2>&1 && { echo "$c"; break; }; done)";
if [ -n "$NODE" ]; then
"$NODE" "$SKILL_DIR/scripts/context.mjs" || echo "context script failed; continue with the skill's normal behavior";
else
echo "no Node runtime; continue with the skill's normal behavior";
fi

Phase 0: the measurement gate — check this first

阶段0:测量准入检查——优先执行

This skill cannot run without a way to observe behavior. Check for all three, and name whichever is missing:
  1. A run archive or a harness that produces one — per-run logs carrying the tool-call trace, a terminal marker, token counts, and the final message.
  2. A build selector — the harness can point a run at a specific source checkout of the corpus (a
    --plugin-dir
    -style override, a configurable skills path, an env var), so two builds are comparable under one runner.
  3. A repeatable task the corpus actually executes end to end.
If any is missing, stop and say so, naming what to build. Do not fall back to a static audit and present it as retuning: an audit can say what looks cuttable and can never say whether cutting helped, which is the error this skill exists to prevent. An audit-only pass is a legitimate thing to want; it is a different request.
State the target model and the harness you found before continuing.
本Skill无法在没有行为观测手段的情况下运行。检查以下三项是否齐全,并指出缺失项:
  1. 运行档案或可生成档案的测试框架——包含工具调用轨迹、终端标记、Token计数和最终消息的单运行日志。
  2. 构建选择器——测试框架可指向语料库的特定源代码版本(如
    --plugin-dir
    风格的覆盖配置、可配置的技能路径、环境变量),以便在同一运行器下对比两个构建版本。
  3. 语料库可完整执行的可重复任务
若有任何一项缺失,立即停止并说明情况,指出需要构建的内容。不要退而求其次进行静态审计并将其当作重新调整:审计只能判断哪些内容看似可删除,却永远无法得知删除是否有效——这正是本Skill要避免的错误。仅审计是合理的需求,但属于不同的请求。
继续执行前,请说明目标模型和找到的测试框架。

Phase 1: mine the archive before spending a run

阶段1:先从档案中挖掘数据,再执行新运行

Historical runs are a free baseline, usually larger than any experiment affordable this week. Read
references/baseline-mining.md
and follow it.
It carries the outcome taxonomy, the fields to extract, and the two corrections that decide whether the baseline is usable at all:
  • Broken runs are a first-class outcome, not a failure. Empty transcripts and error exits score as model failures and silently inflate every effect. Exclude them and check whether they land evenly across arms; a lopsided split is a harness fault wearing a model-effect costume.
  • Track "followed the process" and "did the job" separately. A run can complete the task while skipping the workflow entirely. Collapsed into one number, that reads as success.
历史运行数据是免费的基准,通常比本周可开展的任何实验规模都大。阅读
references/baseline-mining.md
并遵循其中指引。
该文档包含结果分类体系、需提取的字段,以及两个决定基准是否可用的修正规则:
  • 失败运行是一级结果,而非故障。 空记录和错误退出应被标记为模型失败,否则会悄无声息地放大所有影响。排除这些运行,并检查它们在各组中的分布是否均匀;分布不均是测试框架故障,却会伪装成模型影响。
  • 分开追踪“遵循流程”和“完成任务”两个指标。 运行可能跳过工作流直接完成任务。若合并为一个数值,会被误判为成功。

Phase 2: establish the noise floor before any claim

阶段2:在提出任何结论前,建立噪声下限

Run the harness against two identical copies of the corpus, same commit on both sides. Whatever difference appears is noise, and it is the floor every later claim must clear.
Read
references/noise-floor.md
for the protocol, the interleaving rule, and the power calculation that converts the observed variance into a required sample size.
Register the bar now, in writing, before any change exists. A bar chosen after seeing results is not a bar.
Expect the floor to be wider than intuition suggests. If a corpus produces a large spread on fixed inputs, then every small-sample claim about it — including any prior report you were handed — sits inside the envelope of doing nothing.
针对两个完全相同的语料库副本(同一提交版本)运行测试框架。出现的任何差异均为噪声,这是后续所有结论必须跨越的下限。
阅读
references/noise-floor.md
了解协议、交错规则,以及将观测方差转换为所需样本量的功效计算方法。
立即以书面形式记录设定的指标,在进行任何修改之前。看到结果后再选择的指标不能算作有效指标。
预期噪声下限会比直觉更宽。如果语料库在固定输入下产生较大波动,那么所有关于它的小样本结论——包括你收到的任何先前报告——都属于无操作情况下的正常波动范围。

Phase 3: audit the corpus, adversarially

阶段3:对语料库进行对抗性审计

One agent per skill, each reading that skill's full directory, proposing cuts with a target and a reason. Then a second agent per skill whose job is the opposite: defend the existing prose using the project's own documented learnings, its tests, and git history.
Read
references/corpus-audit.md
for the dispatch shape, the finding schema, and the classes worth hunting.
These two passes require independent contexts. The defense is only worth running when it can genuinely disagree with the proposal, which one context arguing both sides cannot do. If the host exposes no way to run them as separate agents, report that as a blocker and stop the audit — do not run proposal and defense inline and present the result as an audit.
Two rules make the difference between an audit and a demolition:
  • A cut with no provenance found after a real search is a confident cut. A cut the defender saves with a citation is off the list. Do not relitigate a defended keep.
  • Absence of evidence is weaker than the project's own standard for a change. Where the guidance requires a reproduced failure or an exact failing path, a search that found nothing is a verification task, not a change. Say which of your cuts rest on that weaker basis.
Expect the audit to contradict the premise you started with. That is its value.
每个Skill分配一个Agent,完整阅读该Skill的目录,提出带有目标和理由的精简建议。然后为每个Skill分配第二个Agent,其任务完全相反:利用项目自身的文档化经验、测试用例和Git历史,为现有文案辩护
阅读
references/corpus-audit.md
了解调度形式、发现结果的 schema,以及值得排查的问题类别。
这两个流程需要独立的上下文。 只有当辩护方能够真正与建议方产生分歧时,辩护才有意义,单一上下文无法做到这一点。若宿主环境无法将它们作为独立Agent运行,请报告这一阻塞问题并停止审计——不要在同一上下文中同时执行建议和辩护,并将结果当作审计结果。
以下两条规则是区分审计与盲目删除的关键:
  • 经过真实搜索后找不到依据的精简建议可放心执行。被辩护方引用依据保留的内容需从清单中移除。 不要重新争论已被辩护保留的内容。
  • 缺乏证据的说服力弱于项目自身的变更标准。 若项目指南要求重现故障或明确的失败路径,未找到相关内容的搜索只能作为验证任务,而非变更依据。请说明哪些精简建议基于这种较弱的依据。
预期审计结果会与你最初的假设相矛盾,这正是审计的价值所在。

Phase 4: cut in surgical passes

阶段4:分阶段精准精简

One problem per agent, each owning a disjoint file set so parallel work cannot collide. Read
references/cut-passes.md
for the loop, the isolation rules, and the shared-asset trap.
references/halt-taxonomy.md
carries the regression classes to hunt, with the before and after of each. Load it when the symptom is stalling, halting, or a run that ends while naming work it did not do. Every one of those classes reduces to prose written as if a second party were waiting, and the fix is never to add capability.
Discipline that survives contact:
  • Fix at the smallest owning layer. Reword only when rewording is the smallest mechanism; prefer deleting the structure that made the wording necessary.
  • Field names, enums, greppable markers and security guards are data. They stay. What goes is the justification clause around them that teaches the model a separate consumer is waiting.
  • Not every stop is the enemy. Some workflows exist to stop and ask; that is the product. Sort every stop by who is actually on the other side before touching it.
  • Never edit tests to make a suite green. A removed string a test pins is a finding to report, not a test to weaken.
每个问题分配一个Agent,每个Agent负责互不重叠的文件集,避免并行工作时出现冲突。阅读
references/cut-passes.md
了解循环流程、隔离规则和共享资产陷阱。
references/halt-taxonomy.md
包含需排查的回归问题类别,以及每个类别的前后状态。当出现停滞、终止或运行结束但未完成声称工作的症状时,请参考该文档。这些问题类别本质上都是文案假设存在第三方等待响应,而修复方案永远不是添加新功能。
需遵守的纪律:
  • 在最小的责任层级修复问题。 只有重写是最小修复机制时才进行重写;优先删除导致该文案存在的结构。
  • 字段名、枚举、可 grep 的标记和安全防护属于数据。 这些内容需保留。要移除的是围绕它们的解释性条款——这些条款会让模型误以为存在另一个等待响应的消费者。
  • 并非所有停止都是问题。 有些工作流的设计就是为了停止并询问用户;这是产品需求。在修改任何停止逻辑前,先明确另一端的对象是谁。
  • 永远不要为了让测试套件全绿而编辑测试用例。 若测试用例依赖的字符串被移除,这是需要报告的发现,而非削弱测试的理由。

Phase 5: measure, then let the failure choose the next fix

阶段5:测量结果,让失败指引下一次修复

After each pass, run the harness and read where it failed, not just whether it did.
A failure that moves to a later phase is progress and names the next target. A failure at the same site means the fix missed. A run that completes the task while skipping the workflow is a different defect than a halt, and only shows up if Phase 1's two metrics stayed separate.
Loop Phase 4 and 5 until the registered bar is cleared. Then stop; a bar cleared is done.
Audit the phases the instrument cannot reach. A probe that skips a phase can never fail in it, so a green streak certifies only what it exercised. List the phases your task never enters, read those files, and treat what you find there as equal in weight to what the runs found. Some of the most consequential defects live where no test looks.
Report the limit. Name the paths that remain unmeasured and what would be needed to measure them. Do not let a cleared bar imply coverage it does not have.
每完成一次迭代后,运行测试框架并查看失败的具体位置,而非仅关注是否失败。
失败点转移到后续阶段是进步,并指明了下一个目标。同一位置重复失败说明修复未命中问题。跳过工作流直接完成任务的运行与终止运行是不同的缺陷,只有在阶段1中分开追踪两个指标时才会被发现。
循环执行阶段4和5,直至达到预先设定的指标。然后停止;达到指标即任务完成。
审计工具无法覆盖的阶段。 跳过某一阶段的探针永远不会在该阶段失败,因此连续成功只能证明它覆盖的部分没问题。列出任务从未进入的阶段,阅读这些阶段的文件,并将发现的问题与运行中发现的问题同等对待。一些影响最重大的缺陷往往存在于测试未覆盖的地方。
报告局限性。 列出仍未被测量的路径,以及测量这些路径所需的条件。不要让达到指标的结果暗示不存在的覆盖范围。

Phase 6: ship

阶段6:交付

Commit each pass separately with its own message so the history says which change was made and why, and so release tooling can classify intent. Keep the measurement artifacts.
Then write the finding down where the next person will hit it: the mechanism, the before and after, the measured numbers, and the hypotheses that died. Record the ones that died. They are what stops the next attempt from re-running a dead end, and they are the part every write-up omits.
为每次迭代单独提交,并附上对应的提交信息,以便历史记录显示做出了哪些修改及原因,同时便于发布工具对意图进行分类。保留测量工件。
然后将发现记录在后续人员会看到的地方:问题机制、修改前后状态、测量数据,以及被证伪的假设。务必记录被证伪的假设。 它们能防止后续尝试重复无效路径,而这正是所有报告都会遗漏的部分。

Workflow shapes

工作流形式

Each phase has an orchestration shape that fits it, and using the wrong one is the common failure. Read
references/workflow-shapes.md
before dispatching a phase: it covers when to fan out by skill versus by problem, why a shared contract must be authored before a parallel rewrite, and which phases must stay serial.
The one rule worth stating inline: fan out by disjoint file ownership, never by item. Items cross files; agents that share a file lose each other's edits.
每个阶段都有适合的编排形式,使用错误的形式是常见的失败原因。在调度阶段前阅读
references/workflow-shapes.md
:它涵盖了何时按Skill拆分任务、何时按问题拆分任务,为何在并行重写前必须制定共享契约,以及哪些阶段必须串行执行。
需要重点强调的一条规则:按互不重叠的文件所有权拆分任务,而非按条目拆分。 条目可能跨文件,共享同一文件的Agent会丢失彼此的修改。