skill-optimizer

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Skill Optimizer

Skill 优化器

Train an existing SKILL.md the way a deep-learning optimizer trains weights: via rollouts, gradient-like reflections, validation-gated acceptance. No model retraining; only the skill markdown changes.

像深度学习优化器训练权重一样训练现有 SKILL.md：通过 rollout、类梯度反思和验证门控接受机制。不进行模型重训练；只会修改 skill Markdown。

When to use

使用场景

Use this skill when:

A pro-workflow skill has accumulated 8+ learn-rule rows for it
The user reports the skill is "getting bloated" or "rules keep being repeated"
The user wants offline, budget-capped improvement over multiple sessions

Do not use when:

Skill has fewer than 8 trajectories (nothing to learn from)
The user wants real-time edits (this is offline, single-shot)
No key has been explicitly configured for the selected provider

在以下情况下使用此 skill：

pro-workflow skill 已累积 8 条以上 learn-rule 记录
用户反馈该 skill 正在“变得臃肿”或“规则不断重复”
用户希望跨多个会话进行离线且受预算限制的改进

以下情况下不要使用：

Skill 的训练轨迹少于 8 条（没有足够的数据可供学习）
用户希望进行实时编辑（这是离线、单次执行的流程）
尚未为所选 provider 明确配置密钥

Architecture (mirrors SkillOpt's six-stage loop)

架构（对应 SkillOpt 的六阶段循环）

text

rollout      pull recent learnings from SQLite (existing learn-rule rows)
reflect      optimizer LLM analyzes a minibatch, proposes add/delete/replace patches
aggregate    vote-merge patches across minibatches
select       clip by LR budget (default: 3 adds, 2 deletes, 3 replaces per step)
update       apply selected patches to a candidate skill content
evaluate     evaluator LLM scores candidate against held-out validation items
gate         accept candidate only if weighted score >= current + acceptThreshold
slow update  at epoch boundary, consolidate accepted edits into a coherent rewrite

Failed candidates are stored in a rejection buffer and fed back to the next reflect step so the optimizer doesn't propose the same patch twice.

text

rollout      pull recent learnings from SQLite (existing learn-rule rows)
reflect      optimizer LLM analyzes a minibatch, proposes add/delete/replace patches
aggregate    vote-merge patches across minibatches
select       clip by LR budget (default: 3 adds, 2 deletes, 3 replaces per step)
update       apply selected patches to a candidate skill content
evaluate     evaluator LLM scores candidate against held-out validation items
gate         accept candidate only if weighted score >= current + acceptThreshold
slow update  at epoch boundary, consolidate accepted edits into a coherent rewrite

失败的候选项会存储在 rejection buffer 中，并反馈给下一次 reflect 步骤，避免优化器重复提出相同的补丁。

Run it

运行

In a plugin session, use the

providers

MCP server's

run_provider_task

tool with

task: "optimizer"

and

args: ["--slug", "<slug>", "--budget-usd", "0.50"]

. Keys come from the plugin configuration dialog. Standalone CLI installations use explicit

PRO_WORKFLOW_*_API_KEY

variables. See provider configuration; never request keys in chat or retrieve existing machine credentials.

bash

/skill-optimize <slug> [options]

Options (all optional; sensible defaults shown):

Flag	Default	Notes
`--epochs N`	3	Outer loop count
`--batch-size N`	8	Trajectories per minibatch
`--minibatches N`	2	Minibatches per epoch
`--holdout N`	6	Validation items reserved (max ~25% of trajectories)
`--budget-usd X`	0.50	Hard cap; loop aborts when spent
`--optimizer-model M`	`claude-sonnet-5`	Reflect + slow-update model
`--evaluator-model M`	`claude-haiku-4-5`	Gate model (cheaper)
`--max-adds N`	3	LR budget per step
`--max-deletes N`	2
`--max-replaces N`	3
`--accept-threshold X`	0.0	Minimum score delta to accept candidate
`--max-skill-tokens N`	2000	Hard cap on candidate length
`--slow-every N`	2	Epochs between consolidation passes
`--json`	off	Machine-readable output

Kill switch:

touch ~/.pro-workflow/STOP

aborts the loop between steps.

在插件会话中，使用

providers

MCP server 的

run_provider_task

工具，并设置

task: "optimizer"

和

args: ["--slug", "<slug>", "--budget-usd", "0.50"]

。密钥来自插件配置对话框。独立 CLI 安装需要显式设置

PRO_WORKFLOW_*_API_KEY

环境变量。请参阅 provider 配置；绝不要在聊天中索取密钥，也不要读取机器上已有的凭据。

bash

/skill-optimize <slug> [options]

所有选项均为可选；下表列出了合理的默认值：

标志	默认值	说明
`--epochs N`	3	外层循环次数
`--batch-size N`	8	每个 minibatch 的训练轨迹数
`--minibatches N`	2	每个 epoch 的 minibatch 数
`--holdout N`	6	预留的验证项数量（最多约占训练轨迹的 25%）
`--budget-usd X`	0.50	硬性上限；达到预算后中止循环
`--optimizer-model M`	`claude-sonnet-5`	用于 reflect + slow update 的模型
`--evaluator-model M`	`claude-haiku-4-5`	门控模型（成本更低）
`--max-adds N`	3	每步的 LR 预算
`--max-deletes N`	2
`--max-replaces N`	3
`--accept-threshold X`	0.0	接受候选项所需的最低分数增量
`--max-skill-tokens N`	2000	候选项长度的硬性上限
`--slow-every N`	2	执行 consolidation 之间的 epoch 间隔
`--json`	off	机器可读输出

紧急停止开关：执行

touch ~/.pro-workflow/STOP

可在步骤之间中止循环。

Output

输出

Candidate accepted → SKILL.md overwritten, hash stamp appended in HTML comment

Run details persist in

optimization_runs

optimization_candidates

optimization_patches

optimization_rejections

Validation set persists in
```
optimization_validation
```
(reusable across runs)

Inspect after:

bash

sqlite3 ~/.pro-workflow/data.db "SELECT id, skill_slug, initial_score, best_score, accepted_steps, rejected_steps, spent_usd FROM optimization_runs ORDER BY id DESC LIMIT 5"

候选项被接受 → 覆盖 SKILL.md，并以 HTML 注释形式追加哈希标记

运行详情持久化到

optimization_runs

、

optimization_candidates

、

optimization_patches

、

optimization_rejections

验证集持久化到
```
optimization_validation
```
（可跨运行复用）

运行后检查：

bash

sqlite3 ~/.pro-workflow/data.db "SELECT id, skill_slug, initial_score, best_score, accepted_steps, rejected_steps, spent_usd FROM optimization_runs ORDER BY id DESC LIMIT 5"

Rules

规则

Validation set is frozen at run start. Never re-derive from new corrections mid-run.
One candidate per step. No parallel branches.
Slow-update output is itself a candidate; it must pass the gate to replace the best.
The optimizer LLM and evaluator LLM may be different models. Mixing a strong optimizer with a cheap evaluator is the SkillOpt-recommended config.

spent_usd >= budget_usd

at any step boundary, the loop ends with

stopped_reason="budget exhausted"

Patches whose anchor is no longer present in the skill (because a prior patch in the same step removed it) are recorded as rejected with reason
```
anchor_missing
```
.

验证集在运行开始时冻结。运行期间绝不能根据新的修正重新推导验证集。
每步仅生成一个候选项。不使用并行分支。
slow update 的输出本身也是候选项；必须通过门控才能替换当前最佳版本。
优化器 LLM 与评估器 LLM 可以使用不同模型。将强大的优化器与低成本评估器结合，是 SkillOpt 推荐的配置。
如果在任意步骤边界处出现
```
spent_usd >= budget_usd
```
，循环将以
```
stopped_reason="budget exhausted"
```
结束。
如果同一步骤中的先前补丁已移除某个补丁所依赖的锚点（anchor），该补丁会记录为 rejected，原因为
```
anchor_missing
```
。

Provenance

来源说明

Inspired by Microsoft SkillOpt (arXiv:2605.23904). The six-stage rollout/reflect/aggregate/select/update/evaluate pipeline, LR budget, rejection buffer, and slow / meta update mechanics are adapted to pro-workflow's existing SQLite + learn-rule data plane. No SkillOpt code is reused. "ReflACT" is not a SkillOpt term and is not used here; the loop is referred to by stage names only.

灵感来自 Microsoft SkillOpt（arXiv:2605.23904）。六阶段 rollout/reflect/aggregate/select/update/evaluate 流水线、LR 预算、rejection buffer，以及 slow / meta update 机制均适配到了 pro-workflow 现有的 SQLite + learn-rule 数据平面。未复用任何 SkillOpt 代码。“ReflACT”不是 SkillOpt 术语，此处也不使用；该循环仅以阶段名称称呼。