Skill Optimizer

When to use

• A pro-workflow skill has accumulated 8+ learn-rule rows for it
• The user reports the skill is "getting bloated" or "rules keep being repeated"
• The user wants offline, budget-capped improvement over multiple sessions

• Skill has fewer than 8 trajectories (nothing to learn from)
• The user wants real-time edits (this is offline, single-shot)
• No key has been explicitly configured for the selected provider

Architecture (mirrors SkillOpt's six-stage loop)

rollout      pull recent learnings from SQLite (existing learn-rule rows)
reflect      optimizer LLM analyzes a minibatch, proposes add/delete/replace patches
aggregate    vote-merge patches across minibatches
select       clip by LR budget (default: 3 adds, 2 deletes, 3 replaces per step)
update       apply selected patches to a candidate skill content
evaluate     evaluator LLM scores candidate against held-out validation items
gate         accept candidate only if weighted score >= current + acceptThreshold
slow update  at epoch boundary, consolidate accepted edits into a coherent rewrite

Run it

providers

run_provider_task

task: "optimizer"

args: ["--slug", "<slug>", "--budget-usd", "0.50"]

PRO_WORKFLOW_*_API_KEY

/skill-optimize <slug> [options]

--epochs N

--batch-size N

--minibatches N

--holdout N

--budget-usd X

--optimizer-model M

claude-sonnet-5

--evaluator-model M

claude-haiku-4-5

--max-adds N

--max-deletes N

--max-replaces N

--accept-threshold X

--max-skill-tokens N

--slow-every N

--json

touch ~/.pro-workflow/STOP

Output

• Candidate accepted → SKILL.md overwritten, hash stamp appended in HTML comment
• Run details persist in

  optimization_runs

  ,

  optimization_candidates

  ,

  optimization_patches

  ,

  optimization_rejections

• Validation set persists in

  optimization_validation

  (reusable across runs)

sqlite3 ~/.pro-workflow/data.db "SELECT id, skill_slug, initial_score, best_score, accepted_steps, rejected_steps, spent_usd FROM optimization_runs ORDER BY id DESC LIMIT 5"

Rules

• Validation set is frozen at run start. Never re-derive from new corrections mid-run.
• One candidate per step. No parallel branches.
• Slow-update output is itself a candidate; it must pass the gate to replace the best.
• The optimizer LLM and evaluator LLM may be different models. Mixing a strong optimizer with a cheap evaluator is the SkillOpt-recommended config.
• If

  spent_usd >= budget_usd

  at any step boundary, the loop ends with

  stopped_reason="budget exhausted"

  .
• Patches whose anchor is no longer present in the skill (because a prior patch in the same step removed it) are recorded as rejected with reason

  anchor_missing

  .

Provenance