Loading...
Loading...
Found 4 Skills
LLM-as-judge for a coding-agent session or an eval-harness result — checks controller attestations first, then project guards, code review, and the repo's design contract, and names the smallest harness owner to fix. Read-only; never executes session content. Use for "judge this session", "analyze this eval", "oceń sesję/agenta".
Turn a real agent failure into one reproducible eval-harness case plus the smallest knowledge change — failing case first, one knowledge owner, semantic assertions, before/after evidence through the repo's configured harness commands. Use for "add harness case", "agent got this wrong", "extend the harness", "rozszerz harness".
Design lightweight eval harnesses for coding or workflow agents, especially when testing whether agents follow steering docs, use tools correctly, preserve user edits, recover from failures, and produce the intended artifact.
Build the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.