Skill Evaluation

Read First

• references/skill-evaluation-policy.md

• references/external-skill-recommendations.md

Workflow

1. Define the failure mode the skill should prevent.
2. Write pressure scenarios using real academic tasks: bad PDFs, missing DOI, unsupported claim, messy repo, ambiguous SOTA, unreliable notebook, or MCP outage.
3. Run or reason through the baseline behavior without assuming the skill helps.
4. Revise the skill frontmatter so it triggers on the right user intents.
5. Keep

   SKILL.md

   lean; move detailed policies to

   references/

   .
6. Validate local structure with

   python3 scripts/validate_skills.py

   .
7. Install-list the package with

   npx -y skills add <repo> --list

   .
8. Record recommended external skills separately from custom internal skills.

Review Criteria

• trigger description starts with

  Use when

• no workflow summary in frontmatter
• references exist and are loaded only when needed
• outputs map to the research project contract
• citations and evidence rules are explicit
• skill does not duplicate a stronger external skill without a reason

Do Not

• Create one giant "research agent" skill.
• Hide fragile procedures in prose when a script or test can check them.
• Copy external skills blindly without adapting output paths and repository contracts.