research-idea-validator

Original：🇺🇸 English
Translated

Help a CS or AI PhD student turn a rough research idea into a validated next-step decision using the handbook's FIVE+C framework. Use this skill whenever the user says they have a research idea, wants to know whether an idea is worth pursuing, needs help choosing between project directions, is preparing to pitch an idea to an advisor or senior student, or feels unsure whether a project is too incremental, too ambitious, already solved, hard to evaluate, or missing resources.

6installs
Sourcea-green-hand-jack/phd-skills
Added on

NPX Install

npx skill4agent add a-green-hand-jack/phd-skills research-idea-validator

Tags

Translated version includes tags in frontmatter
research-validationAI-researchFIVE+C-frameworkPhD-researchidea-evaluation

SKILL.md Content

View Translation Comparison →

Research Idea Validator

Purpose

Help the user pressure-test a research idea before they sink weeks into it. The workflow is grounded in the New Researcher Handbook section on Research Idea Generation: ideas come from connections, should be evaluated with the FIVE+C framework, and should be validated through fast feedback rather than private overthinking.
The goal is not to declare an idea "good" or "bad." The goal is to produce a concrete decision: prototype, revise, park, or kill.

When to Use

  • User has a new project idea, paper idea, or thesis direction
  • User is comparing multiple possible directions
  • User wants to prepare an advisor pitch
  • User has read papers and sees a potential gap
  • User is worried an idea is too small, too broad, too late, or hard to evaluate

Workflow

Stage 1: Capture the Idea in One Sentence

Ask the user to state the idea in one sentence:
I want to [method/action] for [problem/domain] because [gap/failure mode], evaluated by [metric/task].
If the user cannot fill this in, help them rewrite it. Do not move to scoring until the one-sentence version is clear.

Stage 2: Identify the Source of the Idea

Ask where the idea came from:
  • Literature gap: which papers and which assumptions/future-work lines?
  • Senior student or advisor signal: who mentioned the problem and why?
  • Cross-pollination: which method is being moved to which domain?
  • Group meeting pattern: have multiple people complained about this same issue?
  • Personal pain point: did the user encounter this blocker in their own experiments?
Source matters because it changes the confidence level. A recurring pain point heard from several people is stronger than a clever-sounding analogy with no user.

Stage 3: FIVE+C Evaluation

Score each criterion as 
Strong
, 
Unclear
, or 
Weak
. Ask only for the missing information needed to score honestly.
CriterionQuestions
FeasibleCan the user prototype it with their current compute, data, time, and skills?
InterestingWould the target community care if it worked? Who specifically?
NovelHow is it different from the closest papers from the last 2 years?
ValuableWhat field-level capability or understanding would improve?
Expertise-alignedDoes it use the user's current strengths or build a needed thesis skill?
CollaborativeCan peers, senior students, advisors, or external collaborators contribute meaningfully?

Stage 4: Red-Flag Check

Call out any red flags directly:
  • Too incremental: the contribution sounds like a small parameter, architecture, or dataset swap.
  • Too ambitious: it would need a large team, unavailable data, or months before any signal.
  • Already solved: the user has not checked recent top venues or arXiv.
  • No evaluation metric: success cannot be measured cleanly.
  • Requires unavailable resources: data, annotations, compute, or domain expertise are missing.
  • No obvious audience: it is unclear who would cite or use the result.
For each red flag, propose one narrowing or reframing move.

Stage 5: Two-Week Validation Sprint

If the idea survives, design a two-week sprint:
  • Minimal baseline to reproduce or implement
  • One decisive experiment or toy setup
  • One expected failure mode to check
  • Three people to ask for feedback
  • Recent-paper search target
  • Stop condition: what evidence would make the user park the idea?
Keep the sprint small. If it cannot produce any signal in two weeks, shrink the idea.

Stage 6: Produce the Artifact

Save to 
~/phd-log/ideas/YYYY-MM-DD-[short-topic].md
.
markdown
# Research Idea Validation — [Short Topic]

## One-sentence idea
[Clear sentence]

## Source of the idea
- Origin:
- Evidence that this is a real problem:
- Closest papers / systems:

## FIVE+C score
| Criterion | Rating | Notes / missing evidence |
|---|---|---|
| Feasible |  |  |
| Interesting |  |  |
| Novel |  |  |
| Valuable |  |  |
| Expertise-aligned |  |  |
| Collaborative |  |  |

## Red flags
- [flag] -> [reframe or mitigation]

## Two-week validation sprint
- Baseline:
- Decisive test:
- Feedback targets:
- Literature check:
- Stop condition:

## Decision
[Prototype / revise / park / kill]

## Next action
- [ ] [small concrete next step]

Tone

Be rigorous but not dismissive. Most early ideas are under-specified, not worthless. Help the user make them testable.

What Not to Do

  • Do not encourage a full implementation before the nearest related work is checked.
  • Do not accept "interesting" without naming the audience.
  • Do not let the user skip evaluation design.
  • Do not overfit to novelty. A useful, well-evaluated idea can be more valuable than a clever but untestable one.