Vulnerability Validation

Validate security findings by assessing whether they are actually exploitable in the context of this codebase. This skill filters false positives, confirms real vulnerabilities, and generates proof-of-concept exploits.

When to Use This Skill

After commit-security-scan - Validate findings before creating issues or blocking PRs
HIGH/CRITICAL findings - Prioritize validation of severe findings
Before patching - Confirm vulnerability is real before investing in fixes
Security review - Deep-dive validation of specific findings

Prerequisites

.factory/threat-model.md

must exist (from

threat-model-generation

skill)

security-findings.json

must exist (from

commit-security-scan

skill)

Inputs

Input	Description	Required	Default
Findings file	Path to `security-findings.json`	Yes	`security-findings.json`
Threat model	Path to threat model	No	`.factory/threat-model.md`
Finding IDs	Specific findings to validate (comma-separated)	No	All findings
Severity filter	Only validate findings at or above this severity	No	All severities

Instructions

Follow these steps for each finding to validate:

Step 1: Load Context

Read

security-findings.json

from

commit-security-scan

Read
```
.factory/threat-model.md
```
for system context
Identify which findings to validate based on inputs

Step 2: Reachability Analysis

For each finding, determine if the vulnerable code is reachable:

Trace entry points
- Can external users reach this code path?
- What HTTP endpoints, CLI commands, or event handlers lead here?
- Is authentication required to reach this code?
Map the call chain
- Starting from the entry point, trace the path to the vulnerable code
- Document each function call in the chain
- Note any branching conditions that must be satisfied
Classify reachability
- ```
EXTERNAL
```
  - Reachable from unauthenticated external input
- ```
AUTHENTICATED
```
  - Requires valid user session
- ```
INTERNAL
```
  - Only reachable from internal services
- ```
UNREACHABLE
```
  - Dead code or blocked by conditions

Step 3: Control Flow Analysis

Determine if an attacker can control the vulnerable input:

Identify the source
- Where does the tainted data originate?
- HTTP parameter, file upload, database query, environment variable?
Trace data flow
- Follow the data from source to sink (vulnerable function)
- Document each transformation or validation step
- Note any sanitization, encoding, or type conversion
Assess attacker control
- Can the attacker fully control the input?
- Are there length limits, character restrictions, or format validation?
- Does the data pass through any sanitization?

Step 4: Mitigation Assessment

Check if existing security controls prevent exploitation:

Input validation
- Is the input validated before reaching the vulnerable code?
- What validation rules are applied?
Framework protections
- Does the framework provide automatic protection? (e.g., ORM parameterization, React XSS escaping)
- Is the protection enabled and properly configured?
Security middleware
- Are there WAF rules, rate limiting, or other controls?
- Do CSP headers or other browser protections apply?
Reference threat model
- Check the "Existing Mitigations" section for this threat type
- Verify mitigations are actually in place

Step 5: Exploitability Assessment

Determine how difficult it is to exploit:

Rating	Criteria
`EASY`	No special conditions, standard tools, publicly known technique
`MEDIUM`	Requires specific conditions, timing, or chained vulnerabilities
`HARD`	Requires insider knowledge, rare conditions, or advanced techniques
`NOT_EXPLOITABLE`	Theoretical vulnerability but not practically exploitable

Consider:

Attack complexity
Required privileges
User interaction needed
Scope of impact

Step 6: Generate Proof-of-Concept

For confirmed vulnerabilities, create a proof-of-concept:

Craft exploit payload
- Create a minimal payload that demonstrates the vulnerability
- Use benign payloads (no actual damage)
Document the request
- HTTP method, URL, headers, body
- Or CLI command, file input, etc.
Describe expected vs actual behavior
- What should happen (secure behavior)
- What actually happens (vulnerable behavior)

Example PoC structure:

json

{
  "payload": "' OR '1'='1",
  "request": "GET /api/users?search=' OR '1'='1",
  "expected_behavior": "Returns users matching search term",
  "actual_behavior": "Returns all users due to SQL injection"
}

Step 7: Calculate CVSS Score

Assign a CVSS 3.1 score based on:

Metric	Options
Attack Vector (AV)	Network (N), Adjacent (A), Local (L), Physical (P)
Attack Complexity (AC)	Low (L), High (H)
Privileges Required (PR)	None (N), Low (L), High (H)
User Interaction (UI)	None (N), Required (R)
Scope (S)	Unchanged (U), Changed (C)
Confidentiality (C)	None (N), Low (L), High (H)
Integrity (I)	None (N), Low (L), High (H)
Availability (A)	None (N), Low (L), High (H)

Example:

CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N

= 9.1 (Critical)

Step 8: Classify Finding

Based on analysis, classify each finding:

Status	Meaning
`CONFIRMED`	Vulnerability is real and exploitable
`LIKELY`	Probably exploitable but couldn't fully verify
`FALSE_POSITIVE`	Not actually a vulnerability (document why)
`NEEDS_MANUAL_REVIEW`	Requires human security expert review

Step 9: Generate Output

Create

validated-findings.json

json

{
  "validation_id": "val-<timestamp>",
  "validation_date": "<ISO timestamp>",
  "scan_id": "<from security-findings.json>",
  "threat_model_version": "<from threat-model.md>",
  "validated_findings": [
    {
      "id": "VULN-001",
      "status": "CONFIRMED",
      "original_severity": "HIGH",
      "validated_severity": "HIGH",
      "exploitability": "EASY",
      "reachability": "EXTERNAL",
      "existing_mitigations": [],
      "exploitation_path": [
        "User submits search query via GET /api/users?search=<payload>",
        "Express router passes query to searchUsers() handler",
        "Handler passes unsanitized input to SQL template literal",
        "PostgreSQL executes malicious SQL"
      ],
      "proof_of_concept": {
        "payload": "' OR '1'='1",
        "request": "GET /api/users?search=' OR '1'='1",
        "expected_behavior": "Returns users matching search term",
        "actual_behavior": "Returns all users due to SQL injection"
      },
      "cvss_vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
      "cvss_score": 9.1,
      "validation_notes": "Confirmed via code tracing. No input validation or parameterization."
    }
  ],
  "false_positives": [
    {
      "id": "VULN-003",
      "original_severity": "MEDIUM",
      "reason": "Input is validated by Joi schema in middleware before reaching this code. Schema enforces UUID format which prevents injection.",
      "evidence": "See src/middleware/validation.js:45 - Joi.string().uuid()"
    }
  ],
  "needs_manual_review": [
    {
      "id": "VULN-005",
      "original_severity": "HIGH",
      "reason": "Complex data flow through message queue. Unable to fully trace if sanitization occurs in consumer service."
    }
  ],
  "summary": {
    "total_analyzed": 10,
    "confirmed": 5,
    "likely": 2,
    "false_positives": 2,
    "needs_manual_review": 1,
    "by_severity": {
      "CRITICAL": 1,
      "HIGH": 3,
      "MEDIUM": 1,
      "LOW": 0
    }
  }
}

Success Criteria

The skill is complete when:

All specified findings have been analyzed
Each finding has a status (CONFIRMED, LIKELY, FALSE_POSITIVE, NEEDS_MANUAL_REVIEW)
Confirmed findings have exploitation paths documented
Confirmed findings have proof-of-concept exploits
False positives have clear reasoning
```
validated-findings.json
```
is valid JSON
CVSS scores are calculated for confirmed findings

Verification

Run these checks before completing:

bash

# Verify output exists and is valid JSON
cat validated-findings.json | jq . > /dev/null && echo "✓ Valid JSON"

# Check all findings have status
jq '.validated_findings | all(.status)' validated-findings.json

# Check confirmed findings have PoC
jq '.validated_findings | map(select(.status == "CONFIRMED")) | all(.proof_of_concept)' validated-findings.json

# Check false positives have reasoning
jq '.false_positives | all(.reason)' validated-findings.json

Example Invocations

Validate all findings:

Validate the security findings from the last scan.

Validate specific findings:

Validate findings VULN-001 and VULN-002 from security-findings.json.

Validate only HIGH/CRITICAL:

Validate all HIGH and CRITICAL severity findings from the security scan.

Validate with specific files:

Validate findings in security-findings.json using threat model at .factory/threat-model.md.

References

CVSS 3.1 Calculator: https://www.first.org/cvss/calculator/3.1
OWASP Testing Guide: https://owasp.org/www-project-web-security-testing-guide/
Examples:
```
validation-examples.md
```
(in this skill directory)
Upstream:
```
commit-security-scan
```
skill
Downstream:
```
security-patch-generation
```
skill

vulnerability-validation

NPX Install

Tags

SKILL.md Content