🚨 Incident Responder Master Kit

📑 Internal Menu

1. Incident Management Lifecycle
2. Smart Diagnosis & Rapid Fix
3. Runbook Execution & Automation
4. Communication & Stakeholder Management
5. Blameless Post-Mortems & Learning

1. Incident Management Lifecycle

• Detection: Use SLI/SLO alerts to identify issues.
• Triage: Determine severity (P0, P1, P2) and impact.
• Declaration: Declare the incident and assign roles (Commander, Comms, Ops).
• Resolution: Mitigate the symptoms first, solve the root cause second.

2. Smart Diagnosis & Rapid Fix

• Hypothesis Loop: Investigate logs, traces, and metrics to form a hypothesis.
• Verification: Test the hypothesis with safe, reversible actions.
• Fix: Rollback if the last deployment was the culprit, or apply a hotfix. Safety first.

3. Runbook Execution & Automation

• Standard Operating Procedures (SOPs): Follow pre-defined runbooks for common issues (DB Overload, Redis crash).
• Automation: Script repetitive recovery tasks.
• Validation: After mitigation, run smoke tests to ensure service stability.

4. Communication & Stakeholder Management

• Internal: Provide regular updates (every 15-30 mins) to the team.
• External: Update Status Page for customers.
• Clarity: Use clear language (e.g., "Investigating DB latency" vs "The app is down").

5. Blameless Post-Mortems & Learning

• Blameless Culture: Focus on "How" and "Why" the system failed, not "Who" made the mistake.
• Timeline: Document exactly what happened and when.
• Action Items: Define specific, trackable items to prevent recurrence.

🛠️ Execution Protocol

1. Check System Health: Run a quick diagnostic of the target service.
   bash

   python .agent/skills/incident-responder/scripts/health_check.py http://localhost:3000

2. Isolate Issue: Map the failure to specific logs or metrics.
3. Remediate: Apply the fix and verify system stability.
4. Step 5: Document: Start the Post-Mortem.

🧠 Knowledge Modules (Fractal Skills)

1. incident_severity_levels