If you see unfamiliar placeholders or need to check which tools are connected, see CONNECTORS.md.

/validate-data <analysis to review>

• A document or report in the conversation
• A file (markdown, notebook, spreadsheet)
• SQL queries and their results
• Charts and their underlying data
• A description of methodology and findings

• Question framing: Is the analysis answering the right question? Could the question be interpreted differently?
• Data selection: Are the right tables/datasets being used? Is the time range appropriate?
• Population definition: Is the analysis population correctly defined? Are there unintended exclusions?
• Metric definitions: Are metrics defined clearly and consistently? Do they match how stakeholders understand them?
• Baseline and comparison: Is the comparison fair? Are time periods, cohort sizes, and contexts comparable?

• Recalculate a few key numbers independently
• Verify that subtotals sum to totals
• Check that percentages sum to 100% (or close to it) where expected
• Confirm that YoY/MoM comparisons use the correct base periods
• Validate that filters are applied consistently across all metrics

• Do axes start at appropriate values (zero for bar charts)?
• Are scales consistent across comparison charts?
• Do chart titles accurately describe what's shown?
• Could the visualization mislead a quick reader?
• Are there truncated axes, inconsistent intervals, or 3D effects that distort perception?

• Conclusions are supported by the data shown
• Alternative explanations are acknowledged
• Uncertainty is communicated appropriately
• Recommendations follow logically from findings
• The level of confidence matches the strength of evidence

• Additional analyses that would strengthen the conclusions
• Caveats or limitations that should be noted
• Better visualizations or framings for key points
• Missing context that stakeholders would want

1. [Severity: High/Medium/Low] [Issue description and impact]
2. ...

• [Metric]: [Verified / Discrepancy found]
• ...

1. [Improvement and why it matters]
2. ...

• [Caveat that must be communicated]
• ...

---

• Source verification: Confirmed which tables/data sources were used. Are they the right ones for this question?
• Freshness: Data is current enough for the analysis. Noted the "as of" date.
• Completeness: No unexpected gaps in time series or missing segments.
• Null handling: Checked null rates in key columns. Nulls are handled appropriately (excluded, imputed, or flagged).
• Deduplication: Confirmed no double-counting from bad joins or duplicate source records.
• Filter verification: All WHERE clauses and filters are correct. No unintended exclusions.

• Aggregation logic: GROUP BY includes all non-aggregated columns. Aggregation level matches the analysis grain.
• Denominator correctness: Rate and percentage calculations use the right denominator. Denominators are non-zero.
• Date alignment: Comparisons use the same time period length. Partial periods are excluded or noted.
• Join correctness: JOIN types are appropriate (INNER vs LEFT). Many-to-many joins haven't inflated counts.
• Metric definitions: Metrics match how stakeholders define them. Any deviations are noted.
• Subtotals sum: Parts add up to the whole where expected. If they don't, explain why (e.g., overlap).

• Magnitude: Numbers are in a plausible range. Revenue isn't negative. Percentages are between 0-100%.
• Trend continuity: No unexplained jumps or drops in time series.
• Cross-reference: Key numbers match other known sources (dashboards, previous reports, finance data).
• Order of magnitude: Total revenue is in the right ballpark. User counts match known figures.
• Edge cases: What happens at the boundaries? Empty segments, zero-activity periods, new entities.

• Chart accuracy: Bar charts start at zero. Axes are labeled. Scales are consistent across panels.
• Number formatting: Appropriate precision. Consistent currency/percentage formatting. Thousands separators where needed.
• Title clarity: Titles state the insight, not just the metric. Date ranges are specified.
• Caveat transparency: Known limitations and assumptions are stated explicitly.
• Reproducibility: Someone else could recreate this analysis from the documentation provided.

-- Check row count before and after join
SELECT COUNT(*) FROM table_a;  -- 1,000
SELECT COUNT(*) FROM table_a a JOIN table_b b ON a.id = b.a_id;  -- 3,500 (uh oh)

• Always check row counts after joins
• If counts increase, investigate the join relationship (is it really 1:1 or 1:many?)
• Use

  COUNT(DISTINCT a.id)

  instead of

  COUNT(*)

  when counting entities through joins

• Analyzing user behavior of "current users" misses churned users
• Looking at "companies using our product" ignores those who evaluated and left
• Studying properties of "successful" outcomes without "unsuccessful" ones

• "January revenue is $500K vs. December's $800K" -- but January isn't over yet
• "This week's signups are down" -- checked on Wednesday, comparing to a full prior week

• Conversion rate improves because you changed how you count "eligible" users
• Churn rate changes because the definition of "active" was updated

• Group A: 100 users, average revenue $50
• Group B: 10 users, average revenue $200
• Wrong: Average of averages = ($50 + $200) / 2 = $125
• Right: Weighted average = (100*$50 + 10*$200) / 110 = $63.64

• Event timestamps in UTC vs. user-facing dates in local time
• Daily rollups that use different cutoff times

• "Users who completed onboarding have higher retention" -- obviously, they self-selected
• "Power users generate more revenue" -- they became power users BY generating revenue

• Simpson's paradox: Trend reverses when data is aggregated vs. segmented
• Correlation presented as causation without supporting evidence
• Small sample sizes leading to unreliable conclusions
• Outliers disproportionately affecting averages (should medians be used instead?)
• Multiple testing / cherry-picking significant results
• Look-ahead bias: Using future information to explain past events
• Cherry-picked time ranges that favor a particular narrative

1. Calculate the same metric two different ways and verify they match
2. Spot-check individual records -- pick a few specific entities and trace their data manually
3. Compare to known benchmarks -- match against published dashboards, finance reports, or prior analyses
4. Reverse engineer -- if total revenue is X, does per-user revenue times user count approximately equal X?
5. Boundary checks -- what happens when you filter to a single day, a single user, or a single category? Are those micro-results sensible?

• Any metric that changed by more than 50% period-over-period without an obvious cause
• Counts or sums that are exact round numbers (suggests a filter or default value issue)
• Rates exactly at 0% or 100% (may indicate incomplete data)
• Results that perfectly confirm the hypothesis (reality is usually messier)
• Identical values across time periods or segments (suggests the query is ignoring a dimension)

• Table: [schema.table_name] (as of [date])
• Table: [schema.other_table] (as of [date])
• File: [filename] (source: [where it came from])

• [Metric A]: [Exactly how it's calculated]
• [Segment X]: [Exactly how membership is determined]
• [Time period]: [Start date] to [end date], [timezone]

1. [Step 1 of the analysis approach]
2. [Step 2]
3. [Step 3]

• [Assumption 1 and why it's reasonable]
• [Limitation 1 and its potential impact on conclusions]

1. [Finding 1 with supporting evidence]
2. [Finding 2 with supporting evidence]

• [Things the reader should know before acting on this]

undefined

• Save queries and code in version control (git) or a shared docs system
• Note the date of the data snapshot used
• If an analysis is re-run with updated data, document what changed and why
• Link to prior versions of recurring analyses for trend comparison

• Run /validate-data before any high-stakes presentation or decision
• Even quick analyses benefit from a sanity check -- it takes a minute and can save your credibility
• If the validation finds issues, fix them and re-validate
• Share the validation output alongside your analysis to build stakeholder confidence