cc-canary

Original：🇺🇸 English

Translated

1 scripts

Produce a long-form, shareable markdown writeup on whether Claude has regressed on this user's work. A bundled Python script scans `~/.claude/projects/`, computes every metric, and renders a markdown skeleton with tables already filled — in ~2.5s. Claude fills a dozen short narrative placeholders and saves. Writes `./cc-canary-<YYYY-MM-DD>.md` suitable for pasting into a GitHub issue or gist.

3installs

Sourcedelta-hq/cc-canary

Added on2026-04-25

NPX Install

npx skill4agent add delta-hq/cc-canary cc-canary

SKILL.md Content

View Translation Comparison →

cc-canary — long-form regression writeup

Primary question: has Claude regressed on this user's work, and when?

Bundled script

scripts/compute_stats.py

does ~95% of the work in ~2.5s: scans JSONLs, runs inflection + transition-day detection, builds pre/post aggregates, cross-version comparison, hour-of-day, word frequency, three- period thinking depth, visibility transition, per-turn rates, and abnormalities — then renders a complete markdown skeleton with every table filled. Narrative slots are marked

<!-- C: ... -->

.

Default window:

60d

. Accept

7d / 14d / 30d / 60d / 90d / 180d

.

Framing — three-bucket classification

Model-side — same user/task, worse outcomes; cross-version worse; reasoning-depth dropping
User-side — project-mix shift; shorter/imperative prompts; new codebase; shortcut vocab rising
Ambiguous — mixed confounds, borderline effect size, either-way-explainable

Your 3-step job

1. Run the script

Bash(python3 <SKILL_DIR>/scripts/compute_stats.py --window {window} --render-md /tmp/cc-canary-skeleton-{window}.md > /dev/null 2>&1)

<SKILL_DIR>

:

.claude/skills/cc-canary/

→ fallback to

~/.claude/skills/cc-canary/

.

Flags:

--window {Nd}

(required);

--include-agents

(include subagent sessions);

--min-user-words N

(default 10).

If the script fails: report error, retry once with

--include-agents

, else stop. Never fall back to hand-computation — that's the slow path.

2. Read the skeleton

Read /tmp/cc-canary-skeleton-{window}.md

3. Fill every

<!-- C: ... -->

placeholder and save

Write ./cc-canary-{YYYY-MM-DD}.md

End your message with the absolute path:

Wrote /Users/.../cc-canary-{date}.md · paste-ready.

Narrative placeholders

Each placeholder's inline comment already spells out what to write. Summary:

```
verdict-line
```
— HOLDING / SUSPECTED REGRESSION / CONFIRMED REGRESSION / INCONCLUSIVE + brief justification
```
summary
```
— 2–4 sentences: verdict + inflection + biggest pre→post delta. No counter-evidence hedging.
```
timeline
```
— 1–2 paragraphs on the daily series shape
```
xv-para
```
— 1 paragraph on cross-version (if §3 present)
```
finding-N-class
```
× up to 5 — inline classification label
```
finding-N-reason
```
× up to 5 — 2–3 sentences max, evidence-first. Lead with the strongest number (cross-version Δ, §2 value, appendix rate). No signal-line restating, no rhetorical buildup
```
root-cause
```
— 3–5 paragraphs tying strongest signals together
```
what-would-help
```
— 2–4 concrete bullets
```
appendix-a1…a4, b, c, d, e, f, g, h
```
— 1 paragraph each (see table context in skeleton)
```
meta-note
```
— 2–5 sentences, first person, honest, no claimed feelings, acknowledge the recursion

Verdict calibration

HOLDING: ≤1 model-side signal
SUSPECTED REGRESSION: 2–3 model-side signals
CONFIRMED REGRESSION: ≥3 model-side signals + non-empty cross-version showing decline +
```
session_count ≥ 15
```
+ ≥2 models +
```
inflection.gap_sigma ≥ 1.0
```

INCONCLUSIVE:

session_count < 15

OR

inflection.method == "fallback_split_half"

with overlapping confounds

Cap at SUSPECTED when: only one model; <15 sessions; single-project with project starting mid-window; inflection coincides with a visible user-side event.

All the data you need (session_count, model mix, inflection method, cross-version presence) is rendered as plain text in the skeleton.

Grounding example (for

finding-N-reason

)

Classification: model-side. Read:Edit dropped 9.0 → 1.0 (-89%, concerning) while cross-version shows opus-4-7 at 0.39 vs opus-4-6 at 1.00 on the same user's workload. No project-mix shift near the inflection — model is defaulting to edit-first.

Hard rules

Never read, grep, or glob
```
~/.claude/projects/**/*.jsonl
```
. Never run
```
jq
```
/
```
awk
```
/
```
wc
```
on session files. Script owns all that.
Never touch tables or numbers — they came from real data.
Every finding gets a classification label.
Hedge when cross-version is empty or
```
session_count < 15
```
.
Do not verdict CONFIRMED REGRESSION without the full checklist.
Do not save the skeleton as-is — replace every
```

```
first.

Failure modes

Script import error → check
```
python3 -V
```
≥ 3.8; retry once with
```
--include-agents
```
; else stop.
Skeleton < 5KB → likely no sessions in window. Check script error.

inflection.method == fallback_split_half

→ state it; cap at SUSPECTED.

Cross-version Δ
```
None
```
→ div-by-zero when model-A value is 0; note the confound.

cc-canary

NPX Install

Tags

SKILL.md Content

cc-canary — long-form regression writeup

Framing — three-bucket classification

Your 3-step job

1. Run the script

2. Read the skeleton

3. Fill every
``
placeholder and save

Narrative placeholders

Verdict calibration

Grounding example (for
`finding-N-reason`
)

Hard rules

Failure modes

cc-canary

NPX Install

Tags

SKILL.md Content

cc-canary — long-form regression writeup

Framing — three-bucket classification

Your 3-step job

1. Run the script

2. Read the skeleton

3. Fill every  placeholder and save

Narrative placeholders

Verdict calibration

Grounding example (for finding-N-reason)

Hard rules

Failure modes

3. Fill every
``
placeholder and save

Grounding example (for
`finding-N-reason`
)