Prepare Artifacts

Turn a research codebase into a submittable, badge-ready reproducibility artifact. Artifact evaluation is a separate, post-acceptance track at most systems/PL/ML venues with its own deadline, its own appendix, and badges that change per venue per year — this skill builds the package (README, appendix, run instructions, anonymized repo, archival deposit guidance), produces an artifact-readiness checklist and a packaging plan, and lints the artifact directory for the bars reviewers actually check.

It does not run the author's experiments or claim a result reproduces — it prepares and checks the package, and tells the author exactly what reviewers will verify by hand.

When to use

"My paper was accepted — how do I do the artifact evaluation / get a badge?"
"Package / clean up my code for submission." / "anonymize my repo for review."
"What's an artifact appendix / Artifacts Available / Functional / Reusable?"
"Do I need a Zenodo DOI? concept vs version?" / "Software Heritage?"
"Fill out the NeurIPS code/reproducibility or ACL repro checklist."
"What does Reproduced vs Replicated mean for this badge?"
Alongside
```
prepare-camera-ready
```
(de-anonymization + final deposit overlap).

Inputs

The artifact directory — the code/data repo to be packaged (path).
The target venue + track, and ideally
```
venues/conferences/<v>-<year>.yml
```
(supplies the review blind level; create with
```
parse-cfp
```
if missing). The venue profile does NOT encode the artifact track's badge offering or its separate deadline — those are fetched live (step 1).
The paper's major claims (for a per-claim reproduction plan) and whether the artifact is for review-phase (often double-blind) or the final deposit. These change everything (anonymized ZIP vs version DOI).

Process

Fetch the venue's CURRENT Call for Artifacts — mandatory, live. Badge offerings vary per venue per year (OSDI '26 evaluates ONLY "Artifacts Available"; SOSP '26 offers all three). Memory and last year are stale by construction; verify live. From the live CFA confirm: which badges are offered this cycle, the separate artifact deadline, the archival-hosting requirement, the appendix template/length, and the blind model. Snapshots to start from (re-verify, don't trust): references/venue-artifact-rails.md. Record the chosen badge target + artifact deadline in
```
.paper-memory/decisions.md
```
.
Resolve the badge taxonomy and the era trap. Use
```
python3 scripts/badge_advisor.py --badge <name>
```
to print the ACM v1.1 families/tiers and, critically, the Reproduced/Replicated swap: ACM inverted these terms on 2020-05-14, so a pre-2020 badge means the inverse (
```
--era pre-2020
```
). Reproduction is never bit-exact — it must agree within a tolerance that does not change the paper's claims. Background: references/badging-standards.md.
Lint the artifact directory against the bars reviewers check:
```
python3 scripts/check_artifact.py <artifact_dir> \
    --venue venues/conferences/<v>-<year>.yml [--blind double]
```
It reports, with file paths: the ML Code Completeness 5 items (dependency spec, training code, evaluation code, pre-trained models or a documented way to get them, a README with a results table + the exact reproduce command); archival readiness (GitHub-only vs a DOI/SWHID); double-blind anonymization (author names/emails, identifying URLs, a
```
.git
```
directory, PDF/appendix metadata) — driven by the venue's blind level or
```
--blind
```
; and hygiene (a LICENSE, upload-size cap). Flags:
```
--json
```
,
```
--strict
```
,
```
--zip-cap-mb N
```
,
```
--venues-dir
```
. Exit codes: 0 clean, 1 errors, 2 usage. The lint covers FILES only — it cannot prove the build runs, that results reproduce, or that a DOI resolves.
Build the package the venue asks for (with the author, not for them):
- README — overview, exact dependency install, the precise command to reproduce each result, a results table, hardware/runtime expectations, and the license. (ML Code Completeness item 5.)
- Artifact appendix — for USENIX-family Phase 2, a ≤3-page PDF (their LaTeX template): hardware/software/config, the paper's major claims, and a per-claim reproduction procedure + result-comparison method ("agrees if within X%"). For SIGMOD ARI, include experiment scripts AND graph-generation scripts ("similar behavior", not exact numbers).
- Checklists — fill the NeurIPS Paper Checklist / Code policy or the ACL "Responsible NLP Research" checklist accurately: an honest "no"/"n/a" with justification is safe; a missing or misleading filing is the desk-reject (ARR desk-rejects misleading filings since Dec 2024). Do not game boxes to "yes."
Anonymize for double-blind review (if review-phase). Ship an anonymized ZIP without
.git
, or proxy through Anonymous GitHub (anonymous.4open.science), listing every identifying term to scrub. Cover PDF/appendix metadata, acknowledgments, funding, and self-citation phrasing — same rules as the paper (
```
anonymize-paper
```
). Details: references/archival-hosting.md.
Plan the archival deposit. For "Artifacts Available," the permanent copy must be on an archival host — USENIX-family rejects GitHub/personal sites. Use a Zenodo version DOI for the final (a concept DOI is OK only during evaluation) and/or a Software Heritage SWHID (intrinsic, ISO/IEC 18670); they are complementary. Add CITATION.cff/codemeta so the archive emits citation metadata. De-anonymize and deposit the FINAL version at camera-ready (
```
prepare-camera-ready
```
).
Write the artifact-readiness checklist + packaging plan to
```
paper-workspace/submission/artifact-readiness.md
```
and append a line to
```
paper-workspace/INDEX.md
```
. Order by severity; cite each finding's source (the lint, the live CFA, the badge taxonomy). Re-run the lint until the file-level bars pass.

Output

An artifact-readiness checklist (PASS / PASS-WITH-WARNINGS / FAIL with file paths) plus a packaging plan: target badges (from the live CFA), hosting (anonymized review copy + final version DOI/SWHID), the completeness gaps to close, the appendix/checklist to fill, and the separate artifact deadline. Written to
```
paper-workspace/submission/
```
.
Draft README / appendix / checklist content the author edits and owns.

Adapt to your discipline

The badge taxonomy here is ACM/USENIX/SIGMOD/ETAPS/ML-venue specific. For other fields, swap in your venue's artifact/data-availability rules (e.g. journal "data availability statements", FAIR data deposits) — the completeness and anonymization lints read the directory, not a discipline, so they still apply.

Guardrails

Re-verify the venue's CURRENT artifact rules live (step 1 is not optional). Badge offerings change per venue per year; never assume from memory or last year. Overconfidence is highest right after a fetch — re-check the primary CFA.
Never claim a result reproduces, and never demand bit-exact reproduction. ACM/SIGMOD/ETAPS require agreement within a tolerance that doesn't change the paper's claims. This skill prepares and checks the package; it does not run the experiments or judge the science.
The Reproduced/Replicated terms were swapped in 2020 — check the badge era (
```
badge_advisor.py --era
```
) before interpreting them.
Archival hosting is specific: a GitHub URL is not "Available" for the USENIX family — direct authors to a Zenodo version DOI / SWHID.
Anonymization-aware: for double-blind, scrub
```
.git
```
, names, emails, URLs, and metadata before any review-phase upload.
Accurate checklists, not gamed ones: honest "no"/"n/a" with justification is safe; misleading filings get desk-rejected.
Copilot, not pilot: never deposit, never submit to an artifact-evaluation system, never complete a checklist form on the author's behalf. Prepare, lint, explain — the author clicks.
Quote at most the flagged line/path; never bundle the author's artifact into this repo.

Memory

Uses the shared

.paper-memory/

convention (full spec:

paper-memory-convention.md

At start: read
```
lessons.md
```
(skip re-flagging fixed items; lead with any
```
recurring
```
packaging habits, e.g. "you tend to ship a
```
.git
```
directory") and
```
decisions.md
```
(the chosen venue/badge target + artifact deadline).
At end: append the target badge + artifact deadline to
```
decisions.md
```
, and one dated entry per finding worth remembering to
```
lessons.md
```
in the shared
```
- [YYYY-MM-DD] (prepare-artifacts | <scope>) issue -> recommendation
```
format (use
```
reflect-and-improve
```
's
```
reflect_log.py append
```
, which dedupes/dates).
Create
```
.paper-memory/
```
on demand and offer to add it to the project
```
.gitignore
```
. Local-only; never upload it or copy it into this repo.

prepare-artifacts

NPX Install

Tags

SKILL.md Content