Upgrading an ObjectStack metadata project across a protocol major

N

preflight → mechanical chain → semantic residue → acceptance report

os migrate meta

validate

⛔ The boundary — read this before the first command

??

validate

Quickstart

# 0 · preflight — what major is this project on, what major is installed?
grep -rn "protocol" objectstack.config.ts package.json | head
node -p "require('@objectstack/spec/package.json').version"

# 1 · mechanical — replay the chain (reads the config, writes nothing but --out)
os validate > .upgrade/validate-before.txt 2>&1 || true   # the control, kept
os migrate meta --from 16 --step
os migrate meta --from 16 --json > .upgrade/migrate.json
os migrate meta --from 16 --out .upgrade/migrated.stack.json

# 2 · semantic residue — harvest the prescriptions, then work each item
node -e "console.log(require('fs').readFileSync(require.resolve('@objectstack/spec/package.json').replace('package.json','spec-changes.json'),'utf8'))" > .upgrade/spec-changes.json
grep -rho '\[REMOVED\][^"]*' node_modules/@objectstack/spec/json-schema/ | sort -u > .upgrade/tombstones.txt

# 3 · acceptance — all four, not three
os validate                      # green (compare against validate-before.txt)
tsc --noEmit                     # tombstones type the retired keys as `never`
os migrate meta --from 17        # must say "Nothing to migrate"
# → write .upgrade/REPORT.md (template in §3.4)

0 · Preflight

Establish the FROM major — do not guess it

--from

1. manifest.protocol

   in the stack config (

   '16.0.0'

   →

   --from 16

   ). This is the declared answer and the kernel checks it at load time.
2. The last

   @objectstack/spec

   major the project ever installed — read the lockfile history (

   git log -p pnpm-lock.yaml | grep -m5 '@objectstack/spec'

   ) when the manifest is absent or stale.
3. Ask. A manifest that says 16 on a project last touched two years ago is a claim, not a measurement. If (1) and (2) disagree, the lower one is the safe

   --from

   — the chain is idempotent, so replaying a hop that has already been applied is a no-op, while skipping a hop loses its rewrites.

os migrate meta --from 10

Make the work reviewable before you change anything

git checkout -b upgrade/protocol-17
mkdir -p .upgrade         # every artifact this skill produces lands here

.upgrade/

migrate.json

migrated.stack.json

spec-changes.json

tombstones.txt

REPORT.md

1 · Mechanical layer — invoke the chain

What

os migrate meta

actually does

os migrate meta --from 16                       # replay 16 → current
os migrate meta --from 16 --step                # per-hop checkpoint (bisect a failure)
os migrate meta --from 16 --to 17               # stop at a specific major
os migrate meta --from 16 --json                # machine-readable result
os migrate meta --from 16 --out migrated.json   # write the canonicalized stack

• Applied N mechanical change(s)

  — one line per rewritten site, as

  path: from → to (conversionId)

  . This is the diff, already attributed.

• N manual change(s) require your judgment

  — the chain's semantic entries for the majors you crossed, each with a

  why

  and a

  verify

  line. These are Layer 2's input, not a warning to dismiss.

• Migrated stack is schema-valid

  — or the warning that it is not yet, which means a residue item is still blocking the parse.
• Pending data migrations, when the chain crosses into a major with per-deployment data gates — see below.

⚠ The one fact that surprises every operator

os migrate meta

--out

conversionId

--out

os migrate meta --from 16 --out .upgrade/migrated.stack.json
# then, after porting the edits into the real sources:
os migrate meta --from 17 --out .upgrade/recheck.json   # should apply 0 changes

Stored rows: rehydration replays the same conversions

sys_metadata

applyConversionsToStoredItem

• You do not hand-edit

  sys_metadata

  . Ever. A row at rest has no author to ask, so the replay is unconditional and complete by design.
• The rows on disk stay in their old shape until something rewrites them. Rehydration is a read-time projection.
• To make it durable, run the stored pass — read-only by default:
  bash

  os migrate meta --stored                     # preview, writes nothing
  os migrate meta --stored --type view --type object   # narrow the pass
  os migrate meta --stored --apply             # rewrite the rows (prompts)

  --stored

  takes no

  --from

  : a stored row carries its own history, so the pass replays the whole chain. The authored-source flags and the stored-only flags are mutually exclusive, and mixing them is refused rather than ignored.

Data migrations are not metadata migrations

os migrate files-to-references

os migrate value-shapes

--apply

2 · Semantic residue — the part that is yours

2.1 Harvest the instruction sources — all of them ship

@objectstack/spec

os migrate meta --from N --json

.specChanges

node_modules/@objectstack/spec/spec-changes.json

perMajor[].converted

perMajor[].migrated

node_modules/@objectstack/spec/json-schema/**

src/**/*.zod.ts

[REMOVED] …

node_modules/@objectstack/spec/CHANGELOG.md

tsc

# every tombstone prescription the installed spec carries, deduped
grep -rho '\[REMOVED\][^"]*' node_modules/@objectstack/spec/json-schema/ | sort -u

# the FROM → TO table for one retired key
grep -n -B4 -A20 'transform' node_modules/@objectstack/spec/CHANGELOG.md | less

Not reachable from a consumer project, so do not send anyone there: the conversion and migration registries (

src/conversions/registry.ts

,

src/migrations/registry.ts

) and the platform repo's generated upgrade guide are not in the published package — only

src/**/*.zod.ts

is. Their consumer-facing projection is

spec-changes.json

and the chain's own

--json

output, which is exactly what the table above points at.

2.2 The three residue classes

undefined

2.3 A worked R1 — the retired field-mapping

transform

{
  "connectors": [{
    "name": "sap_erp",
    "fieldMappings": [
      { "source": "order_value", "target": "order_total",
        "transform": { "type": "javascript", "expression": "value / 100" } }
    ]
  }]
}

field-mapping-transform-removed

mapping.fieldMapping[].transform

none

constant

map

split

join

lookup

params

2.4 Decide alone, or ask the owner

1. The prescription names exactly one target. The tombstone or conversion summary gives a single FROM → TO, with the value unchanged.
2. The evidence is in the project. A grep in the repo settles it — the skill that already owns the tool, the import that already exists, the field the predicate already references.
3. Being wrong fails a gate. A mistaken choice breaks

   tsc

   or

   validate

   rather than changing behaviour quietly.

1. Two or more real targets, and the choice is a business statement — the

   transform

   case above.
2. The change is observable without a test failing — security posture (an authentication default), row visibility (an access predicate), retry counts, retention. A wrong call here ships silently and is discovered by an auditor.
3. Capability has to be re-declared somewhere new, so choosing wrong removes a capability instead of breaking a build. Agent tooling that has to move inside a specific skill is the canonical shape.
4. The source is dead or undocumented in their repo — nothing to decide from. Say so; do not invent a rationale.

transform

AWAITING DECISION

2.5 Working an R2 — retired APIs in the project's own code

# from the chain's own output — the surfaces it says it cannot fix for you
os migrate meta --from 16 --json | node -e "
  let s=''; process.stdin.on('data',d=>s+=d).on('end',()=>{
    for (const t of JSON.parse(s).todos) console.log(t.surface);
  })"
# then, for each surface, search the project (not node_modules)
grep -rn "<surface-token>" src/ app/ --include='*.ts' --include='*.tsx'

• A renamed context field keeps compiling. When a read moves from one key to another and the old key is simply absent afterwards, the code reads

  undefined

  and every branch quietly takes its false path. Verify against a real dispatch, not a fixture — invoke the path and assert the value observed under the canonical key.
• A rename is not always the fix. If the old read was itself wrong, renaming it migrates the defect rather than the code. Read the semantic entry's

  reason

  before applying its

  replacement

  .

2.6 Working an R3 — the prose sweep

# every retired key name the installed spec knows, as a search list
grep -rho '\[REMOVED\] `[^`]*`' node_modules/@objectstack/spec/json-schema/ \
  | sed 's/.*`\(.*\)`.*/\1/' | sort -u > .upgrade/retired-names.txt

*.md

3 · Acceptance — what "upgraded" means

3.1 Typed

tsc --noEmit

never

tsc

3.2 Parse-gated

tsc

3.3 Validate

os validate            # green is the criterion
os validate --strict   # warnings become errors — agree with the owner whether this is the bar

os validate

os validate

⚠ A green

validate

does NOT mean the chain has nothing left to do

Some conversions are migration-chain-only: the loader deliberately does not apply them and no tombstone rejects the old shape, because the change is a default flip rather than a rename — auto-applying it would stamp a constraint onto sources that deliberately omit it. The 16 → 17 crossing has one:

field-required-notnull-explicit

, which writes the physical

storage.notNull

that

required

used to imply on its own.

A project carrying only that shape validates green while the chain still has work. So

validate

green is necessary and not sufficient, and the criterion that closes the gap is the replay:

bash

os migrate meta --from <target-major>    # must report "Nothing to migrate"

Run both. A report that cites only

validate

cannot see this class at all.

3.4 The report — the human half

.upgrade/REPORT.md

# Protocol 16 → 17 upgrade — <project>

**Status:** complete | complete with N open decisions
**Spec:** <installed @objectstack/spec version>  ·  **Chain:** 16 → 17
**Verified:** `os validate` green · `tsc --noEmit` green · replay-from-17 applies 0 changes

## 1 · Mechanical (applied by the chain)

| Site | Change | Conversion |
|:--|:--|:--|
| `objects[crm_lead].fields.name` | `required: true` → `+ storage.notNull: true` | `field-required-notnull-explicit` |
| … | | |

_N sites, M conversions. Ported into sources from `os migrate meta --out`._

## 2 · Semantic residue (decided)

### `connector.fieldMappings[].transform` — RESOLVED
- **Site:** `src/connectors/sap.ts:24`
- **Prescription:** <verbatim from the tombstone>
- **Options:** import-mapping `transform` · ETL step · delete
- **Decision:** delete — owner confirmed the values arrive pre-scaled.
  _Decided by: <who>, <date>._
- **Verified:** `os validate` green; connector sync run against staging, 200 rows, values unchanged.

## 3 · Open decisions

| Item | Site | Options | Recommendation | Blocking? |
|:--|:--|:--|:--|:--|
| `agent.tools` → which skill | `src/ai/support-bot.ts:12` | `case_management` · new skill | `case_management` | no — parses without it |

## 4 · Pending, per deployment

- [ ] `os migrate files-to-references` — media values only warn until it passes.
- [ ] `os migrate value-shapes` — stored reference/JSON values unchecked until it passes.
- [ ] `os migrate meta --stored --apply` — rows rehydrate correctly today; this makes it durable.

## 5 · Not changed, and why

- <retired surface the project never used>  — no occurrences.

3.5 Prove the gate is real, do not assume it

# a stack that still carries a retired key, fed straight to the schema
os validate .upgrade/residue-probe.config.mjs

1. Rejected with the prescription — a tombstoned key. This is what a tombstone looks like when it works:

   ✗ connectors.0.fieldMappings.0.transform
     invalid_type: `FieldMapping.transform` … was removed in @objectstack/spec
     17.0.0 (#5552, ADR-0049) … Delete the key. The transform pipeline that IS
     enforced is the import mapping's … Run `os migrate meta --from 16` to
     rewrite it automatically.
     expected: never

   Note what the error is not: not "unrecognized key", not a deprecation label. The fix-it text is the error.
2. Accepted, and the chain rewrites it — a conversion with a live load-path acceptance window. Your evidence for that key is the chain's diff, not the parse gate.
3. Accepted, and the chain still rewrites it — a migration-chain-only conversion (§3.3).

   validate

   cannot see this class at all; only the replay can.

The v17 prescription set, as of

17.0.0-rc.5

17.0.0-rc.5

17.0.0-rc.5

@objectstack/spec

17.0.0-rc.5

17.0.0

retiredKey()

*.zod.ts

[REMOVED]

json-schema/

RETIRED_KEYS_BY_MAJOR[17]

• data/ExternalFieldMapping:transform

• integration/ConnectorFieldMapping:transform

• shared/FieldMapping:transform

RETIRED_DEFS_BY_MAJOR[17]

• shared/FieldMappingTransform

Why these two tables are short and the counts above are not. They record retirements registered under the exact-key gates that created them, which are newer than most of protocol 17's work; they are explicitly not a backfill of every retirement ever. The 45 conversions and 113 tombstones are the real size of the v17 surface. Use the tables to answer "was this retirement formally registered", and the conversions/tombstones to answer "what do I have to change" — the second question is the upgrade's question.

How this table is refreshed

# protocol version, support floor, and the per-major conversion / semantic counts
node -e "
  const p = require.resolve('@objectstack/spec/package.json');
  const j = require(p.replace('package.json','spec-changes.json'));
  const e = j.perMajor.find(x => x.to === 17);
  console.log(j.protocolVersion, 'floor', j.supportFloor, '| 16→17:',
              e.converted.length, 'converted,', e.migrated.length, 'semantic');
"

# tombstone prescriptions actually present in this install
grep -rho '\[REMOVED\][^"]*' node_modules/@objectstack/spec/json-schema/ | sort -u | wc -l

rc.5

The v17-canonical shapes, compiled

import { ObjectSchema } from '@objectstack/spec/data';
import { defineAgent } from '@objectstack/spec/ai';

// `conditionalRequired` → `requiredWhen`; `required` now also states the
// physical constraint explicitly via `storage.notNull`.
export const Lead = ObjectSchema.create({
  name: 'crm_lead',
  label: 'Lead',
  fields: {
    name: { type: 'text', required: true, storage: { notNull: true } },
    status: { type: 'select', required: true, storage: { notNull: true } },
    due_date: { type: 'date', requiredWhen: 'record.stage == "closed"' },
    notes: { type: 'textarea' },
  },
});

// Agent capability is reached through skills — there is no inline tool list.
export const SupportBot = defineAgent({
  name: 'support_bot',
  label: 'Support Bot',
  role: 'Front-line support triage',
  instructions: 'Answer support questions and open cases when needed.',
  skills: ['case_management'],
});

Failure modes

migrate meta

--out

--out

conversionId

validate

undefined

??

validate

validate

validate

validate

--apply

--apply

--yes

--force

--type

--database-url

--stored

--stored

MigrationFloorError

--from

Guardrails (binding)

1. Run the chain first, always. It is the only source of an attributed, schema-proved diff.
2. One conversion id per commit, where the project's review culture allows it. The

   conversionId

   is the commit's subject line and its justification.
3. No tolerant reads, ever — not in the metadata, not in the project's code. A retirement that gets re-admitted at the consumer is a defect that moved into a repo with no gate over it.
4. Ask about intent, decide about mechanics. The split in 2.4 is the contract with the project's owner.
5. The report is a deliverable, not a summary. No report, not done.
6. Never leave a pending per-deployment data migration unnamed. A gate nobody was told about is served by nobody.

Cross-skill routing

• Authoring the corrected metadata — load the domain skill for the shape you are fixing (data, ui, automation, ai, api, i18n).
• Any CEL predicate you rewrite while resolving a residue item — load formula.
• Runtime, plugin, and CLI questions the upgrade turns up — load platform.
• This skill covers the consumption side of a retirement. Designing one inside the ObjectStack platform repo is a different job with its own internal playbook, and it is not this one.