data-export-and-restore

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Data export and restore

数据导出与恢复

In an app with no server, the export file is the only copy of a user's data that can outlive the phone, and restore is the only recovery path they can invoke themselves. Both are dangerous in opposite directions: a lossy export silently becomes someone's backup, and a half-applied restore destroys the very database it was meant to protect. This skill governs the portable artifacts; the on-device
VACUUM INTO
snapshot and WAL rules belong to
persistence-drift
.
Depth lives in
references/formats-and-encoding.md
(CSV/JSON/PDF mechanics) and
references/restore-and-merge-policy.md
(validation ladder, merge strategies).
在无服务器的应用中,导出文件是唯一一份能够比手机存续更久的用户数据副本,而恢复是用户唯一能够自行调用的恢复路径。 两者在相反方向上都存在危险:有损导出会在不知情的情况下变成某人的备份,而只应用了一半的恢复会破坏它本应保护的数据库。 本 skill 管理可移植 artifacts;设备上的
VACUUM INTO
快照和 WAL 规则属于
persistence-drift
。
深入细节位于
references/formats-and-encoding.md
(CSV/JSON/PDF 机制)和
references/restore-and-merge-policy.md
(验证阶梯、合并策略)。

Non-negotiable rules

不可妥协的规则

  1. Backup and export are two different features — never conflate them. Backup is exact, machine-read, round-trippable, and the only thing restore accepts. Export (CSV for a spreadsheet, PDF for a human) is an interop artifact, lossy by design, and must be refused as a restore source. WHY: the moment a CSV is restorable, a format built for readability becomes the file someone's records depend on.
  2. Every backup carries an envelope, and restore reads it first.
    formatVersion
    ,
    schemaVersion
    ,
    appVersion
    ,
    exportedAtUtc
    , a checksum over the payload, then the payload. Restore verifies the checksum, refuses
    schemaVersion
    greater than the app's with a plain message, and migrates an older payload through the same forward-only path as the database (
    run-migration
    ). WHY: an unversioned file cannot be refused, only misread.
  3. Restore is all-or-nothing and never edits the live database in place. Import into a fresh staging database, validate everything, close every handle, then publish by atomic rename with the previous file kept as a rollback until the new one opens cleanly. WHY: a partial restore is worse than a failed one — it leaves a state the user cannot describe and you cannot reproduce.
  4. Machine formats carry canonical values only. Integer minor units, SI integers, ISO-8601 UTC instants, stable ids, untranslated enum codes. Never a localized numeral, a formatted date, a currency symbol, or a translated label in a file that will be parsed. Localize only in human-facing exports, which are never re-imported. WHY:
    ١٢٫٥
    and
    12.5
    and
    12,5
    are the same quantity and three different parses.
  5. Identity is content-assigned, never a rowid. Every exportable row carries a stable id (UUID/ULID) minted at creation. WHY: re-importing the same file must be idempotent — with autoincrement ids it duplicates every record instead.
  6. State the merge policy and show it before it runs. Replace-all (wipe then import) and merge-by-id (last-write-wins on a stored
    updatedAtUtc
    ) are different promises; the confirmation names which one is about to happen and what will be lost. WHY: "Restore" reads as "add my data back" to a user who is about to lose a week.
  7. Stream to a temp file, publish by rename. Build the artifact through an
    IOSink
    into the app's own temp directory and rename it into place only when the last byte is flushed. WHY: a 100k-row export assembled in a
    String
    OOMs a cheap phone, and a half-written file must never be shareable.
  8. Nothing leaves the sandbox without an explicit user action, through an injected Gateway. No auto-upload, no background sync, no "helpful" cloud copy. The share sheet and file-save picker sit behind a
    ShareGateway
    /
    FileExportGateway
    (
    service-boundary-and-native
    ), faked in tests.
  9. CSV is escaped twice: RFC 4180 and against formula injection. Quote any field containing the delimiter, a quote, CR or LF, doubling embedded quotes; and neutralize a leading
    =
    ,
    +
    ,
    -
    ,
    @
    , tab or CR so a spreadsheet renders the text instead of executing it. WHY: an unescaped cell is a code-execution vector in the recipient's spreadsheet, and a lost row in yours.
  10. Export and restore return typed results, never throw at the UI.
    Result<ExportArtifact, ExportFailure>
    /
    Result<RestoreReport, RestoreFailure>
    with a distinct failure per refusal reason (checksum, unsupported version, malformed payload, no space, permission denied). WHY: "Restore failed" with no reason is indistinguishable from data loss to the person reading it.
  11. A failed export leaves no artifact. Delete the temp file on every failure path and never publish a partial one. WHY: a truncated file that looks like a backup is the failure mode that surfaces months later.
  12. Say what an export is. Exports are plaintext and unencrypted unless you built encryption on purpose; the UI must say so at the moment of sharing, and the claim must match the code (see
    release-and-store-shipping
    for claim wording).
  1. 备份和导出是两项不同的功能——绝不能混为一谈。 备份必须精确、可供机器读取、可往返转换,并且是恢复唯一接受的内容。导出(用于电子表格的 CSV、供人阅读的 PDF)是一种互操作 artifact,设计上就是有损的,必须拒绝将其作为恢复来源。原因:一旦 CSV 可以恢复,为可读性而构建的格式就会变成某人的记录所依赖的文件。
  2. 每个备份都带有 envelope,并且恢复首先读取 envelope。 依次包含
    formatVersion
    、
    schemaVersion
    、
    appVersion
    、
    exportedAtUtc
    、payload checksum,然后才是 payload。恢复会验证 checksum;如果
    schemaVersion
    高于应用当前版本,则用明确易懂的消息拒绝;较旧的 payload 则通过与数据库相同的仅向前迁移路径(
    run-migration
    )进行迁移。原因:没有版本信息的文件无法被明确拒绝,只能被错误解读。
  3. 恢复必须全有或全无,且绝不原地修改正在使用的数据库。 先导入一个全新的 staging database,验证所有内容,关闭所有 handle,然后通过 atomic rename 发布;在新文件能够正常打开之前,始终保留旧文件作为 rollback。原因:部分恢复比恢复失败更糟糕——它会留下一种用户无法描述、开发者也无法复现的状态。
  4. 机器格式只能包含 canonical values。 使用整数 minor units、SI integers、ISO-8601 UTC instants、stable ids 和未翻译的 enum codes。对于需要被解析的文件,绝不能使用本地化数字、格式化日期、货币符号或翻译后的 label。只在面向人类的导出中进行本地化,而这类导出永远不会被重新导入。原因:
    ١٢٫٥
    、
    12.5
    和
    12,5
    表示相同的数量,却会被解析为三种不同结果。
  5. 身份由内容分配,绝不能使用 rowid。 每个可导出行都携带一个在创建时生成的 stable id(UUID/ULID)。原因:重新导入同一个文件必须具备幂等性——如果使用自增 id,每条记录都会被重复创建。
  6. 声明合并策略,并在执行前展示给用户。 Replace-all(先清空再导入)和 merge-by-id(基于存储的
    updatedAtUtc
    执行 last-write-wins)代表两种不同的承诺;确认界面必须明确说明即将执行哪一种策略,以及会丢失哪些内容。原因:对一个即将失去一周数据的用户而言,“恢复”很容易被理解为“把我的数据添加回来”。
  7. 流式写入临时文件,再通过 rename 发布。 通过
    IOSink
    在应用自己的临时目录中构建 artifact,只有在最后一个字节完成 flush 后,才将其 rename 到目标位置。原因:在廉价手机上,将 10 万行导出拼接进
    String
    会导致 OOM,而半写入的文件绝不能被分享。
  8. 任何内容都不得在未经用户明确操作的情况下离开 sandbox,并且必须通过注入的 Gateway。 不允许自动上传、后台同步或出于“善意”自动创建云副本。分享面板和文件保存选择器位于
    ShareGateway
    /
    FileExportGateway
    (
    service-boundary-and-native
    )之后,并在测试中提供 fake。
  9. CSV 需要经过两层转义:RFC 4180 和公式注入防护。 对包含 delimiter、quote、CR 或 LF 的任意字段加引号,并将嵌入的引号加倍;同时处理前置的
    =
    、
    +
    、
    -
    、
    @
    、tab 或 CR,使电子表格将其作为文本呈现,而不是执行。原因:未转义的单元格会成为接收方电子表格中的代码执行向量,同时也可能导致你的数据丢失一行。
  10. 导出和恢复返回类型化结果,绝不在 UI 层抛出异常。 使用
    Result<ExportArtifact, ExportFailure>
    /
    Result<RestoreReport, RestoreFailure>
    ,并为每种拒绝原因提供独立的 failure(checksum、不支持的版本、格式错误的 payload、空间不足、权限被拒绝)。原因:只显示“恢复失败”而不说明原因,对阅读该消息的人而言,与数据丢失无法区分。
  11. 导出失败时不得留下 artifact。 在每条失败路径上删除临时文件,绝不发布部分完成的 artifact。原因:一个看起来像备份的截断文件,往往会在几个月后才暴露为问题。
  12. 明确说明导出文件的性质。 除非特意实现了加密,否则导出文件是明文且未加密的;UI 必须在分享时明确告知用户,而且声明必须与代码实际行为一致(相关措辞参见
    release-and-store-shipping
    )。

The envelope

Envelope

dart
/// The only shape restore accepts. Version fields come FIRST so a truncated or
/// foreign file is rejected by the header rather than by a mid-parse exception.
({
  int formatVersion,        // this envelope's own shape — bump independently of the DB
  int schemaVersion,        // the DB schema the payload was written from
  String appVersion,        // provenance for support, never a compatibility check
  DateTime exportedAtUtc,   // UTC, from the injected Clock
  String payloadSha256,     // checksum over the payload bytes, verified before parsing
  Map<String, Object?> payload,
});
formatVersion
and
schemaVersion
are separate on purpose: the envelope can gain a field without a database migration, and the database can migrate without changing the file shape. A single "version" number conflates them and forces a false choice on the next change. Complete, runnable shape:
examples/backup_envelope.dart
.
dart
/// The only shape restore accepts. Version fields come FIRST so a truncated or
/// foreign file is rejected by the header rather than by a mid-parse exception.
({
  int formatVersion,        // this envelope's own shape — bump independently of the DB
  int schemaVersion,        // the DB schema the payload was written from
  String appVersion,        // provenance for support, never a compatibility check
  DateTime exportedAtUtc,   // UTC, from the injected Clock
  String payloadSha256,     // checksum over the payload bytes, verified before parsing
  Map<String, Object?> payload,
});
formatVersion
与
schemaVersion
刻意分开:envelope 可以在不进行数据库迁移的情况下增加字段,而数据库也可以在不改变文件结构的情况下完成迁移。只使用一个“版本”数字会混淆两者,并在下一次变更时迫使开发者做出错误选择。完整、可运行的结构见
examples/backup_envelope.dart
。

Restore: validate, stage, swap

恢复:验证、暂存、交换

dart
Future<Result<RestoreReport, RestoreFailure>> restore(File picked) async {
  // 1. Header and integrity BEFORE anything is parsed or opened.
  final envelope = readEnvelope(picked);
  switch (envelope) {
    case Err(:final failure): return Err(failure);           // malformed / bad checksum
    case Ok(:final value) when value.schemaVersion > kSchemaVersion:
      return const Err(RestoreFailure.newerThanApp());       // refuse, never guess
    case Ok():
      break;
  }

  // 2. Import into a STAGING database — the live one is never touched yet.
  final staging = await openStagingDatabase();
  final imported = await staging.transaction(() => _importAll(envelope.value.payload));
  if (imported case Err(:final failure)) {
    await staging.close();
    await deleteStagingFiles();                              // user's data still intact
    return Err(failure);
  }

  // 3. Migrate the staged copy forward through the SAME path as the app database.
  //    4. Close every handle, then publish by rename, keeping the old file until the
  //       new one opens cleanly. See references/restore-and-merge-policy.md.
  return publishStagedDatabase(staging);
}
The staging file, the migration path, and the rename ordering are the whole correctness argument — a restore that writes directly into the open app database has no failure mode that leaves the user where they started.
dart
Future<Result<RestoreReport, RestoreFailure>> restore(File picked) async {
  // 1. Header and integrity BEFORE anything is parsed or opened.
  final envelope = readEnvelope(picked);
  switch (envelope) {
    case Err(:final failure): return Err(failure);           // malformed / bad checksum
    case Ok(:final value) when value.schemaVersion > kSchemaVersion:
      return const Err(RestoreFailure.newerThanApp());       // refuse, never guess
    case Ok():
      break;
  }

  // 2. Import into a STAGING database — the live one is never touched yet.
  final staging = await openStagingDatabase();
  final imported = await staging.transaction(() => _importAll(envelope.value.payload));
  if (imported case Err(:final failure)) {
    await staging.close();
    await deleteStagingFiles();                              // user's data still intact
    return Err(failure);
  }

  // 3. Migrate the staged copy forward through the SAME path as the app database.
  //    4. Close every handle, then publish by rename, keeping the old file until the
  //       new one opens cleanly. See references/restore-and-merge-policy.md.
  return publishStagedDatabase(staging);
}
staging file、迁移路径以及 rename 顺序共同构成了完整性的正确性论证——直接写入已打开应用数据库的恢复流程,不存在能够让用户停留在原状态的失败路径。

Choosing a format

选择格式

NeedFormatRestorable?
Full-fidelity backup the app reads backEnvelope + JSON payload (or a copied DB file)Yes — the only one
Rows for a spreadsheet or another toolCSV, RFC 4180, canonical valuesNo
Something a human reads, prints, or filesPDF, fully localized and formattedNo
Human-facing exports localize at the edge like any other rendering (
value-objects-money-and-units
,
i18n-rtl-l10n
); machine formats never do. Copying the live database file directly is a valid backup only through the WAL-safe primitive in
persistence-drift
— never
File.copy
on an open database.
需求格式可恢复?
应用能够重新读取的完整保真备份Envelope + JSON payload(或复制的 DB file)是——唯一可恢复的格式
供电子表格或其他工具读取的行CSV、RFC 4180、canonical values否
供人阅读、打印或归档的内容PDF,完全本地化并格式化否
面向人类的导出会像其他渲染一样在输出边界进行本地化(
value-objects-money-and-units
、
i18n-rtl-l10n
);机器格式则绝不会进行本地化。只有通过
persistence-drift
中的 WAL-safe primitive 直接复制正在使用的数据库文件,才是有效的备份方式——绝不能对已打开的数据库执行
File.copy
。

Tests that must exist

必须存在的测试

  • Round trip on a hostile fixture. Export → import into an empty database → export again → byte-identical. The fixture carries apostrophes, quotes, commas, embedded newlines, emoji, RTL text with bidi marks, whitespace-only strings, the largest supported number, a DST-ambiguous local instant, and a null-vs-empty pair.
    'test1'
    survives everything and proves nothing.
  • Idempotent re-import. Importing the same file twice produces the same state under merge-by-id — not duplicates.
  • Every refusal is a test: truncated file, flipped checksum byte,
    schemaVersion
    one higher than the app, an older
    schemaVersion
    that must migrate, empty payload, and a file that is not this format at all. Each asserts the typed failure and that the live database is byte-unchanged.
  • Failure leaves no artifact. Force a mid-write failure; assert no publishable file exists in the export directory.
  • CSV escaping. A cell containing
    =cmd()
    , one containing
    a,b"c\nd
    , and an RTL cell each survive a write→read round trip through a real parser.
  • 在对抗性 fixture 上执行往返测试。 导出 → 导入空数据库 → 再次导出 → 字节完全一致。fixture 必须包含 apostrophe、quote、comma、embedded newline、emoji、带有 bidi mark 的 RTL 文本、仅含空白的字符串、支持的最大数值、DST-ambiguous local instant,以及 null 与 empty 的对照项。
    'test1'
    能通过所有测试,但这什么也证明不了。
  • 幂等重新导入。 在 merge-by-id 策略下,将同一个文件导入两次必须产生相同状态,而不是生成重复记录。
  • 每一种拒绝都必须有测试: 文件被截断、checksum 中的某个字节被翻转、
    schemaVersion
    比应用高一个版本、必须迁移的较旧
    schemaVersion
    、空 payload,以及完全不属于该格式的文件。每个测试都要断言对应的类型化 failure,并验证正在使用的数据库在字节层面保持不变。
  • 失败不得留下 artifact。 强制触发写入中途失败;断言导出目录中不存在可发布文件。
  • CSV 转义。 包含
    =cmd()
    的单元格、包含
    a,b"c\nd
    的单元格,以及 RTL 单元格,都必须通过真实 parser 完成 write→read 往返测试并保持内容不变。

Anti-patterns

反模式

  • CSV as the backup format — no envelope, no types, no versioning; it will be someone's only copy.
  • An unversioned export file — the first schema change makes every existing file either unreadable or, worse, silently misread.
  • Restoring straight into the live database — no rollback exists; a mid-import failure leaves a state nobody can describe.
  • Autoincrement rowids as export identity — re-import duplicates everything.
  • Formatting numbers or dates in a machine format — locale-dependent output that round-trips only on the machine that wrote it.
  • Building the export in a
    String
    /
    StringBuffer
    — fine for 100 rows, an OOM crash at 100k, on the cheapest device you support.
  • Writing directly to the shareable path — a crash mid-write publishes a truncated file that still looks like a backup.
  • Auto-uploading or auto-backing-up "for safety" — it converts an offline app into a data processor, changing the store privacy declaration and breaking the claim.
  • Trusting a picked file's extension or name — validate the envelope, not the path.
  • A generic "Import failed" message — the user cannot tell a wrong file from lost data; name the reason from the typed failure.
  • Silently merging when the user expected replace (or vice versa) — state the policy in the confirmation, in the same words as the code.
  • 将 CSV 作为备份格式——没有 envelope、类型或版本控制;它会成为用户唯一的数据副本。
  • 无版本的导出文件——第一次 schema 变更就会让所有现有文件无法读取,或者更糟,被静默误解。
  • 直接恢复到正在使用的数据库——不存在 rollback;导入中途失败会留下无人能够描述的状态。
  • 使用自增 rowid 作为导出身份——重新导入会复制所有数据。
  • 在机器格式中格式化数字或日期——输出依赖 locale,只能在写入它的机器上完成往返转换。
  • 在
    String
    /
    StringBuffer
    中构建导出
    ——处理 100 行没问题,但在支持的最低端设备上处理 10 万行时会因 OOM 崩溃。
  • 直接写入可分享路径——写入中途崩溃会发布一个截断文件,而它看起来仍像一份有效备份。
  • 为“保险起见”自动上传或自动备份——这会把离线应用变成数据处理程序,改变隐私存储声明,并破坏原有承诺。
  • 信任 picked file 的扩展名或文件名——必须验证 envelope,而不是路径。
  • 只显示通用的“导入失败”消息——用户无法区分文件错误与数据丢失;必须根据类型化 failure 说明具体原因。
  • 在用户预期 replace 时静默执行 merge,或反之——确认界面必须使用与代码完全一致的措辞声明策略。

Definition of done

完成标准

  • Backup and export are separate paths; export artifacts are refused by restore.
  • Every backup carries
    formatVersion
    ,
    schemaVersion
    ,
    appVersion
    ,
    exportedAtUtc
    , and a payload checksum, in that header-first order.
  • Restore verifies the checksum, refuses a newer
    schemaVersion
    , and migrates an older payload through the same forward-only path as the database.
  • Restore imports into a staging database and publishes by rename; a failure leaves the live database byte-unchanged.
  • Machine formats carry canonical values and stable content ids only.
  • The merge-vs-replace policy is explicit in code and named in the confirmation UI.
  • Writes stream to a temp file and are published by atomic rename; failures delete the temp file and publish nothing.
  • Share/save goes through an injected Gateway on an explicit user action; nothing uploads automatically.
  • CSV output is RFC 4180 quoted and formula-injection escaped.
  • Export/restore return typed
    Result
    s with one failure per refusal reason.
  • Round-trip, idempotent-re-import, every-refusal, no-partial-artifact, and CSV escaping tests pass on a hostile fixture.
  • 备份和导出使用独立路径;恢复必须拒绝 export artifacts。
  • 每个备份都按 header-first 顺序携带
    formatVersion
    、
    schemaVersion
    、
    appVersion
    、
    exportedAtUtc
    和 payload checksum。
  • 恢复会验证 checksum,拒绝更高的
    schemaVersion
    ,并通过与数据库相同的仅向前路径迁移较旧的 payload。
  • 恢复会导入 staging database 并通过 rename 发布;失败时,正在使用的数据库在字节层面保持不变。
  • 机器格式仅包含 canonical values 和 stable content ids。
  • merge-vs-replace 策略在代码中明确声明,并在确认 UI 中准确标出。
  • 写入以流式方式输出到临时文件,并通过 atomic rename 发布;失败时删除临时文件,不发布任何内容。
  • 分享和保存通过注入的 Gateway 响应用户明确操作;任何内容都不会自动上传。
  • CSV 输出采用 RFC 4180 quoting,并防范公式注入。
  • 导出和恢复返回类型化
    Result
    ,每种拒绝原因对应一种 failure。
  • 在对抗性 fixture 上通过往返、幂等重新导入、全部拒绝场景、无部分 artifact 和 CSV 转义测试。

Related skills

相关 skills

  • persistence-drift
    — the WAL-safe on-device snapshot primitive and the DAO layer the importer writes through; never
    File.copy
    a live database.
  • run-migration
    — the forward-only path an older payload is migrated through.
  • error-handling-typed-results
    — the
    Result
    /
    Failure
    spine and the never-lose-data guarantees this feature is the user-facing end of.
  • value-objects-money-and-units
    — canonical storage (minor units, SI, UTC) that makes a machine format round-trip.
  • i18n-rtl-l10n
    — localize-at-render, and why localized numerals never enter a file.
  • service-boundary-and-native
    — the
    ShareGateway
    /file-picker seam and its fake.
  • release-and-store-shipping
    — the privacy claims an export feature must not break.
  • testing-strategy
    — hostile fixtures, property/round-trip tests, injected
    Clock
    .
  • persistence-drift
    ——用于设备上的 WAL-safe 快照原语,以及 importer 写入所经过的 DAO 层;绝不要对正在使用的数据库执行
    File.copy
    。
  • run-migration
    ——较旧 payload 迁移时经过的仅向前路径。
  • error-handling-typed-results
    ——构成
    Result
    /
    Failure
    主干,并为该功能提供面向用户的绝不丢失数据保证。
  • value-objects-money-and-units
    ——canonical storage(minor units、SI、UTC),确保机器格式可以完成往返转换。
  • i18n-rtl-l10n
    ——在渲染时进行本地化,以及本地化数字为何绝不能进入文件。
  • service-boundary-and-native
    ——
    ShareGateway
    /文件选择器边界及其 fake。
  • release-and-store-shipping
    ——导出功能不得破坏的隐私声明。
  • testing-strategy
    ——对抗性 fixtures、property/round-trip 测试和注入的
    Clock
    。

References

参考资料