convex-verify

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Prove a feature works — seed, drive, assert

证明功能有效——seed（注入）、drive（驱动）、assert（断言）

A green typecheck proves the code parses; it does not prove a non-owner is actually denied, that a query returns the right rows, or that a mutation has the effect it claims. This capability closes that gap with the loop the whole field is missing: seed → drive → assert, run in-process with

convex-test

so it needs no deployment. Its highest-value assertions are the NEGATIVE ones — the caller who should be refused — because those are exactly the authz defects the 30-app corpus shows are the #1 real bug and the ones a happy-path demo never catches.

类型检查通过只能证明代码能够解析；它无法证明非所有者确实会被拒绝、query 会返回正确的数据行，或 mutation 会产生其声称的效果。该能力通过整个领域都缺乏的闭环来弥补这一差距：seed → drive → assert，使用

convex-test

在进程内运行，因此无需部署。价值最高的断言是负向断言——即本应被拒绝的调用者确实被拒绝——因为包含 30 个应用的语料库表明，这些正是实际 bug 中排名第一的 authz 缺陷，也是只演示 happy path 永远无法发现的问题。

Workflow

工作流

IDENTIFY the feature to prove: the specific exported query/mutation/action (or a small set) the user just built/changed, and its intended behavior — who should be allowed, what data should come back, what a mutation should change. If the intent is unstated, ask one focused question rather than guessing the contract.
SET UP
```
convex-test
```
: ensure
```
convex-test
```
+
```
vitest
```
are dev deps AND a
```
vitest.config.ts
```
sets
```
test.environment: "edge-runtime"
```
with
```
server.deps.inline: ["convex-test"]
```
— WITHOUT that config,
```
convexTest(schema)
```
fails at runtime with
```
import.meta.glob is not a function
```
(verified). Also install
```
@edge-runtime/vm
```
. Then
```
convexTest(schema)
```
gives a
```
t
```
handle. Reuse the project's existing test setup if present (compose with the
```
test
```
capability, don't fork it).
SEED realistic data through the app's OWN functions where possible (so the seed exercises the same validators/mutations a real user would), falling back to
```
t.run(async (ctx) => ctx.db.insert(...))
```
for fixtures the public API can't create. Seed at least: the caller's own rows AND a second user's rows, so cross-user access is testable.
DRIVE the feature as DIFFERENT identities with
```
t.withIdentity({ subject, tokenIdentifier, ... })
```
: call the function as (a) the legitimate owner, (b) a different authenticated user, and (c) unauthenticated (
```
t
```
with no identity). Use the real identity shape the app's auth uses (subject/tokenIdentifier), matching how ownership is resolved.
ASSERT behavior — POSITIVE and NEGATIVE:
- positive: the owner gets the expected rows / the mutation made the expected change (
```
expect(await t.withIdentity(owner).query(api.x.y, args)).toEqual(...)
```
  ).
- NEGATIVE (the load-bearing half): a different user calling the same function is REFUSED —
```
await expect(t.withIdentity(other).mutation(api.x.cancel, {id})).rejects.toThrow(/forbidden|not authorized|403/)
```
  — and an unauthenticated caller is refused where auth is required. A feature is not proven until the wrong caller is shown to be blocked.
- data-scope: a list/query returns ONLY the caller's rows, never the second user's (assert the second user's row is absent).
RUN the tests (
```
npx vitest run
```
) and report: what was proven (each positive + negative assertion that passed), and — critically — any assertion that FAILED, because a failed negative assertion is a real authz hole found before ship. Emit findings on the bus (specs/finding.schema.json, class authz/correctness, evidence kind probe-result with the exact failing call) for anything that didn't behave.
Do NOT weaken a test to make it pass: if the owner-only query returns another user's row, the FIX is in the function (hand to convex-authz), not in the assertion. A test changed until it's green proves nothing.

确定要证明的功能：用户刚刚构建或更改的特定导出 query/mutation/action（或一小组）及其预期行为——谁应当被允许、应返回哪些数据、mutation 应改变什么。如果未说明意图，应提出一个聚焦的问题，而不是猜测契约。
设置
```
convex-test
```
：确保
```
convex-test
```
和
```
vitest
```
是 dev deps，并且
```
vitest.config.ts
```
设置了
```
test.environment: "edge-runtime"
```
和
```
server.deps.inline: ["convex-test"]
```
——缺少此配置时，
```
convexTest(schema)
```
会在运行时因
```
import.meta.glob is not a function
```
而失败（已验证）。还需安装
```
@edge-runtime/vm
```
。随后，
```
convexTest(schema)
```
会提供一个
```
t
```
句柄。如果项目中已有测试设置，请复用它（与
```
test
```
capability 组合使用，不要另起一套）。
尽可能通过应用自身的函数 seed 逼真的数据（这样数据注入会走与真实用户相同的 validators/mutations）；对于公共 API 无法创建的 fixture，则回退到
```
t.run(async (ctx) => ctx.db.insert(...))
```
。至少 seed：调用者自己的数据行，以及第二个用户的数据行，以便测试跨用户访问。
使用
```
t.withIdentity({ subject, tokenIdentifier, ... })
```
以不同身份 drive 该功能：将函数分别作为 (a) 合法所有者、(b) 另一个已认证用户和 (c) 未认证用户（不带 identity 的
```
t
```
）调用。使用应用 auth 实际采用的 identity 结构（subject/tokenIdentifier），并与所有权的解析方式保持一致。
断言行为——正向和负向：
- 正向：所有者获得预期数据行，或 mutation 产生了预期更改（
```
expect(await t.withIdentity(owner).query(api.x.y, args)).toEqual(...)
```
  ）。
- 负向（承担关键作用的一半）：调用同一函数的其他用户会被拒绝——
```
await expect(t.withIdentity(other).mutation(api.x.cancel, {id})).rejects.toThrow(/forbidden|not authorized|403/)
```
  ——在需要认证的地方，未认证调用者也会被拒绝。只有证明错误调用者确实被阻止，功能才算得到证明。
- data-scope：list/query 只返回调用者自己的数据行，绝不返回第二个用户的数据行（断言第二个用户的数据行不存在）。
运行测试（
```
npx vitest run
```
）并报告：证明了什么（每项通过的正面和负向断言），以及——尤其关键——任何失败的断言，因为负向断言失败意味着在发布前发现了一个真实的 authz 漏洞。对于任何未按预期行为运行的断言，在 bus 上发出 finding（
```
specs/finding.schema.json
```
，class 为 authz/correctness，evidence kind 为 probe-result，并附上准确的失败调用）。
不要削弱测试来让它通过：如果 owner-only query 返回了其他用户的数据行，修复应落在函数中（移交给 convex-authz），而不是修改断言。一个被改到变绿的测试什么也证明不了。

Rules

规则

Prove behavior, not compilation: every verification includes at least one NEGATIVE assertion (a caller who should be refused is refused) — the happy path alone is not proof.
Drive the feature as multiple identities with t.withIdentity (owner, other user, unauthenticated) using the app's real subject/tokenIdentifier shape.
Seed both the caller's rows AND a second user's rows so cross-user access and data-scope are actually testable.
A vitest.config.ts with environment 'edge-runtime' + convex-test inlined is REQUIRED for convex-test to run (import.meta.glob needs it); author it, don't just author the test file.
Run in-process with convex-test — no deployment needed; compose with the
```
test
```
capability's setup rather than forking it.
Never weaken an assertion to make it pass: a failing negative test is a real defect → hand the fix to convex-authz/convex-expert, don't edit the test until it's green.
Emit a bus finding for any assertion that failed (authz/correctness, evidence: the failing probe call) so a composite pass or self-heal can pick it up.
This drives a SPECIFIC built feature; a request to set up a test framework generally is the
```
test
```
capability.

证明行为，而非编译：每次验证至少包含一项负向断言（应被拒绝的调用者确实被拒绝）——只有 happy path 不足以构成证明。
使用
```
t.withIdentity
```
以不同身份（所有者、其他用户、未认证用户）drive 功能，并采用应用真实的 subject/tokenIdentifier 结构。
同时 seed 调用者自己的数据行和第二个用户的数据行，确保真正可以测试跨用户访问和 data-scope。
必须使用包含 environment
```
'edge-runtime'
```
并将
```
convex-test
```
inline 的
```
vitest.config.ts
```
，
```
convex-test
```
才能运行（
```
import.meta.glob
```
需要此配置）；应编写该配置，而不只是编写测试文件。
使用
```
convex-test
```
在进程内运行——无需部署；应与
```
test
```
capability 的设置组合，而不是另建一套。
绝不削弱断言来让它通过：负向测试失败就是真实缺陷 → 将修复移交给 convex-authz/convex-expert，不要编辑测试直到其变绿。
对于任何失败的断言，都要在 bus 上发出 finding（authz/correctness，evidence：失败的 probe 调用），以便 composite pass 或 self-heal 流程接手处理。
该流程用于 drive 一个已构建的特定功能；如果请求是搭建通用测试框架，则应使用
```
test
```
capability。