cli-e2e-testcase-writer

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

CLI E2E Testcase Writer

CLI E2E测试用例编写指南

Work on one domain per run. Produce exactly two artifacts for that domain:

workflow testcase files under
```
tests/cli_e2e/{domain}/
```
```
tests/cli_e2e/{domain}/coverage.md
```

Focus on domain testcase files. Do not change shared E2E support code such as

tests/cli_e2e/core.go

unless the user explicitly asks. Treat

tests/cli_e2e/demo/

as reference only.

每次仅针对一个领域开展工作。为该领域生成恰好两类产物：

位于
```
tests/cli_e2e/{domain}/
```
下的工作流测试用例文件
```
tests/cli_e2e/{domain}/coverage.md
```
文件

重点关注领域测试用例文件。除非用户明确要求，否则不要修改共享的E2E支持代码（如

tests/cli_e2e/core.go

）。仅将

tests/cli_e2e/demo/

作为参考示例。

Core standard

核心标准

Make the testcase scenario-based and self-contained.
Prove one workflow end to end: create plus follow-up read, or mutate plus teardown.
Prefer one file per workflow or one closely related feature.
For mutable flows, prove persisted state with read-after-write assertions, not just exit code.
Leave prerequisite-heavy paths uncovered when they cannot be proven, and explain why in
```
coverage.md
```
.

测试用例需基于场景且具备自包含性。
完整验证一个端到端工作流：创建+后续读取，或修改+清理。
优先为每个工作流或一组紧密相关的功能单独创建一个文件。
对于可变流程，需通过“写后读”断言验证持久化状态，而不仅仅依赖退出码。
若某些依赖前置条件过多的路径无法验证，则暂不覆盖，并在
```
coverage.md
```
中说明原因。

Workflow

工作流程

1. Explore the live CLI before writing code

1. 编写代码前先探索实时CLI

bash

lark-cli --help
lark-cli <domain> --help
lark-cli <domain> +<shortcut> -h
lark-cli <domain> <group> --help
lark-cli <domain> <group> <method> -h
lark-cli schema <domain>.<group>.<method>

bash

lark-cli --help
lark-cli <domain> --help
lark-cli <domain> +<shortcut> -h
lark-cli <domain> <group> --help
lark-cli <domain> <group> <method> -h
lark-cli schema <domain>.<group>.<method>

2. Count leaf commands for the denominator

2. 统计叶子命令数量作为分母

A leaf command is one that executes an action — it has no further subcommands.
If
```
lark-cli <domain> <group> --help
```
lists no subcommands,
```
<group>
```
itself is the leaf.
Count
```
task +create
```
as one leaf and
```
task tasks get
```
as one leaf.
Do not count parameter combinations.
Reuse coverage already present under
```
tests/cli_e2e/{domain}/
```
. Do not count
```
tests/cli_e2e/demo/
```
.

叶子命令指执行具体操作的命令——无进一步子命令。
若
```
lark-cli <domain> <group> --help
```
未列出子命令，则
```
<group>
```
本身即为叶子命令。
将
```
task +create
```
算作一个叶子命令，
```
task tasks get
```
也算作一个叶子命令。
不统计参数组合。
复用
```
tests/cli_e2e/{domain}/
```
下已有的测试覆盖。不统计
```
tests/cli_e2e/demo/
```
中的内容。

3. Choose the proof surface before editing

3. 编写前确定验证范围

Identify the provable risks for the touched workflow: invalid input, missing prerequisite, identity or permission, state transition, output shape, cleanup safety. If only the happy path is testable, document the blocked risk areas in

coverage.md

识别当前工作流中可验证的风险点：无效输入、缺失前置条件、身份或权限、状态转换、输出格式、清理安全性。若仅能测试正常路径，则在

coverage.md

中记录无法覆盖的风险领域。

4. Add or update the workflow testcase

4. 添加或更新工作流测试用例

Use
```
clie2e.RunCmd(ctx, clie2e.Request{...})
```
.
Put command path and plain flags in
```
Args
```
; put JSON in
```
Params
```
(URL/path parameters) and
```
Data
```
(request body).
Prefer one top-level test per workflow with
```
t.Run
```
substeps.
Register teardown on
```
parentT.Cleanup
```
so it survives subtest failures.
When touching an existing command, verify the JSON response shape is stable: assert status type, field paths, and identifiers consumed by later steps before changing assertions.

使用
```
clie2e.RunCmd(ctx, clie2e.Request{...})
```
。
将命令路径和普通标志放入
```
Args
```
；将JSON放入
```
Params
```
（URL/路径参数）和
```
Data
```
（请求体）。
优先为每个工作流创建一个顶层测试，并通过
```
t.Run
```
定义子步骤。
在
```
parentT.Cleanup
```
上注册清理操作，确保即使子测试失败也能执行清理。
修改现有命令时，需先验证JSON响应格式是否稳定：在修改断言前，先断言状态类型、字段路径以及后续步骤会用到的标识符。

5. Run and iterate

5. 运行并迭代

Run

go test ./tests/cli_e2e/{domain} -count=1

while iterating and before finishing. If command shape or behavior is unclear, re-check help or schema (step 1) before changing assertions.

迭代过程中及完成前，运行

go test ./tests/cli_e2e/{domain} -count=1

。若命令格式或行为不明确，先重新查看帮助或schema（步骤1），再修改断言。

6. Refresh the domain outputs

6. 更新领域输出内容

Update the workflow testcase files.
Update
```
coverage.md
```
: recompute the denominator from live help output, mark each command as
```
shortcut
```
or
```
api
```
, and keep one command table for the whole domain.

更新工作流测试用例文件。
更新
```
coverage.md
```
：根据实时帮助输出重新计算分母，标记每个命令为
```
shortcut
```
或
```
api
```
，并为整个领域保留一个命令表格。

Testcase rules

测试用例规则

Override
```
BinaryPath
```
,
```
DefaultAs
```
, or
```
Format
```
on
```
clie2e.Request
```
only when the testcase truly needs it.

Use

require.NoError

result.AssertExitCode

result.AssertStdoutStatus

assert

, and

gjson

Shortcut responses (
```
{ok: bool}
```
) assert
```
true
```
; API responses (
```
{code: int}
```
) assert
```
0
```
.
Use
```
t.Helper()
```
only for setup or assertion helpers that are called from multiple tests.
Use table-driven tests only when the scenario shape repeats across inputs.
For expected failures, assert stderr content and exit code when the environment makes them deterministic.
If identity or external fixtures cannot be proven, leave the command uncovered and document the prerequisite rather than faking confidence.

仅当测试用例确实需要时，才覆盖
```
clie2e.Request
```
中的
```
BinaryPath
```
、
```
DefaultAs
```
或
```
Format
```
。

使用

require.NoError

、

result.AssertExitCode

、

result.AssertStdoutStatus

、

assert

和

gjson

。

快捷方式响应（
```
{ok: bool}
```
）断言为
```
true
```
；API响应（
```
{code: int}
```
）断言为
```
0
```
。
仅在多个测试调用的设置或断言辅助函数中使用
```
t.Helper()
```
。
仅当场景格式在不同输入间重复时，才使用表格驱动测试。
对于预期失败的情况，若环境能保证结果确定性，则断言stderr内容和退出码。
若无法验证身份或外部依赖，则暂不覆盖该命令，并记录前置条件，而非伪造测试通过的假象。

coverage.md

coverage.md规范

Keep

coverage.md

brief and mechanical. Include:

a domain-specific H1 title
a metrics section with denominator, covered count, and coverage rate
a summary section restating each
```
Test...
```
workflow, key
```
t.Run(...)
```
proof points, and main blockers
one command table for all commands

Recommended structure:

markdown

undefined

保持

coverage.md

简洁、结构化，需包含：

领域专属的H1标题
指标部分：分母、已覆盖数量、覆盖率
总结部分：重述每个
```
Test...
```
工作流、关键
```
t.Run(...)
```
验证点，以及主要未覆盖原因
一个包含所有命令的表格

推荐结构：

markdown

undefined

<Domain> CLI E2E Coverage

<Domain> CLI E2E测试覆盖

Metrics

指标

Denominator: N leaf commands
Covered: N
Coverage: N%

分母：N个叶子命令
已覆盖：N个
覆盖率：N%

Summary

总结

TestXxx: ... key
```
t.Run(...)
```
proof points ...
Blocked area: ...

TestXxx：... 关键
```
t.Run(...)
```
验证点 ...
未覆盖领域：...

Command Table

命令表格

Status	Cmd	Type	Testcase	Key parameter shapes	Notes / uncovered reason
✓	task +create	shortcut	task_status_workflow_test.go::TestTask_StatusWorkflow	basic create; create with due
✕	task +assign	shortcut		none	requires real user open_id


- Mark each command `shortcut` or `api`.
- Write testcase entries in `go test -run` friendly form.
- Commands only exercised in `parentT.Cleanup` teardown are not counted as covered.
- Do not split covered and uncovered commands into separate sections.

状态	命令	类型	测试用例	关键参数格式	备注/未覆盖原因
✓	task +create	shortcut	task_status_workflow_test.go::TestTask_StatusWorkflow	基础创建；带截止时间的创建
✕	task +assign	shortcut		无	需要真实用户的open_id


- 标记每个命令为`shortcut`或`api`。
- 测试用例条目采用`go test -run`友好的格式。
- 仅在`parentT.Cleanup`清理操作中执行的命令不算作已覆盖。
- 不要将已覆盖和未覆盖命令拆分为不同章节。

Guardrails

注意事项

Run as bot identity only; do not assume
```
--as user
```
works.
Do not place new real coverage under
```
tests/cli_e2e/demo/
```
.
Do not depend on preexisting remote data.
Do not fabricate open_ids, chats, docs, or other remote fixtures.
Prefer deterministic negative cases over tenant-dependent assertions.
Do not guess
```
Params
```
or
```
Data
```
fields when help or schema can tell you the exact shape.
Do not hardcode obvious defaults unless the command truly requires explicit flags.
Do not put agent, model, or vendor brand names in visible remote test data; use neutral prefixes such as
```
lark-cli-e2e-
```
or
```
<domain>-e2e-
```
.
A command is covered only when the testcase asserts returned fields or persisted state, not just exit code.
Cleanup-only execution is not primary coverage, except
```
delete
```
in the same workflow that created the resource.

仅以机器人身份运行；不要假设
```
--as user
```
可用。
不要将新的真实测试覆盖放在
```
tests/cli_e2e/demo/
```
下。
不要依赖预先存在的远程数据。
不要伪造open_id、聊天、文档或其他远程依赖。
优先选择确定性的异常用例，而非依赖租户的断言。
当帮助或schema能告知确切格式时，不要猜测
```
Params
```
或
```
Data
```
字段。
除非命令明确要求显式标志，否则不要硬编码明显的默认值。
不要在可见的远程测试数据中包含代理、模型或供应商品牌名称；使用中性前缀，如
```
lark-cli-e2e-
```
或
```
<domain>-e2e-
```
。
只有当测试用例断言返回字段或持久化状态时，才算覆盖该命令，而非仅验证退出码。
仅清理操作不算作主要覆盖，除非是在创建资源的同一工作流中的
```
delete
```
操作。