Back to Details

caveman-manage

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Manage eval-gated experiments

管理评估门控实验

Treat every lifecycle change as a production control action. Read current state and results, then report one supported recommendation or block. Current agent MCP is intentionally read-only: control-api does not yet enforce a complete lifecycle transition table and evidence gate atomically.
将每一项生命周期变更视为生产控制操作。读取当前状态和结果，然后给出一项受支持的建议或阻止操作。当前Agent的MCP为只读模式：control-api尚未以原子方式强制执行完整的生命周期流转表和证据门控。

Non-negotiable gates

不可妥协的管控规则

  1. A request to review, inspect, explain, or recommend authorizes reads only.
  2. Never approve an experiment whose results are pending, whose required guardrails are absent, or whose evidence reports a breach.
  3. Never convert experiment lift into 
    verified_savings
    . Only active real traffic plus provider-causal, provider-complete ledger evidence can do that.
  4. Never supply an organization id. Project and tenant scope come from the logged-in Caveman identity and server RBAC.
  5. Never execute a lifecycle mutation, even after user approval. Exact 
    <action>:<experiment_id>
    strings are agent-generatable and are not proof of human intent.
  6. Unknown states and server errors fail closed. Report exact 
    cave_snake_code
    .
  1. 仅允许针对查看、检查、解释或建议类请求执行读取操作。
  2. 绝不批准结果待处理、缺少必要防护机制或证据显示存在违规的实验。
  3. 不得将实验收益转换为
    verified_savings
    。只有活跃真实流量加上供应商因果性、完整的账本证据才能完成此操作。
  4. 不得提供组织ID。项目和租户范围来自已登录的Caveman身份及服务器RBAC权限控制。
  5. 即使获得用户批准，也不得执行生命周期变更操作。精确的
    <action>:<experiment_id>
    字符串可由Agent生成，不能作为人工意图的证明。
  6. 未知状态和服务器错误需执行关闭操作。请报告精确的
    cave_snake_code
    。

Step 1 — Load project and experiment

步骤1 — 加载项目和实验

Prefer MCP:
text
caveman_context {}
caveman_experiment_get {"action":"get","experiment_id":"<id>"}
caveman_experiment_get {"action":"results","experiment_id":"<id>"}
Use 
{"action":"list"}
when the user has not named an id.
CLI fallback:
bash
caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>
Stop if login, project, experiment, or results are unavailable.
优先使用MCP：
text
caveman_context {}
caveman_experiment_get {"action":"get","experiment_id":"<id>"}
caveman_experiment_get {"action":"results","experiment_id":"<id>"}
当用户未指定ID时，使用
{"action":"list"}
。
CLI备选方案：
bash
caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>
若登录、项目、实验或结果不可用，则停止操作。

Step 2 — Evaluate evidence

步骤2 — 评估证据

Report:
  • current lifecycle state and safety class;
  • control and candidate sample sizes;
  • quality or eval result;
  • latency, error, cost, retry, drop, and escalation guardrails when present;
  • evidence cost;
  • rollback or hold reason;
  • whether result is pending, failed, promotable, or active.
Absence is not a pass. If a required field is absent, state 
evidence incomplete
and do not propose approval.
需报告：
  • 当前生命周期状态和安全等级；
  • 对照组和候选组样本量；
  • 质量或评估结果；
  • （若存在）延迟、错误、成本、重试、丢弃和升级防护机制；
  • 证据成本；
  • 回滚或暂停原因；
  • 结果是否处于待处理、失败、可推广或活跃状态。
缺失不等于通过。若必填字段缺失，需标注
evidence incomplete
且不得提议批准。

Step 3 — Propose one action

步骤3 — 提议一项操作

Allowed actions:
  • start
    — only from a startable draft or queued state with configured graders;
  • approve
    — only with complete passing evidence and a safety class the current role may approve;
  • cancel
    — stop a non-active experiment the user no longer wants;
  • rollback
    — revert an active or harmful change through the server's linked policy path. Current deployments may reject this honestly with 
    cave_not_implemented
    ; never describe that response as a rollback.
Show recommendation and id:
text
Proposed action: approve experiment 7f...
Reason: candidate passed quality and every configured guardrail.
Execution: blocked until server-authoritative lifecycle and evidence gates ship.
Do not treat earlier generic statements such as "manage it" or "do what is best" as mutation approval.
允许的操作：
  • start
    — 仅适用于已配置评分器的可启动草稿或排队状态；
  • approve
    — 仅当证据完整且通过，且当前角色有权批准对应安全等级时可用；
  • cancel
    — 停止用户不再需要的非活跃实验；
  • rollback
    — 通过服务器关联的策略路径恢复活跃或有害变更。当前部署可能会以
    cave_not_implemented
    为由拒绝此操作；切勿将该响应描述为回滚。
展示建议和ID：
text
Proposed action: approve experiment 7f...
Reason: candidate passed quality and every configured guardrail.
Execution: blocked until server-authoritative lifecycle and evidence gates ship.
请勿将诸如“管理它”或“做最优操作”之类的早期通用表述视为变更批准。

Step 4 — Block unsafe execution

步骤4 — 阻止不安全执行

Do not emit or run an executable lifecycle command. Explain that current server does not yet enforce every evidence/state transition atomically. CLI and MCP agent surfaces therefore expose experiment reads only.
不得发出或执行可执行的生命周期命令。需说明当前服务器尚未以原子方式强制执行所有证据/状态流转。因此CLI和MCP Agent界面仅支持实验读取操作。

Step 5 — Re-read after external operator action

步骤5 — 外部操作员操作后重新读取

If operator says they executed command, read detail and results again. Report server-observed post-state, audit or result response, and any policy-delivery status returned. Never infer success from operator intent alone.
Use this close:
text
Action: <action> <experiment-id>
Before: <state>
Server response: <status and cave_snake_code if any>
After: <re-read state>
Basis: experiment evidence only. Verified savings unchanged unless the signed
ledger independently records active, provider-causal real-traffic savings.
若操作员表示已执行命令，请重新读取详情和结果。报告服务器观测到的后续状态、审计或结果响应，以及返回的任何策略交付状态。切勿仅根据操作员意图推断操作成功。
使用以下结束语：
text
Action: <action> <experiment-id>
Before: <state>
Server response: <status and cave_snake_code if any>
After: <re-read state>
Basis: experiment evidence only. Verified savings unchanged unless the signed
ledger independently records active, provider-causal real-traffic savings.