jev-desktop

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

jev-desktop

jev-desktop

This skill requires a clone of the agent-desktop repository (the scripts live in
scripts/jev/
and ship in no installed package) and a
TYPESAFE_API_KEY
.
Two entry points over the same screen reading.
run.mjs
takes a goal and drives until it is met.
act.mjs
takes one step and hands the plan back to you. Neither puts the tree in your context.
此 skill 需要克隆 agent-desktop 仓库(脚本位于
scripts/jev/
,不会随任何已安装包一起分发),并设置
TYPESAFE_API_KEY
。
两个入口基于同一套屏幕读取机制。
run.mjs
接收一个目标并持续驱动,直至目标达成。
act.mjs
每次执行一个步骤,并将计划交还给你。二者都不会将树放入你的上下文。

The loop

循环

sh
node scripts/jev/run.mjs --app Finder "open the Applications folder"
node scripts/jev/run.mjs --app TextEdit --cursor --text "A line to write." "write the supplied sentence into the document"
Each turn reads the screen, asks for one operation and a target for that operation in the same request, carries it out, and reads again. It prints one JSON line per turn and one when it stops (shown trimmed).
json
{"turn":{"step":1,"operation":"CLICK","target":"treeitem \"Applications\"",
         "confidence":1,"ok":true,"delivery":"delivered_verified","changed":true}}
{"stop":"done","confidence":0.96}
The operations are
CLICK
,
TYPE_TEXT
,
CHECK
,
UNCHECK
,
EXPAND
,
COLLAPSE
,
SCROLL
,
DRILL
,
WIDEN
,
WAIT
,
DONE
and
BLOCKED
. Only the ones something on screen can receive are offered:
DRILL
for a region that reports it holds more,
WIDEN
only after a
DRILL
.
SCROLL
always scrolls down.
delivery
is
waited
for
WAIT
and
looked
for
DRILL
and
WIDEN
.
The target is asked once per operation.
click_target
is chosen from the elements that advertise
Click
,
type_text_target
from those that advertise
SetValue
, and so on. Only the head matching the chosen operation is read, so a target can never be incompatible with the verb that acts on it.
Looking is an operation. A window is read with
--skeleton
first, so a dense document costs the same first look as a small panel. A region that was cut off reports how much it holds, and
DRILL
pins it as the root for later turns.
WIDEN
gives the whole window back.
CHECK
and
UNCHECK
instead of a toggle.
Both are idempotent: a box already in the wanted state stays there.
You supply the text. Nothing here writes a value, so a run never puts a string on screen that you did not choose. Pass
--text
once per value and they are consumed in order, or pass a function as the
text
option and it is asked for each field with that field's description. When there is no value left to give,
TYPE_TEXT
is not offered at all, so the run stops rather than inventing one.
Text goes in by whichever route the application accepts. A direct value write (
set-value
) is one verified call. Only when the application refuses it and the refusal says a retry is safe does the value go through the clipboard and a
cmd+v
paste (macOS only). The clipboard is put back when the run ends, or emptied if it held anything but text. One key press per character is never used: it drops characters and loses capitals.
What leaves the machine. Every turn posts a description of the screen to the decision endpoint (TypeSafe's
api.typesafe.ai
by default, or whatever
TYPESAFE_BASE_URL
points at): each element's role, its accessible name or description, up to sixty characters of the value it holds, its state, the window title, and the recent actions, excluding supplied text and detailed execution evidence. That is enough to send the contents of a private document or a filled form to the configured remote service. Secure text fields are already withheld by agent-desktop and never reach the request. Pass
--no-values
to either script to withhold what every other field holds as well; targeting gets harder, because a value is often the only thing that tells two unnamed rows apart. The default is to include values.
run.mjs
redacts explicit supplied strings from action turns and later recent-action history, including when later target names or execution evidence repeat them. Live screen descriptions remain separate: if the application displays that text, its accessible name or field value may still reach the remote service under the settings above.
act.mjs
preserves its exact executable
argv
in stdout, so that field can include a value supplied with
--text
; stdout may also contain accessible names and element paths. Choose the endpoint and output destination with that exposure in mind.
A step that is hard to undo is not taken quietly. An ordinary step needs 0.70 confidence in its target, one rated destructive (0.50 or more) needs 0.90, and below 0.55 nothing runs. When the bar is not met the run stops and names the candidate it would have acted on, so you decide instead of it.
act.mjs
gets the rating in its single request.
run.mjs
rates risk with a second model call (counted toward the 80), only for targets at 0.70 up to 0.90, so by default a destructive step at target confidence 0.90 or higher runs unrated. If that call returns no number, the step counts as destructive. Pass
--confirm-destructive
to
run.mjs
or
act.mjs
to stop for confirmation on every step rated destructive, regardless of confidence.
--cursor
makes the run watchable.
It starts a session, shows a cursor that travels to each element before the operation lands, and turns it off at the end. If the session cannot start, the run continues without a cursor. The cursor is drawn where the element is, so bring the application in front of your terminal or it arrives behind it.
It stops on
DONE
, on
BLOCKED
, on a confidence too low for the risk, after 40 operations (
WAIT
,
DRILL
and
WIDEN
count), after 80 model calls (risk ratings count), or after three turns in a row that changed nothing (
WAIT
turns do not count). It also stops with
action_failed
when an operation fails, and with
nothing on this screen can be acted on
when no element advertises an action.
A screen with more actionable elements than a choice can carry says so, in the request and in the turn it reports (
truncated
), and the policy is told to look inside a region rather than call the goal impossible.
--root @ref
starts inside a region when you already know which one.
sh
node scripts/jev/run.mjs --app Finder "open the Applications folder"
node scripts/jev/run.mjs --app TextEdit --cursor --text "A line to write." "write the supplied sentence into the document"
每一轮都会读取屏幕,在同一个请求中询问一项操作及该操作的目标,执行后再重新读取。每轮打印一行 JSON,停止时再打印一行(下面经过精简)。
json
{"turn":{"step":1,"operation":"CLICK","target":"treeitem \"Applications\"",
         "confidence":1,"ok":true,"delivery":"delivered_verified","changed":true}}
{"stop":"done","confidence":0.96}
可用操作包括
CLICK
、
TYPE_TEXT
、
CHECK
、
UNCHECK
、
EXPAND
、
COLLAPSE
、
SCROLL
、
DRILL
、
WIDEN
、
WAIT
、
DONE
和
BLOCKED
。仅提供屏幕内容能够接收的操作:对于报告包含更多内容的区域可使用
DRILL
,而
WIDEN
仅可在
DRILL
之后使用。
SCROLL
始终向下滚动。对于
WAIT
,
delivery
为
waited
;对于
DRILL
和
WIDEN
,则为
looked
。
每个操作只询问一次目标。
click_target
从声明支持
Click
的元素中选择,
type_text_target
从声明支持
SetValue
的元素中选择,依此类推。只读取与所选操作匹配的 head,因此目标绝不会与执行它的动词不兼容。
查看本身是一项操作。 首先使用
--skeleton
读取窗口,因此密集文档的首次查看成本与小面板相同。被截断的区域会报告其包含的内容量,而
DRILL
会将其固定为后续各轮的根节点。
WIDEN
会重新返回整个窗口。
使用
CHECK
和
UNCHECK
,而不是 toggle。
两者都是幂等的:已经处于目标状态的复选框会保持不变。
由你提供文本。 这里不会写入任何值,因此一次运行绝不会在屏幕上写入你未选择的内容。每个值传递一次
--text
,它们会按顺序使用;也可以将函数作为
text
选项传入,此时会针对每个字段询问该字段的描述。当没有剩余可提供的值时,根本不会提供
TYPE_TEXT
,因此运行会停止,而不是自行编造一个值。
文本通过应用接受的方式输入。 直接值写入(
set-value
)是一次经过验证的调用。只有当应用拒绝该写入且拒绝信息明确表示重试安全时,文本才会通过剪贴板和
cmd+v
粘贴(仅限 macOS)输入。运行结束时会恢复剪贴板原内容;如果其中原本保存的不是文本,则会清空剪贴板。绝不会逐字符发送按键,因为这会丢失字符和大写字母。
会离开本机的数据。 每一轮都会向决策端点(默认是 TypeSafe 的
api.typesafe.ai
,或
TYPESAFE_BASE_URL
指向的任何地址)发送屏幕描述:每个元素的 role、可访问名称或描述、最多六十个字符的值、状态、窗口标题以及近期操作,但不包含所提供的文本和详细执行证据。这些信息足以将私密文档内容或已填写表单发送至配置的远程服务。安全文本字段已由 agent-desktop 屏蔽,绝不会进入请求。可向任一脚本传入
--no-values
,以同时屏蔽所有其他字段持有的值;这会使定位变得更难,因为值往往是区分两个未命名行的唯一依据。默认包含值。
run.mjs
会从操作轮次及后续近期操作历史中隐去明确提供的字符串,即使后续目标名称或执行证据再次出现这些字符串也一样。实时屏幕描述仍单独存在:如果应用显示了相应文本,其可访问名称或字段值仍可能根据上述设置发送到远程服务。
act.mjs
会在 stdout 中保留其精确的可执行
argv
,因此该字段可能包含通过
--text
提供的值;stdout 还可能包含可访问名称和元素路径。选择端点和输出目的地时,应将这种数据暴露风险纳入考虑。
难以撤销的步骤不会在无人察觉的情况下执行。 普通步骤要求目标置信度达到 0.70;被评定为破坏性(不低于 0.50)的步骤要求达到 0.90;低于 0.55 时不会执行任何操作。未达到门槛时,运行会停止并指出原本会操作的候选项,由你作出决定。
act.mjs
在其单次请求中获得该评级。
run.mjs
仅针对目标置信度处于 0.70 至 0.90 之间的目标,通过第二次模型调用评估风险(计入 80 次调用上限),因此默认情况下,目标置信度达到或超过 0.90 的破坏性步骤无需评级便执行。如果该调用未返回数值,则该步骤按破坏性处理。向
run.mjs
或
act.mjs
传入
--confirm-destructive
,可使每个被评定为破坏性的步骤都停止并等待确认,无论其置信度如何。
--cursor
让运行过程可见。
它会启动一个会话,显示一个在操作生效前移动到各元素上的光标,并在结束时关闭光标。如果会话无法启动,运行会在不显示光标的情况下继续。光标会绘制在元素所在位置,因此请将应用置于终端前方,否则光标会出现在应用后方。
它会在以下情况停止:遇到
DONE
、遇到
BLOCKED
、置信度低于风险要求、执行 40 次操作后(
WAIT
、
DRILL
和
WIDEN
均计入)、发起 80 次模型调用后(风险评级计入),或连续三轮没有产生任何变化后(
WAIT
轮次不计入)。操作失败时,它也会以
action_failed
停止;当屏幕上没有任何元素声明支持操作时,则以
nothing on this screen can be acted on
停止。
当屏幕中的可操作元素多于一次 choice 可承载的数量时,请求和所报告的轮次(
truncated
)中都会明确说明这一点,同时 policy 会收到指示:应查看某个区域内部,而不是直接判定目标无法实现。
当你已知某个区域时,使用
--root @ref
可从该区域内部开始。

One step at a time

一次处理一个步骤

You keep the goal, the plan and the memory. You send one sentence. You get back one small object.
sh
node scripts/jev/act.mjs --app TextEdit --execute \
  --text "Morning over the dock." "type this text into the main writing area"
json
{
  "ok": true,
  "intent": "type this text into the main writing area",
  "app": "TextEdit", "window": "Untitled 3", "window_id": "w-4521", "surface": "sheet",
  "element": { "ref": "@s1:e139", "role": "textfield", "name": "Save As:",
               "where": "sheet \"save\" > group" },
  "command": "set-value",
  "argv": ["set-value", "@s1:e139", "poem.txt"],
  "decision": "act",
  "why": null,
  "confidence": { "target": 0.99, "command": 0.88 },
  "gates": { "present": 0.94, "destructive": 0.31, "needs_text": 0.87 },
  "runner_up": [{ "ref": "@s1:e137", "what": "textfield", "p": 0.02 }],
  "notes": [],
  "executed": { "ok": true, "delivery": "delivered_verified" }
}
你保留目标、计划和记忆。你发送一句话,收到一个小对象。
sh
node scripts/jev/act.mjs --app TextEdit --execute \
  --text "Morning over the dock." "type this text into the main writing area"
json
{
  "ok": true,
  "intent": "type this text into the main writing area",
  "app": "TextEdit", "window": "Untitled 3", "window_id": "w-4521", "surface": "sheet",
  "element": { "ref": "@s1:e139", "role": "textfield", "name": "Save As:",
               "where": "sheet \"save\" > group" },
  "command": "set-value",
  "argv": ["set-value", "@s1:e139", "poem.txt"],
  "decision": "act",
  "why": null,
  "confidence": { "target": 0.99, "command": 0.88 },
  "gates": { "present": 0.94, "destructive": 0.31, "needs_text": 0.87 },
  "runner_up": [{ "ref": "@s1:e137", "what": "textfield", "p": 0.02 }],
  "notes": [],
  "executed": { "ok": true, "delivery": "delivered_verified" }
}

Calling it

调用方式

FlagMeaning
--app <name>
Required.
--window-id <id>
Read one window of the app, by the id
agent-desktop list-windows --app <name>
reports. Needed only when the app has more than one visible window and none of them is focused, which otherwise fails with
AMBIGUOUS_TARGET
. The run stays on that window: a window the run opens elsewhere is not read. If the window closes after an action,
run.mjs
reports that turn and stops with
window_closed
. A
--root
or an open surface outside the window gives
root_outside_window
or
surface_outside_window
(a stop in
run.mjs
, the
error
in
act.mjs
). Every
run.mjs
stop reports the window it read in
screen.window_id
.
run.mjs
takes it too.
--no-values
Withhold accessible field values from the decision request. Supported by both scripts; the default includes values.
--execute
act.mjs
only. Run the command when
decision
is
act
. Without it, nothing runs.
--text "…"
Text the intent needs. Jev returns choices, never strings.
run.mjs
takes it repeatedly;
act.mjs
takes one value.
--root @ref
Resolve inside one container instead of the whole window.
--bin <path>
act.mjs
only. agent-desktop binary. Defaults to the release build, then
PATH
.
run.mjs
reads
AGENT_DESKTOP_BIN
instead, with the same default.
Environment:
TYPESAFE_API_KEY
is required (both scripts fail with
TYPESAFE_API_KEY unset
).
TYPESAFE_BASE_URL
and
TYPESAFE_MODEL
are covered below.
JEV_VERIFY_MS
sets the paste read-back deadline in ms (default 2000, 0 to 60000).
argv
is the exact agent-desktop command.
double-click
and
hover
carry
--headed
;
scroll
always goes down.
Phrase the intent as an action, not as an element.
"type this text into the main writing area"
resolves to
type
.
"the main writing area"
names no operation and can resolve to
focus
. Describe the target the way a person would,
"the field holding the name the file will be saved under"
, not the way the tree names it.
Flag含义
--app <name>
必填。
--window-id <id>
按 ID 读取应用的某个窗口,该 ID 由
agent-desktop list-windows --app <name>
返回。仅当应用有多个可见窗口且没有窗口获得焦点时需要使用;否则会因
AMBIGUOUS_TARGET
失败。运行将始终停留在该窗口上:在其他位置打开的窗口不会被读取。如果操作后窗口关闭,
run.mjs
会报告该轮并以
window_closed
停止。位于窗口之外的
--root
或打开的 surface 会产生
root_outside_window
或
surface_outside_window
(在
run.mjs
中表示停止,在
act.mjs
中表示
error
)。每次
run.mjs
停止都会在
screen.window_id
中报告所读取的窗口。
run.mjs
也会读取该 ID。
--no-values
从决策请求中屏蔽可访问字段值。两个脚本都支持;默认包含值。
--execute
仅限
act.mjs
。当
decision
为
act
时运行命令。不带该参数时,不会执行任何操作。
--text "…"
意图所需的文本。Jev 返回候选项,而不会返回字符串。
run.mjs
可重复传入;
act.mjs
接收一个值。
--root @ref
在一个容器内解析,而不是在整个窗口内解析。
--bin <path>
仅限
act.mjs
。agent-desktop 可执行文件。默认先使用 release build,然后查找
PATH
。
run.mjs
则读取
AGENT_DESKTOP_BIN
,采用相同默认规则。
环境:两个脚本都要求设置
TYPESAFE_API_KEY
(未设置时会因
TYPESAFE_API_KEY unset
而失败)。下文会介绍
TYPESAFE_BASE_URL
和
TYPESAFE_MODEL
。
JEV_VERIFY_MS
设置粘贴回读截止时间,单位为 ms(默认 2000,范围 0 到 60000)。
argv
是精确的 agent-desktop 命令。
double-click
和
hover
会携带
--headed
;
scroll
始终向下滚动。
将意图表述为操作,而不是元素。
"type this text into the main writing area"
会解析为
type
。
"the main writing area"
没有指定任何操作,因此可能解析为
focus
。应像人一样描述目标,例如
"the field holding the name the file will be saved under"
,而不是按照树的命名方式描述。

Reading
decision

解读
decision

decision
What it meansWhat you do
act
One element clearly matches and the confidence clears the bar for this action's risk.Nothing. With
--execute
it already ran.
confirm
The match is plausible but under the bar.Ask the user, or re-phrase the intent and call again.
abstain
Jev answered
none
, the element is probably not on this screen, or two elements fit equally.
The screen is not where you think. Open the surface you need, then call again.
needs_text
The command takes text and none was supplied.Call again with
--text
.
why
carries the reason in one sentence for every
decision
except
act
(where it is
null
).
runner_up
shows what else it considered, which is usually enough to tell a wrong screen from a vague intent.
decision
含义你该做什么
act
一个元素明确匹配,且置信度达到该操作风险对应的门槛。无需操作。使用
--execute
时命令已经执行。
confirm
匹配有可能成立,但未达到门槛。询问用户,或重新表述意图后再次调用。
abstain
Jev 回答
none
、元素可能不在当前屏幕上,或两个元素同样匹配。
当前屏幕并非你预期的位置。打开所需 surface 后再次调用。
needs_text
命令需要文本,但未提供文本。使用
--text
再次调用。
why
会为除
act
外的每个
decision
提供一句话形式的理由,而
act
的值为
null
。
runner_up
展示还考虑了哪些候选项,通常足以判断是选错了屏幕,还是意图表述得不够明确。

What the script decides, not Jev

由脚本而非 Jev 决定的内容

  • Risk sets the bar. The answer says what; confidence says whether to act. An ordinary action needs 0.70. One that Jev rates
    destructive
    at 0.50 or more needs 0.90: writing a file, deleting, sending, confirming a warning. Below 0.55 nothing acts at all.
  • The command is reconciled against the element. Both questions are answered in parallel and neither sees the other, so code checks the chosen verb against the element's advertised actions. A readonly combobox gets
    click
    , never
    set-value
    . A
    --text
    payload with a non-text verb corrects the verb, because the caller supplying text is evidence Jev does not have.
  • An open surface wins. When a sheet, alert, menu or popover is up, the script reads the window, then that surface, and offers the surface's elements instead of the window's. In the window tree those elements carry
    offscreen
    and would all be dropped.
  • Disabled, hidden and action-less elements are never offered, nor a cell inside a treeitem (the row beside it acts). Everything else is, including unnamed rows: a row is told apart by the value it holds, and a Choice does better with the full list than with a shortlist.
  • 风险决定门槛。 答案说明执行什么;置信度决定是否执行。普通操作需要 0.70。被 Jev 评定为
    destructive
    且评分不低于 0.50 的操作需要 0.90,例如写入文件、删除、发送或确认警告。低于 0.55 时不会执行任何操作。
  • 命令会与元素进行核对。 两个问题会并行回答,且彼此不可见,因此代码会将选定的动词与元素声明支持的操作进行检查。只读 combobox 会得到
    click
    ,绝不会得到
    set-value
    。如果
    --text
    携带的文本与非文本动词不匹配,系统会修正该动词,因为调用方提供文本这一证据是 Jev 无法获得的。
  • 打开的 surface 优先。 当 sheet、alert、menu 或 popover 打开时,脚本会先读取窗口,再读取该 surface,并提供 surface 中的元素,而不是窗口中的元素。在窗口树中,这些元素会带有
    offscreen
    标记,从而全部被丢弃。
  • 禁用、隐藏及不支持操作的元素绝不会提供,treeitem 内部的 cell 也不会提供(由其旁边的 row 执行操作)。其他所有元素均可提供,包括未命名的行:行与行之间通过其持有的值进行区分,而 Choice 使用完整列表的效果优于使用候选项短列表。

How the request is shaped

请求的组织方式

act.mjs
sends one call with five questions. They are answered in parallel, so the speculative ones cost tokens and no latency.
QuestionTypeAsks
target
choiceWhich element the intent refers to, plus
none
.
command
choiceWhich of the 16 interaction verbs it asks for.
present
noulIs the thing on this screen at all?
destructive
noulWould this be hard to undo?
needs_text
noulDoes this need text from the caller?
A Choice accepts 255 options, so up to 254 elements go in one pass (both scripts). When the first
act.mjs
pass lands under 0.70, the top five are re-asked with richer descriptions. A screen with more than 254 elements says so in
notes
; use
--root @ref
.
act.mjs
发起一次调用,其中包含五个问题。这些问题会并行回答,因此推测性问题会消耗额外 token,但不会增加延迟。
问题类型询问内容
target
choice意图指向哪个元素,以及
none
。
command
choice意图要求执行 16 个交互动词中的哪一个。
present
noul目标是否确实位于当前屏幕上。
destructive
noul操作是否难以撤销。
needs_text
noul操作是否需要调用方提供文本。
一个 Choice 最多接受 255 个候选项,因此单次最多可传入 254 个元素(两个脚本均如此)。当
act.mjs
的首次执行结果低于 0.70 时,会使用更丰富的描述重新询问前五个候选项。元素超过 254 个的屏幕会在
notes
中说明;此时应使用
--root @ref
。

Deciding through OpenRouter

通过 OpenRouter 作出决策

The decision endpoint is
TYPESAFE_BASE_URL
, defaulting to TypeSafe's native
https://api.typesafe.ai/v1/systemone
. Configured endpoints must use HTTPS; HTTP is allowed only for
localhost
or a literal loopback IP, which keeps local test routers usable. URL credentials are rejected, and redirects fail closed so the bearer key cannot be forwarded to another endpoint. Point it at OpenRouter's Decisions router to use an OpenRouter key as
TYPESAFE_API_KEY
:
sh
TYPESAFE_BASE_URL=https://openrouter.ai/api/alpha/decisions \
TYPESAFE_MODEL=typesafe/jev-1.13 \
TYPESAFE_API_KEY=sk-or-... \
node scripts/jev/run.mjs --app Finder "open the Applications folder"
TYPESAFE_MODEL
may stay at its
jev-latest
default: against OpenRouter a bare name is sent as the registered
~typesafe/<name>
alias. Or pin a routed id such as
typesafe/jev-1.13
. Do not set
typesafe/jev-latest
: OpenRouter answers
400 Model does not exist
. OpenRouter's Decisions endpoint is alpha and may change shape.
决策端点由
TYPESAFE_BASE_URL
指定,默认使用 TypeSafe 原生的
https://api.typesafe.ai/v1/systemone
。配置的端点必须使用 HTTPS;仅允许对
localhost
或字面量回环 IP 使用 HTTP,以便本地测试路由器继续可用。URL 凭据会被拒绝,重定向也会采用 fail-closed 策略,从而防止 bearer key 被转发到其他端点。要将 OpenRouter key 作为
TYPESAFE_API_KEY
使用,可将端点指向 OpenRouter 的 Decisions router:
sh
TYPESAFE_BASE_URL=https://openrouter.ai/api/alpha/decisions \
TYPESAFE_MODEL=typesafe/jev-1.13 \
TYPESAFE_API_KEY=sk-or-... \
node scripts/jev/run.mjs --app Finder "open the Applications folder"
TYPESAFE_MODEL
可以保留默认的
jev-latest
:连接 OpenRouter 时,纯名称会作为已注册的
~typesafe/<name>
别名发送。也可以固定使用路由 ID,例如
typesafe/jev-1.13
。不要设置
typesafe/jev-latest
,因为 OpenRouter 会返回
400 Model does not exist
。OpenRouter 的 Decisions 端点仍处于 alpha 阶段,其结构可能发生变化。

Known limits

已知限制

  • act.mjs
    reads the screen once per call.
    run.mjs
    reads it before and after each action, and skips the read after
    DRILL
    or
    WIDEN
    . Without a pinned region, each read takes a second snapshot when a sheet, alert, menu or popover is open. A read takes about 3 s on a dense app;
    --root
    is far cheaper.
  • act.mjs
    does not retry: a 429 ends the call.
    run.mjs
    retries 429, 503 and 529 twice, after 500 ms then 1000 ms, before it fails.
  • Held input is unavailable, so no sustained key and no drag with hold. See
    crates/core/src/commands/input_hold_policy.rs
    .
  • press
    is not resolved here. It needs no element, so send it directly.
  • act.mjs
    每次调用只读取一次屏幕。
    run.mjs
    会在每个操作前后读取屏幕,并在
    DRILL
    或
    WIDEN
    后跳过读取。如果没有固定区域,当 sheet、alert、menu 或 popover 打开时,每次读取都需要额外创建一次快照。在密集应用中,一次读取约需 3 秒;使用
    --root
    的成本要低得多。
  • act.mjs
    不会重试:遇到 429 时调用即结束。
    run.mjs
    会在失败前分别等待 500 ms 和 1000 ms,对 429、503 和 529 各重试两次。
  • 无法使用 held input,因此不能执行持续按键,也不能执行带 hold 的拖拽。请参阅
    crates/core/src/commands/input_hold_policy.rs
    。
  • 此处不会解析
    press
    。它不需要元素,因此请直接发送。

Files

文件

FileHolds
scripts/jev/policy.mjs
The operations, how a screen is described, and when to stop. No application is touched here.
scripts/jev/screen.mjs
Which elements a command can reach, which surface is open, how an element is described. Shared by both entry points.
scripts/jev/desktop.mjs
The only code that speaks to agent-desktop.
scripts/jev/run.mjs
The loop and its command line.
scripts/jev/act.mjs
The single-step resolver.
File内容
scripts/jev/policy.mjs
操作、屏幕描述方式以及停止条件。此处不会接触任何应用。
scripts/jev/screen.mjs
某个命令可访问哪些元素、哪个 surface 处于打开状态,以及如何描述元素。由两个入口共享。
scripts/jev/desktop.mjs
唯一与 agent-desktop 通信的代码。
scripts/jev/run.mjs
循环及其命令行。
scripts/jev/act.mjs
单步解析器。

Checks

检查

WAIT
in
run.mjs
pauses 250 ms before the screen is read again. A failed operation stops the run at once as
action_failed
, keeping the error code and delivery state, and is never repeated. A failed focus, clipboard write or paste ends the run the same way. A binary whose failed
set-value
reports no error disposition fails with
BINARY_TOO_OLD
.
run.mjs
exits 0 only when the run stops at
done
; every other stop, and any error (
{"ok":false,"error":…}
), exits 1, and a usage error exits 2.
act.mjs
exits 0 unless the executed command failed or was not executed (
executed.code
is
NOT_EXECUTED
when
decision
is not
act
); an error exits 1 and a usage error exits 2.
After a paste the field is read back. If it does not hold the requested text the run stops with
TEXT_VERIFICATION_FAILED
and does not type again. The read-back compares the exact string, so a field that normalizes text (trims, changes case or line endings) reports a failure even when the text landed. The read is retried until the
JEV_VERIFY_MS
deadline before it fails.
sh
node scripts/jev/act.test.mjs
node scripts/jev/run.test.mjs
node scripts/jev/run.integration.test.mjs
node scripts/jev/evidence.test.mjs
Turns keep native
steps
,
post_state
and
details
for the caller; they are excluded from model history. A verified
AlreadyInState
step with
outcome: "skipped"
and
delivery: "not_delivered"
means no input was needed. A failed postcondition keeps
ok: false
and any nested action evidence.
在
run.mjs
中,
WAIT
会暂停 250 ms,然后再次读取屏幕。操作失败时会立即以
action_failed
停止运行,并保留错误代码和 delivery 状态,且不会重复该操作。焦点设置、剪贴板写入或粘贴失败时,也会以相同方式结束运行。如果二进制文件在
set-value
失败后未报告任何错误处置状态,则会以
BINARY_TOO_OLD
失败。
只有当运行以
done
停止时,
run.mjs
才退出 0;其他所有停止情况及任何错误(
{"ok":false,"error":…}
)均退出 1,使用错误则退出 2。除非执行的命令失败或未被执行,否则
act.mjs
退出 0;当
decision
不是
act
时,
executed.code
为
NOT_EXECUTED
。错误退出 1,使用错误退出 2。
粘贴后会回读字段。如果字段未持有请求的文本,运行会以
TEXT_VERIFICATION_FAILED
停止,且不会再次输入。回读会比较精确字符串,因此即使文本实际已写入,对文本进行规范化处理的字段,例如去除空白、改变大小写或换行符,也会报告失败。读取会持续重试,直到达到
JEV_VERIFY_MS
截止时间后才失败。
sh
node scripts/jev/act.test.mjs
node scripts/jev/run.test.mjs
node scripts/jev/run.integration.test.mjs
node scripts/jev/evidence.test.mjs
各轮会为调用方保留原生的
steps
、
post_state
和
details
;这些内容会从模型历史中排除。经过验证的
AlreadyInState
步骤若带有
outcome: "skipped"
和
delivery: "not_delivered"
,表示无需输入。失败的后置条件会保留
ok: false
以及任何嵌套的操作证据。