Back to Details

agent-device

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Mobile Automation with agent-device

基于agent-device的移动自动化

Quick start

快速开始

bash
agent-device open Settings --platform ios
agent-device snapshot -i
agent-device click @e3
agent-device wait text "Camera"
agent-device alert wait 10000
agent-device fill @e5 "test"
agent-device close
If not installed, run:
bash
npx -y agent-device
bash
agent-device open Settings --platform ios
agent-device snapshot -i
agent-device click @e3
agent-device wait text "Camera"
agent-device alert wait 10000
agent-device fill @e5 "test"
agent-device close
如果尚未安装，请运行：
bash
npx -y agent-device

Core workflow

核心工作流程

  1. Open app or just boot device: 
    open [app]
  2. Snapshot: 
    snapshot
    to get full XCTest accessibility tree snapshot
  3. Interact using refs (
    click @ref
    , 
    fill @ref "text"
    )
  4. Re-snapshot after navigation or UI changes
  5. Close session when done
  1. 打开应用或仅启动设备：
    open [app]
  2. 拍摄快照：使用
    snapshot
    获取完整的XCTest无障碍树快照
  3. 使用引用进行交互（
    click @ref
    、
    fill @ref "text"
    ）
  4. 导航或UI变化后重新拍摄快照
  5. 完成操作后关闭会话

Commands

命令

Navigation

导航

bash
agent-device open [app]           # Boot device/simulator; optionally launch app
agent-device close [app]          # Close app or just end session
agent-device session list         # List active sessions
bash
agent-device open [app]           # 启动设备/模拟器；可选择启动应用
agent-device close [app]          # 关闭应用或仅结束会话
agent-device session list         # 列出活跃会话

Snapshot (page analysis)

快照（页面分析）

bash
agent-device snapshot                  # Full XCTest accessibility tree snapshot
agent-device snapshot -i               # Interactive elements only (recommended)
agent-device snapshot -c               # Compact output
agent-device snapshot -d 3             # Limit depth
agent-device snapshot -s "Camera"      # Scope to label/identifier
agent-device snapshot --raw            # Raw node output
agent-device snapshot --backend xctest # default: XCTest snapshot (fast, complete, no permissions)
agent-device snapshot --backend ax     # macOS Accessibility tree (fast, needs permissions, less fidelity, optional)
XCTest is the default: fast and complete and does not require permissions. Use it in most cases and only fall back to AX when something breaks.
bash
agent-device snapshot                  # 完整的XCTest无障碍树快照
agent-device snapshot -i               # 仅显示可交互元素（推荐使用）
agent-device snapshot -c               # 精简输出
agent-device snapshot -d 3             # 限制深度
agent-device snapshot -s "Camera"      # 按标签/标识符筛选范围
agent-device snapshot --raw            # 原始节点输出
agent-device snapshot --backend xctest # 默认：XCTest快照（快速、完整、无需权限）
agent-device snapshot --backend ax     # macOS无障碍树（快速、需要权限、保真度较低、可选）
XCTest是默认选项：快速、完整且无需权限。大多数情况下使用它，仅在出现问题时再切换到AX。

Find (semantic)

查找（语义化）

bash
agent-device find "Sign In" click
agent-device find text "Sign In" click
agent-device find label "Email" fill "user@example.com"
agent-device find value "Search" type "query"
agent-device find role button click
agent-device find id "com.example:id/login" click
agent-device find "Settings" wait 10000
agent-device find "Settings" exists
bash
agent-device find "Sign In" click
agent-device find text "Sign In" click
agent-device find label "Email" fill "user@example.com"
agent-device find value "Search" type "query"
agent-device find role button click
agent-device find id "com.example:id/login" click
agent-device find "Settings" wait 10000
agent-device find "Settings" exists

Settings helpers (simulators)

模拟器设置助手

bash
agent-device settings wifi on
agent-device settings wifi off
agent-device settings airplane on
agent-device settings airplane off
agent-device settings location on
agent-device settings location off
Note: iOS wifi/airplane toggles status bar indicators, not actual network state. Airplane off clears status bar overrides.
bash
agent-device settings wifi on
agent-device settings wifi off
agent-device settings airplane on
agent-device settings airplane off
agent-device settings location on
agent-device settings location off
注意：iOS的wifi/飞行模式开关仅切换状态栏指示器，不改变实际网络状态。关闭飞行模式会清除状态栏覆盖设置。

App state

应用状态

bash
agent-device appstate
agent-device apps --metadata --platform ios
agent-device apps --metadata --platform android
bash
agent-device appstate
agent-device apps --metadata --platform ios
agent-device apps --metadata --platform android

Interactions (use @refs from snapshot)

交互操作（使用快照中的@引用）

bash
agent-device click @e1
agent-device focus @e2
agent-device fill @e2 "text"           # Tap then type
agent-device type "text"               # Type into focused field
agent-device press 300 500             # Tap by coordinates
agent-device long-press 300 500 800    # Long press (where supported)
agent-device scroll down 0.5
agent-device back
agent-device home
agent-device app-switcher
agent-device wait 1000
agent-device wait text "Settings"
agent-device alert get
bash
agent-device click @e1
agent-device focus @e2
agent-device fill @e2 "text"           # 点击后输入
agent-device type "text"               # 在已聚焦的输入框中输入
agent-device press 300 500             # 按坐标点击
agent-device long-press 300 500 800    # 长按（支持的设备）
agent-device scroll down 0.5
agent-device back
agent-device home
agent-device app-switcher
agent-device wait 1000
agent-device wait text "Settings"
agent-device alert get

Get information

获取信息

bash
agent-device get text @e1
agent-device get attrs @e1
agent-device screenshot --out out.png
bash
agent-device get text @e1
agent-device get attrs @e1
agent-device screenshot --out out.png

Trace logs (AX/XCTest)

跟踪日志（AX/XCTest）

bash
agent-device trace start               # Start trace capture
agent-device trace start ./trace.log   # Start trace capture to path
agent-device trace stop                # Stop trace capture
agent-device trace stop ./trace.log    # Stop and move trace log
bash
agent-device trace start               # 开始跟踪捕获
agent-device trace start ./trace.log   # 开始跟踪捕获并保存到指定路径
agent-device trace stop                # 停止跟踪捕获
agent-device trace stop ./trace.log    # 停止跟踪并移动日志文件

Devices and apps

设备与应用

bash
agent-device devices
agent-device apps --platform ios
agent-device apps --platform android          # default: launchable only
agent-device apps --platform android --all
agent-device apps --platform android --user-installed
bash
agent-device devices
agent-device apps --platform ios
agent-device apps --platform android          # 默认：仅显示可启动的应用
agent-device apps --platform android --all
agent-device apps --platform android --user-installed

Best practices

最佳实践

  • Always snapshot right before interactions; refs invalidate on UI changes.
  • Prefer 
    snapshot -i
    to reduce output size.
  • On iOS, 
    xctest
    is the default and does not require Accessibility permission.
  • If XCTest returns 0 nodes (foreground app changed), agent-device falls back to AX when available.
  • open <app>
    can be used within an existing session to switch apps and update the session bundle id.
  • If AX returns the Simulator window or empty tree, restart Simulator or use 
    --backend xctest
    .
  • Use 
    --session <name>
    for parallel sessions; avoid device contention.
  • 始终在交互前拍摄快照；UI变化后引用会失效。
  • 优先使用
    snapshot -i
    以减小输出体积。
  • 在iOS上，
    xctest
    是默认选项，无需无障碍权限。
  • 如果XCTest返回0个节点（前台应用已更改），agent-device会在可用时自动回退到AX。
  • 可在现有会话中使用
    open <app>
    切换应用并更新会话的Bundle ID。
  • 如果AX返回模拟器窗口或空树，请重启模拟器或使用
    --backend xctest
    。
  • 使用
    --session <name>
    进行并行会话；避免设备资源冲突。

References

参考资料

  • references/snapshot-refs.md
  • references/session-management.md
  • references/permissions.md
  • references/recording.md
  • references/coordinate-system.md
  • references/snapshot-refs.md
  • references/session-management.md
  • references/permissions.md
  • references/recording.md
  • references/coordinate-system.md

Missing features roadmap (high level)

待实现功能路线图（概览）

See references/missing-features.md for planned parity with agent-browser.
请查看references/missing-features.md了解计划实现的与agent-browser的功能对齐内容。