Back to Details

control-ui

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Control UI

控制UI

Use local browser automation to verify UI behavior with evidence. First reuse the repo's own Playwright, browser, or Electron harness if it exists; otherwise assemble a temporary local harness around the app's dev server or Chromium debug port.
使用本地浏览器自动化功能，结合验证证据来确认UI行为。首先复用仓库中已有的Playwright、浏览器或Electron工具集；若没有，则围绕应用的开发服务器或Chromium调试端口搭建临时本地工具集。

What It Is Used For

适用场景

  • Reproducing UI bugs that depend on real browser focus, keyboard input, scrolling, resizing, or rendering.
  • Verifying visual or accessibility changes with screenshots and snapshots.
  • Checking local web, IDE, or Electron behavior before shipping.
  • Capturing console logs, network logs, CPU profiles, traces, or heap snapshots.
  • Creating before/after evidence for 
    verify-this
    .
  • 复现依赖真实浏览器焦点、键盘输入、滚动、缩放或渲染的UI Bug。
  • 通过截图和快照验证视觉或无障碍相关变更。
  • 发布前检查本地网页、IDE或Electron应用的行为。
  • 捕获控制台日志、网络日志、CPU分析报告、追踪信息或堆快照。
  • 为
    verify-this
    创建操作前后的验证证据。

Setup Pattern

搭建模式

  1. Start the app locally using the repo's documented dev command.
  2. Discover existing local harnesses: Playwright tests, Cypress specs, Storybook, browser scripts, Electron launch scripts, or snapshot tools.
  3. For a web app, connect to the local URL with the existing browser tooling.
  4. For Electron/Chromium, enable a remote debugging port when supported.
  5. Select the correct page by stable app markers, not by tab order alone.
  6. Prefer accessibility roles, labels, and stable 
    data-*
    selectors over coordinates.
  1. 使用仓库文档中记录的开发命令在本地启动应用。
  2. 查找已有的本地工具集：Playwright测试用例、Cypress测试脚本、Storybook、浏览器脚本、Electron启动脚本或快照工具。
  3. 对于Web应用，使用现有浏览器工具连接到本地URL。
  4. 对于Electron/Chromium应用，若支持则启用远程调试端口。
  5. 通过稳定的应用标记选择正确页面，不要仅依赖标签页顺序。
  6. 优先使用无障碍角色、标签和稳定的
    data-*
    选择器，而非坐标定位。

Generic Web Harness

通用Web工具集

Use the repo's installed browser tooling when possible. If the repo already has Playwright, a minimal one-off probe looks like:
javascript
import { chromium } from "playwright";

const browser = await chromium.launch();
const page = await browser.newPage({ viewport: { width: 1280, height: 800 } });
await page.goto("http://127.0.0.1:<port>");
await page.getByRole("button", { name: /submit/i }).click();
await page.screenshot({ path: "/tmp/ui-harness-after.png", fullPage: true });
await browser.close();
Do not add Playwright as a project dependency just for this probe unless the user asks. Prefer existing dev dependencies or external browser tools already available in the environment.
尽可能使用仓库中已安装的浏览器工具。若仓库已配置Playwright，一个简易的一次性探测脚本如下：
javascript
import { chromium } from "playwright";

const browser = await chromium.launch();
const page = await browser.newPage({ viewport: { width: 1280, height: 800 } });
await page.goto("http://127.0.0.1:<port>");
await page.getByRole("button", { name: /submit/i }).click();
await page.screenshot({ path: "/tmp/ui-harness-after.png", fullPage: true });
await browser.close();
除非用户要求，否则不要仅为该探测脚本添加Playwright作为项目依赖。优先使用已有的开发依赖或环境中已有的外部浏览器工具。

Generic CDP Harness

通用CDP工具集

For Electron or a Chromium app launched with 
--remote-debugging-port=<port>
, connect over CDP:
javascript
import { chromium } from "playwright";

const browser = await chromium.connectOverCDP("http://127.0.0.1:<debug-port>");
const pages = browser.contexts().flatMap((context) => context.pages());
let page;
for (const candidate of pages) {
  if (await candidate.locator("<app-root-selector>").count()) {
    page = candidate;
    break;
  }
}

if (!page) {
  console.log(await Promise.all(pages.map(async (p) => ({
    title: await p.title(),
    url: p.url(),
  }))));
  throw new Error("No matching app page found");
}

await page.screenshot({ path: "/tmp/ui-harness-cdp.png", fullPage: true });
await browser.close();
Replace 
<app-root-selector>
with a stable marker from the current repo, such as a root app node, landmark, or product-specific 
data-*
attribute.
对于通过
--remote-debugging-port=<port>
启动的Electron或Chromium应用，可通过CDP连接：
javascript
import { chromium } from "playwright";

const browser = await chromium.connectOverCDP("http://127.0.0.1:<debug-port>");
const pages = browser.contexts().flatMap((context) => context.pages());
let page;
for (const candidate of pages) {
  if (await candidate.locator("<app-root-selector>").count()) {
    page = candidate;
    break;
  }
}

if (!page) {
  console.log(await Promise.all(pages.map(async (p) => ({
    title: await p.title(),
    url: p.url(),
  }))));
  throw new Error("No matching app page found");
}

await page.screenshot({ path: "/tmp/ui-harness-cdp.png", fullPage: true });
await browser.close();
将
<app-root-selector>
替换为当前仓库中的稳定标记，例如根应用节点、地标或产品专属的
data-*
属性。

Interaction Loop

交互流程

  1. Capture a page snapshot or screenshot before acting.
  2. Choose a target from the latest page structure.
  3. Perform exactly one structural action: click, type, keypress, drag, scroll, navigate, or resize.
  4. Capture a fresh snapshot/screenshot.
  5. Verify the expected state change.
  6. Save artifacts for before/after comparisons when the user asked for proof.
  1. 执行操作前捕获页面快照或截图。
  2. 根据最新页面结构选择目标元素。
  3. 执行恰好一项结构化操作：点击、输入、按键、拖拽、滚动、导航或缩放。
  4. 捕获新的快照/截图。
  5. 验证预期的状态变更。
  6. 当用户要求提供证明时，保存用于前后对比的工件。

CDP Capabilities

CDP功能

Use raw CDP only when higher-level browser APIs are insufficient:
  • Performance: CPU profiles, traces, paint flashing, FPS meter, layout shift inspection.
  • Memory: heap snapshots and forced GC for leak investigations.
  • Network: request blocking, throttling, cache disablement, request/response logs.
  • Rendering: viewport changes, color scheme emulation, reduced motion, accessibility checks.
  • Debugging: console streaming, exception capture, DOM snapshots.
仅当更高层级的浏览器API无法满足需求时，才使用原生CDP：
  • 性能：CPU分析报告、追踪信息、绘制闪烁、FPS计数器、布局偏移检查。
  • 内存：堆快照和强制垃圾回收，用于内存泄漏排查。
  • 网络：请求拦截、流量限制、缓存禁用、请求/响应日志。
  • 渲染：视口变更、配色方案模拟、减少动画、无障碍检查。
  • 调试：控制台流、异常捕获、DOM快照。

Page Selection

页面选择

When multiple app windows/tabs share a debug port:
  • Prefer a positive marker for the surface under test, such as an app root selector.
  • Use a negative marker to avoid the wrong surface when necessary.
  • If no page matches, list available page titles and URLs instead of guessing.
当多个应用窗口/标签页共享同一调试端口时：
  • 优先使用被测界面的正向标记，例如应用根选择器。
  • 必要时使用负向标记排除错误界面。
  • 若无匹配页面，列出可用页面的标题和URL，而非猜测。

Guardrails

注意事项

  • Do not rely on stale element references after navigation or structural changes.
  • Avoid coordinate clicks unless a fresh screenshot was captured immediately before the click.
  • Keep test data local and disposable.
  • Do not store screenshots or heap snapshots from privacy-sensitive workspaces unless the user explicitly agrees.
  • Do not hard-code selectors, ports, or script paths from another repository. Discover the current repo's local app markers.
  • Clean up dev servers, debug sessions, and temp profiles when done.
  • 导航或结构变更后，不要依赖过期的元素引用。
  • 除非点击前刚捕获了新截图，否则避免使用坐标点击。
  • 测试数据需保持本地可丢弃状态。
  • 除非用户明确同意，否则不要存储来自隐私敏感工作区的截图或堆快照。
  • 不要硬编码来自其他仓库的选择器、端口或脚本路径。需自动发现当前仓库的本地应用标记。
  • 完成操作后清理开发服务器、调试会话和临时配置文件。