smoke-test

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Smoke Test Skill

冒烟测试技能

Creates a new Mastra project using

create-mastra@<tag>

and performs smoke testing of the Mastra Studio in Chrome.

使用

create-mastra@<tag>

创建新的Mastra项目，并在Chrome中对Mastra Studio执行冒烟测试。

Usage

使用方法

/smoke-test --directory <path> --name <project-name> --tag <version> [--pm <package-manager>] [--llm <provider>]
/smoke-test -d <path> -n <project-name> -t <version> [-p <package-manager>] [-l <provider>]

/smoke-test --directory <path> --name <project-name> --tag <version> [--pm <package-manager>] [--llm <provider>]
/smoke-test -d <path> -n <project-name> -t <version> [-p <package-manager>] [-l <provider>]

Parameters

参数说明

Parameter	Short	Description	Required	Default
`--directory`	`-d`	Parent directory where project will be created	Yes	-
`--name`	`-n`	Project name (will be created as subdirectory)	Yes	-
`--tag`	`-t`	Version tag for create-mastra (e.g., `latest` , `alpha` , `0.10.6` )	Yes	-
`--pm`	`-p`	Package manager: `npm` , `yarn` , `pnpm` , or `bun`	No	`npm`
`--llm`	`-l`	LLM provider: `openai` , `anthropic` , `groq` , `google` , `cerebras` , `mistral`	No	`openai`

参数名	缩写	描述	必填	默认值
`--directory`	`-d`	项目将创建在其中的父目录	是	-
`--name`	`-n`	项目名称（将作为子目录创建）	是	-
`--tag`	`-t`	create-mastra的版本标签（例如： `latest` , `alpha` , `0.10.6` ）	是	-
`--pm`	`-p`	包管理器： `npm` , `yarn` , `pnpm` , 或 `bun`	否	`npm`
`--llm`	`-l`	LLM提供商： `openai` , `anthropic` , `groq` , `google` , `cerebras` , `mistral`	否	`openai`

Examples

示例

undefined

undefined

Minimal (required params only)

最简用法（仅必填参数）

/smoke-test -d ~/projects -n my-test-app -t latest

Full specification

完整参数指定

/smoke-test --directory ~/projects --name my-test-app --tag alpha --pm pnpm --llm anthropic

Using short flags

使用短标志

/smoke-test -d ./projects -n smoke-test-app -t 0.10.6 -p bun -l openai

undefined

/smoke-test -d ./projects -n smoke-test-app -t 0.10.6 -p bun -l openai

undefined

Step 0: Parameter Validation (MUST RUN FIRST)

步骤0：参数验证（必须首先执行）

CRITICAL: Before proceeding, parse the ARGUMENTS and validate:

Parse arguments from the ARGUMENTS string provided above
Check required parameters:
- ```
--directory
```
  or
```
-d
```
  : REQUIRED - fail if missing
- ```
--name
```
  or
```
-n
```
  : REQUIRED - fail if missing
- ```
--tag
```
  or
```
-t
```
  : REQUIRED - fail if missing
Apply defaults for optional parameters:
- ```
--pm
```
  or
```
-p
```
  : Default to
```
npm
```
  if not provided
- ```
--llm
```
  or
```
-l
```
  : Default to
```
openai
```
  if not provided
Validate values:
- ```
pm
```
  must be one of:
```
npm
```
  ,
```
yarn
```
  ,
```
pnpm
```
  ,
```
bun
```
- ```
llm
```
  must be one of:
```
openai
```
  ,
```
anthropic
```
  ,
```
groq
```
  ,
```
google
```
  ,
```
cerebras
```
  ,
```
mistral
```
- ```
directory
```
  must exist (or will be created)
- ```
name
```
  should be a valid directory name (no spaces, special chars)

If validation fails: Stop and show usage help with the missing/invalid parameters.

If
-h
or
--help
is passed: Show this usage information and stop.

关键：在继续之前，解析并验证参数：

解析参数：从上方提供的参数字符串中解析参数
检查必填参数：
- ```
--directory
```
  或
```
-d
```
  ：必填 - 缺失则终止并提示
- ```
--name
```
  或
```
-n
```
  ：必填 - 缺失则终止并提示
- ```
--tag
```
  或
```
-t
```
  ：必填 - 缺失则终止并提示
应用可选参数默认值：
- ```
--pm
```
  或
```
-p
```
  ：未提供则默认使用
```
npm
```
- ```
--llm
```
  或
```
-l
```
  ：未提供则默认使用
```
openai
```
验证参数值：
- ```
pm
```
  必须是以下之一：
```
npm
```
  ,
```
yarn
```
  ,
```
pnpm
```
  ,
```
bun
```
- ```
llm
```
  必须是以下之一：
```
openai
```
  ,
```
anthropic
```
  ,
```
groq
```
  ,
```
google
```
  ,
```
cerebras
```
  ,
```
mistral
```
- ```
directory
```
  必须存在（或会被创建）
- ```
name
```
  必须是有效的目录名称（无空格、特殊字符）

如果验证失败：终止操作并显示包含缺失/无效参数的使用帮助。

如果传入
-h
或
--help
：显示此使用信息并终止操作。

Prerequisites

前置条件

This skill requires the Claude-in-Chrome MCP server for browser automation. Ensure it's configured and running.

此技能需要Claude-in-Chrome MCP服务器进行浏览器自动化。请确保其已配置并运行。

Execution Steps

执行步骤

Step 1: Create the Mastra Project

步骤1：创建Mastra项目

Run the create-mastra command with explicit parameters to avoid interactive prompts:

undefined

使用明确的参数运行create-mastra命令，避免交互式提示：

undefined

For npm

对于npm

npx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

For yarn

对于yarn

yarn create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

For pnpm

对于pnpm

pnpm create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

For bun

对于bun

bunx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e


**Flags explained:**

- `-c agents,tools,workflows,scorers` - Include all components
- `-l <provider>` - Set the LLM provider
- `-e` - Include example code

Being explicit with all parameters ensures the CLI runs non-interactively.

Wait for the installation to complete. This may take 1-2 minutes depending on network speed.

bunx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e


**标志说明**：

- `-c agents,tools,workflows,scorers` - 包含所有组件
- `-l <provider>` - 设置LLM提供商
- `-e` - 包含示例代码

明确指定所有参数可确保CLI以非交互式模式运行。

等待安装完成。根据网络速度，这可能需要1-2分钟。

Step 2: Verify Project Structure

步骤2：验证项目结构

After creation, verify the project has:

```
package.json
```
with mastra dependencies
```
src/mastra/index.ts
```
exporting a Mastra instance
```
.env
```
file (may need to be created)

创建完成后，验证项目包含以下内容：

包含mastra依赖的
```
package.json
```
导出Mastra实例的
```
src/mastra/index.ts
```
```
.env
```
文件（可能需要创建）

Step 2.5: Add Agent Network for Network Mode Testing

步骤2.5：添加Agent网络以进行网络模式测试

To enable Network mode testing, add an agent network configuration:

Create activity-agent.ts in
```
src/mastra/agents/
```
:

typescript

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';

export const activityAgent = new Agent({
  id: 'activity-agent',
  name: 'Activity Agent',
  instructions: `You are a helpful activity planning assistant that suggests activities based on weather conditions.`,
  model: '<provider>/<model>', // e.g., 'openai/gpt-4o'
  memory: new Memory(),
});

Create planner-network.ts in
```
src/mastra/agents/
```
:

typescript

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';
import { weatherAgent } from './weather-agent';
import { activityAgent } from './activity-agent';

export const plannerNetwork = new Agent({
  id: 'planner-network',
  name: 'Planner Network',
  instructions: `You are a coordinator that manages weather and activity agents.`,
  model: '<provider>/<model>',
  agents: { weatherAgent, activityAgent }, // This makes it a network agent
  memory: new Memory(), // Memory is REQUIRED for network agents
});

Update index.ts to register the new agents:

typescript

import { activityAgent } from './agents/activity-agent';
import { plannerNetwork } from './agents/planner-network';

// In Mastra config:
agents: { weatherAgent, activityAgent, plannerNetwork },

Note: Network mode requires

agents

property (sub-agents) and

memory

(mandatory).

要启用网络模式测试，请添加Agent网络配置：

在
src/mastra/agents/
中创建activity-agent.ts：

typescript

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';

export const activityAgent = new Agent({
  id: 'activity-agent',
  name: 'Activity Agent',
  instructions: `You are a helpful activity planning assistant that suggests activities based on weather conditions.`,
  model: '<provider>/<model>', // e.g., 'openai/gpt-4o'
  memory: new Memory(),
});

在
src/mastra/agents/
中创建planner-network.ts：

typescript

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';
import { weatherAgent } from './weather-agent';
import { activityAgent } from './activity-agent';

export const plannerNetwork = new Agent({
  id: 'planner-network',
  name: 'Planner Network',
  instructions: `You are a coordinator that manages weather and activity agents.`,
  model: '<provider>/<model>',
  agents: { weatherAgent, activityAgent }, // 这使其成为网络Agent
  memory: new Memory(), // 网络模式必须配置memory
});

更新index.ts以注册新Agent：

typescript

import { activityAgent } from './agents/activity-agent';
import { plannerNetwork } from './agents/planner-network';

// 在Mastra配置中：
agents: { weatherAgent, activityAgent, plannerNetwork },

注意：网络模式需要

agents

属性（子Agent）和

memory

（必填）。

Step 3: Configure Environment Variables

步骤3：配置环境变量

Based on the selected LLM provider, check for the required API key:

Provider	Required Environment Variable
openai	`OPENAI_API_KEY`
anthropic	`ANTHROPIC_API_KEY`
groq	`GROQ_API_KEY`
google	`GOOGLE_GENERATIVE_AI_API_KEY`
cerebras	`CEREBRAS_API_KEY`
mistral	`MISTRAL_API_KEY`

Check in this order:

Check global environment first: Run
```
echo $<ENV_VAR_NAME>
```
to see if the key is already set globally
- If set globally, the project will inherit it - no
```
.env
```
  file needed
- Skip to Step 4
Check project
.env
file: If not set globally, check if
```
.env
```
exists in the project and contains the key
Ask user only if needed: If the key is not available globally or in
```
.env
```
:
- Ask the user for the API key
- Create the
```
.env
```
  file with the provided key

Only check for the ONE key matching the selected provider - don't check for all providers.

根据所选的LLM提供商，检查所需的API密钥：

提供商	所需环境变量
openai	`OPENAI_API_KEY`
anthropic	`ANTHROPIC_API_KEY`
groq	`GROQ_API_KEY`
google	`GOOGLE_GENERATIVE_AI_API_KEY`
cerebras	`CEREBRAS_API_KEY`
mistral	`MISTRAL_API_KEY`

检查顺序：

首先检查全局环境：运行
```
echo $<ENV_VAR_NAME>
```
查看密钥是否已在全局设置
- 如果已全局设置，项目将自动继承 - 无需创建
```
.env
```
  文件
- 直接跳至步骤4
检查项目的.env文件：如果未全局设置，检查项目中是否存在
```
.env
```
文件且包含该密钥
仅在必要时询问用户：如果密钥未在全局或
```
.env
```
中找到：
- 向用户请求API密钥
- 使用提供的密钥创建
```
.env
```
  文件

仅检查与所选提供商匹配的那一个密钥 - 无需检查所有提供商的密钥。

Step 4: Start the Development Server

步骤4：启动开发服务器

Navigate to the project directory and start the dev server:

cd <directory>/<project-name>
<packageManager> run dev

The server typically starts on

http://localhost:4111

. Wait for the server to be ready before proceeding.

导航到项目目录并启动开发服务器：

cd <directory>/<project-name>
<packageManager> run dev

服务器通常会在

http://localhost:4111

启动。在继续之前，请等待服务器准备就绪。

Step 5: Smoke Test the Studio

步骤5：对Studio进行冒烟测试

Use the Chrome browser automation tools to test the Mastra Studio.

使用Chrome浏览器自动化工具测试Mastra Studio。

5.1 Initial Setup

5.1 初始设置

Get browser context using
```
tabs_context_mcp
```
Create a new tab using
```
tabs_create_mcp
```
Navigate to
```
http://localhost:4111
```

使用
```
tabs_context_mcp
```
获取浏览器上下文
使用
```
tabs_create_mcp
```
创建新标签页
导航至
```
http://localhost:4111
```

5.2 Test Checklist

5.2 测试清单

Perform the following smoke tests using the Chrome automation tools:

Navigation & Basic Loading

Studio loads successfully (page contains "Mastra Studio" or shows agents list)
Take a screenshot of the home page

Agents Page (

/agents

)

Navigate to agents page
Verify at least one agent is listed (the example agent from
```
--default
```
)
Take a screenshot

Agent Detail (

/agents/<agentId>/chat

)

Click on an agent to view details
Verify the agent overview panel loads
Verify model settings panel is visible
Take a screenshot

Agent Chat

Send a test message to the agent (e.g., "Hello, can you help me?")
Wait for response
Verify response appears in the chat
Take a screenshot of the conversation

Network Mode (

/agents/planner-network/chat

)

Navigate to the planner-network agent
Select "Network" in Chat Method settings
Send a message: "What activities can I do in [city] based on the weather?"
Verify network coordination (shows weatherAgent indicator)
Verify completion check shows success
Take a screenshot

Tools Page (

/tools

)

Navigate to tools page
Verify tools list loads (should show weatherTool)
Take a screenshot

Tool Execution (

/tools/weatherTool

)

Click on the weatherTool to open detail page
Find the city input field and enter a test city (e.g., "Tokyo")
Click Submit button
Wait for execution to complete
Verify JSON output appears with weather data (temp, condition, etc.)
Take a screenshot

Workflows Page (

/workflows

)

Navigate to workflows page
Verify workflows list loads (should show weatherWorkflow)
Take a screenshot

Workflow Execution (

/workflows/weatherWorkflow

)

Click on the weatherWorkflow to open detail page
Verify visual graph displays (shows workflow steps)
Find the city input field and enter a test city (e.g., "London")
Click Run button
Wait for execution to complete
Verify steps show success (green checkmarks)
Click to view JSON output modal
Verify execution details with timing appear
Take a screenshot

Settings Page (

/settings

)

Navigate to settings page
Verify settings page loads
Take a screenshot

Observability Page (

/observability

)

Navigate to observability page
Verify traces list shows recent activity (from previous tests)
Click on a trace to view details
Verify timeline view shows steps and timing
Take a screenshot

Scorers Page (

/scorers

)

Navigate to scorers page
Verify scorers list loads (shows 3 example scorers)
Take a screenshot

Scorer Detail (use direct URL navigation)

Navigate directly to
```
/scorers/completeness-scorer
```
(don't click - use URL navigation)
Verify scorer detail page loads with name, description, and scores table
Take a screenshot
Note: Use direct URL navigation for scorer details due to client-side routing timing issues

Additional Pages (verify load only)

Templates page (
```
/templates
```
) - Gallery of starter templates
Request Context page (
```
/request-context
```
) - JSON editor
Processors page (
```
/processors
```
) - Empty state OK
MCP Servers page (
```
/mcps
```
) - Empty state OK

使用Chrome自动化工具执行以下冒烟测试：

导航与基础加载

Studio成功加载（页面包含"Mastra Studio"或显示Agent列表）
截取主页截图

Agent页面 (

/agents

)

导航至Agent页面
验证至少有一个Agent被列出（来自
```
--default
```
的示例Agent）
截取截图

Agent详情 (

/agents/<agentId>/chat

)

点击某个Agent查看详情
验证Agent概览面板加载完成
验证模型设置面板可见
截取截图

Agent聊天

向Agent发送测试消息（例如："Hello, can you help me?"）
等待响应
验证响应出现在聊天窗口中
截取对话截图

网络模式 (

/agents/planner-network/chat

)

导航至planner-network Agent
在聊天方法设置中选择"Network"
发送消息："What activities can I do in [city] based on the weather?"
验证网络协调（显示weatherAgent标识）
验证完成检查显示成功
截取截图

工具页面 (

/tools

)

导航至工具页面
验证工具列表加载完成（应显示weatherTool）
截取截图

工具执行 (

/tools/weatherTool

)

点击weatherTool打开详情页
找到城市输入框并输入测试城市（例如："Tokyo"）
点击提交按钮
等待执行完成
验证显示包含天气数据（温度、状况等）的JSON输出
截取截图

工作流页面 (

/workflows

)

导航至工作流页面
验证工作流列表加载完成（应显示weatherWorkflow）
截取截图

工作流执行 (

/workflows/weatherWorkflow

)

点击weatherWorkflow打开详情页
验证可视化图谱显示（展示工作流步骤）
找到城市输入框并输入测试城市（例如："London"）
点击运行按钮
等待执行完成
验证步骤显示成功（绿色对勾）
点击查看JSON输出模态框
验证显示包含时间信息的执行详情
截取截图

设置页面 (

/settings

)

导航至设置页面
验证设置页面加载完成
截取截图

可观测性页面 (

/observability

)

导航至可观测性页面
验证跟踪列表显示最近的活动（来自之前的测试）
点击某个跟踪查看详情
验证时间线视图显示步骤和时间信息
截取截图

评分器页面 (

/scorers

)

导航至评分器页面
验证评分器列表加载完成（显示3个示例评分器）
截取截图

评分器详情（使用直接URL导航）

直接导航至
```
/scorers/completeness-scorer
```
（不要点击 - 使用URL导航）
验证评分器详情页加载完成，包含名称、描述和评分表
截取截图
注意：由于客户端路由计时问题，评分器详情页请使用直接URL导航

其他页面（仅验证加载）

模板页面 (
```
/templates
```
) - 入门模板库
请求上下文页面 (
```
/request-context
```
) - JSON编辑器
处理器页面 (
```
/processors
```
) - 空状态为正常情况
MCP服务器页面 (
```
/mcps
```
) - 空状态为正常情况

5.3 Report Results

5.3 报告结果

After completing all tests, provide a summary:

Total tests passed/failed
Any errors encountered
Screenshots captured
Recommendations for issues found

完成所有测试后，提供总结：

通过/失败的测试总数
遇到的任何错误
捕获的截图
针对发现问题的建议

Quick Reference

快速参考

Step	Action
Create Project	`cd <directory> && npx create-mastra@<tag> <name> -c agents,tools,workflows,scorers -l <provider> -e`
Install Deps	Automatic during creation
Set Env Vars	Check global env first, then `.env` , ask user only if needed
Start Server	`cd <directory>/<name> && npm run dev`
Studio URL	`http://localhost:4111`

步骤	操作
创建项目	`cd <directory> && npx create-mastra@<tag> <name> -c agents,tools,workflows,scorers -l <provider> -e`
安装依赖	创建过程中自动完成
设置环境变量	首先检查全局环境，然后检查 `.env` ，仅在必要时询问用户
启动服务器	`cd <directory>/<name> && npm run dev`
Studio地址	`http://localhost:4111`

Troubleshooting

故障排除

Server won't start

Verify
```
.env
```
has required API key
Check if port 4111 is available
Try
```
<pm> install
```
to reinstall dependencies

Browser can't connect

Wait a few seconds for server to fully start
Check terminal for server ready message
Verify no firewall blocking localhost

Agent chat fails

Verify API key is valid
Check server logs for errors
Ensure LLM provider API is accessible

服务器无法启动

验证
```
.env
```
包含所需的API密钥
检查端口4111是否可用
尝试
```
<pm> install
```
重新安装依赖

浏览器无法连接

等待几秒让服务器完全启动
检查终端中的服务器就绪消息
验证没有防火墙阻止localhost

Agent聊天失败

验证API密钥有效
检查服务器日志中的错误
确保LLM提供商API可访问

Studio Routes

Studio路由

Feature	Route
Agents	`/agents`
Workflows	`/workflows`
Tools	`/tools`
Scorers	`/scorers`
Observability	`/observability`
MCP Servers	`/mcps`
Processors	`/processors`
Templates	`/templates`
Request Context	`/request-context`
Settings	`/settings`

功能	路由
Agents	`/agents`
Workflows	`/workflows`
Tools	`/tools`
Scorers	`/scorers`
Observability	`/observability`
MCP Servers	`/mcps`
Processors	`/processors`
Templates	`/templates`
Request Context	`/request-context`
Settings	`/settings`

Notes

注意事项

The
```
-e
```
flag includes example agents, making smoke testing meaningful
If the user doesn't specify an LLM provider, default to OpenAI as it's most common
Take screenshots at each major step for documentation/debugging
Keep the dev server running in the background during testing
Always use explicit flags (
```
-c
```
,
```
-l
```
,
```
-e
```
) to ensure non-interactive execution
Network mode requires the
```
agents
```
property AND
```
memory
```
in the Agent constructor
Scorer detail pages may have issues with browser automation but work manually
Observability traces appear automatically after running agents or workflows

```
-e
```
标志包含示例Agent，使冒烟测试更有意义
如果用户未指定LLM提供商，默认使用OpenAI，因为它最常用
在每个主要步骤截取截图，用于文档/调试
在测试过程中保持开发服务器在后台运行
始终使用明确的标志（
```
-c
```
,
```
-l
```
,
```
-e
```
）确保非交互式执行
网络模式要求Agent构造函数中包含
```
agents
```
属性和
```
memory
```
评分器详情页可能在浏览器自动化中存在问题，但手动操作正常
在运行Agent或工作流后，可观测性跟踪会自动显示