Back to Details

file-test-bug

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

File Test Bug

提交测试Bug

Creates a GitHub issue in 
microsoft/github-copilot-for-azure
for integration test failures.
为集成测试失败在
microsoft/github-copilot-for-azure
仓库创建GitHub Issue。

Input

输入

  • Skill name (required): e.g., 
    azure-role-selector
    , 
    appinsights-instrumentation
  • Test run (optional): Timestamp of test run. Defaults to most recent with logs for the skill.
  • 技能名称（必填）：例如
    azure-role-selector
    、
    appinsights-instrumentation
  • 测试运行时间（可选）：测试运行的时间戳。默认使用该技能最近带日志的测试运行记录。

Steps

步骤

  1. Ask user for skill name if not provided
  2. Parse 
    tests/reports/junit.xml
    for failures matching the skill
  3. Find test run directory (specified or most recent with matching logs)
  4. Read 
    agent-metadata.md
    from 
    tests/reports/test-run-<timestamp>/<skillname>-<testname>/
  5. For each failure, read the actual line of code from the test file using the location (file:line) from junit.xml
  6. REQUIRED - Write diagnosis BEFORE creating issue:
    • Analyze the agent-metadata.md to understand what the agent did
    • Compare agent behavior to what the test expected (from the assertion)
    • Identify the root cause (skill issue, test issue, or model behavior)
    • Write 2-3 sentences per failed test explaining WHY it failed
    • Suggest potential fixes (update skill, update test, or update fixtures)
  7. Create issue via 
    github-mcp-server-create_issue
    :
owner: microsoft
repo: github-copilot-for-azure
title: Integration test failure in <skill-name>
labels: ["bug", "integration-test"]
body: |
  ## Failed Tests
  - <test-name>: <error message>
  
  ## Diagnosis
  
  ### Root Cause
  <1-2 sentences explaining WHY the test failed based on agent-metadata.md analysis>
  
  ### Analysis per Test
  - **<test-name>**: <what agent did vs what test expected>
  
  ### Suggested Fix
  <one of: update skill, update test assertions, provide test fixtures>

  ## Details
  ### <test-name>
  **Error:** <failure from junit.xml>
  **Location:** <file:line>
  ```typescript
  <actual line of code from the test file at the specified line number>
<details> <summary>agent-metadata.md</summary>
<full contents of agent-metadata.md file, verbatim>
</details> ```
  1. 若未提供技能名称，询问用户获取
  2. 解析
    tests/reports/junit.xml
    ，查找与该技能匹配的失败用例
  3. 找到测试运行目录（指定的目录或最近带匹配日志的目录）
  4. 从
    tests/reports/test-run-<timestamp>/<skillname>-<testname>/
    路径读取
    agent-metadata.md
    文件
  5. 针对每个失败用例，利用junit.xml中记录的位置（文件:行号）从测试文件中读取对应的代码行
  6. 必填 - 创建Issue前先撰写诊断内容：
    • 分析agent-metadata.md文件，理解Agent的执行行为
    • 对比Agent的实际行为与测试预期结果（来自断言信息）
    • 确定根本原因（技能问题、测试问题或模型行为问题）
    • 每个失败测试用例撰写2-3句话解释失败原因
    • 提出潜在修复方案（更新技能、更新测试或更新测试夹具）
  7. 通过
    github-mcp-server-create_issue
    创建Issue：
owner: microsoft
repo: github-copilot-for-azure
title: Integration test failure in <skill-name>
labels: ["bug", "integration-test"]
body: |
  ## Failed Tests
  - <test-name>: <error message>
  
  ## Diagnosis
  
  ### Root Cause
  <1-2 sentences explaining WHY the test failed based on agent-metadata.md analysis>
  
  ### Analysis per Test
  - **<test-name>**: <what agent did vs what test expected>
  
  ### Suggested Fix
  <one of: update skill, update test assertions, provide test fixtures>

  ## Details
  ### <test-name>
  **Error:** <failure from junit.xml>
  **Location:** <file:line>
  ```typescript
  <actual line of code from the test file at the specified line number>
<details> <summary>agent-metadata.md</summary>
<full contents of agent-metadata.md file, verbatim>
</details> ```

Important

重要说明

Include the complete, unmodified contents of each 
agent-metadata.md
file in the issue body. Do NOT summarize or truncate the logs. Wrap each log in a 
<details>
block with the test name as the summary.
在Issue正文中包含每个
agent-metadata.md
文件的完整、未修改内容。请勿总结或截断日志。将每个日志包裹在
<details>
块中，以测试用例名称作为摘要。