Independent, repeatable Huawei Cloud Skill testing framework. Does not depend on skill-creator; can test any existing Huawei Cloud Skill. Focuses on real-environment functional testing and multi-skill orchestration scenarios.

1. Chain Verification — Before each Phase, check that the previous phase's JSON exists; if missing, refuse to execute
2. Agent-proof — Write operations require user confirmation for each item; automatic gate bypassing is not allowed
3. Three-Track Layering — Clear gates between Tiers; Tier 1 must be completed before entering Tier 2
4. Batch Repeatable — Supports

   --skills "skill-a,skill-b"

   or

   --all-installed

5. Fallback Strategy — When only 1 skill, Phase 5/6 automatically downgrade to single-skill lifecycle testing. Sibling auto-scan is ON by default (Phase 5/6 自动从被测 skill 同目录找其他 huawei-cloud-* skill 做编排组合测试). Use

   --no-siblings

   to opt-out.
6. Standardized JSON Output — All phases output in a unified schema; Phase 7 merges into a single report
7. Real-Environment First — All Tier 2 orchestrations execute against real Huawei Cloud; no mocks or simulations

1. hcloud CLI installed and authenticated (for Tier 2 CLI mode testing) — Reference: https://support.huaweicloud.com/qs-hcli/hcli_02_003.html
2. Python 3.8+ +

   huaweicloudsdk

   packages (for SDK mode testing) — SDK Reference: https://console.huaweicloud.com/apiexplorer/#/sdkcenter
3. Huawei Cloud AK/SK — 自动扫描所有以

   HUAWEI

   /

   HW

   /

   HWC

   开头的环境变量（键名含 AK/SK 标记的键值对）。If missing, the framework emits the env-var setup template to stderr and exits 77. NEVER ask the user to type AK/SK in chat; user must set env vars in their shell profile out-of-band and re-run.（完整协议见

   references/agent-protocol.md

   ）
4. Target Skill must be under

   $SKILL_INSTALL_DIR/

   (auto-detected:

   ~/.agents/skills/

   →

   ~/.hermes/skills/

   → default) or a user-specified path
5. jq command (all JSON processing depends on it)
6. API Reference: https://console.huaweicloud.com/apiexplorer/#/openapi

Tier 1: Single-Skill Unit Testing
   Phase 0: Installation Verification (install/uninstall/reinstall)
   Phase 1: Feature Extraction (metadata + commands + resource types + doc checks)
   Phase 2: Technical Research (CLI→SDK→API three-level availability)
   Phase 3: Test Case Generation (functional TC-F + boundary + negative + API TC-A)
   Phase 4: Real-Environment Execution (read-only automatic + write ops require confirmation)

Tier 2: Integration Testing — Real-Environment Orchestration
   Phase 5: Multi-Skill Orchestration (conflict scan / scenario derivation / self-check)
   Phase 6: End-to-End Flow (resource lifecycle: create→query→update→delete)

Tier 3: Final Report
   Phase 7: Consolidated Report (merge phase-0~6 JSON, verdict + issues + reasons)

各 Phase 的完整实现规范（步骤、判定标准、JSON 字段）见

references/phase-details.md

。

详细步骤、链式验证规则、JSON schema 见

references/phase-details.md

与

references/output-schema-spec.md

。

• references/architecture.md

  — Three-track eight-phase architecture diagram (Mermaid)

• references/output-schema-spec.md

  — Complete JSON field specification for each phase (including Phase 7 final report schema)

• references/phase-transition-rules.md

  — Phase transition / fallback / skip rules

• references/acceptance-criteria.md

  — Quality gates + 17-item report acceptance checklist

• references/verification-method.md

  — How to manually verify each phase (PowerShell + Git Bash)

• references/phase-details.md

  — 各 Phase 完整实现规范（步骤、判定标准、JSON 字段）

• references/agent-protocol.md

  — 凭证请求协议（AK/SK 缺失时的完整处理流程）

• references/cli-installation-guide.md

  — How to install and configure hcloud CLI (prerequisite for Phase 2/4)

• references/iam-policies.md

  — Minimum IAM permissions required to run the tester

• templates/phase-report-schema.json

  — JSON Schema for

  phase-N-summary.json

  (N=0..6)

• templates/test-case-schema.json

  — JSON Schema for individual test cases (

  TC-F-*

  /

  TC-A-*

  /

  OF-*

  /

  FF-*

  )

• templates/scenario-template.json

  — Reference example for Phase 6 multi-skill scenario derivation

<skill-name>-test-files/

skills/
├── huawei-cloud-rds-query/                  ← skill source (untouched)
└── huawei-cloud-rds-query-test-files/       ← test artifacts (created on first run, kept across runs)
    ├── phases/
    │   ├── phase-0-summary.json
    │   ├── phase-1-summary.json
    │   ├── ...
    │   └── phase-7-summary.json
    └── reports/
        ├── report-20260724-152309/          ← one subdir per run (timestamped)
        │   ├── test-report.json
        │   └── test-report.md
        ├── report-20260725-090000/
        │   ├── test-report.json
        │   └── test-report.md
        └── ...

• Phase 0~6 output

  phase-N-summary.json

  to

  <skill-name>-test-files/phases/

• Phase 7 merges them into

  <skill-name>-test-files/reports/report-<timestamp>/test-report.{json,md}

• Test artifacts are preserved across runs (no auto-cleanup, ever).

  --fresh

  does NOT delete — it archives old

  phases/*.json

  to

  phases/archive/<timestamp>/

  so the chain check resets while history stays intact.

  reports/

  is always kept.
• The test-installed copy of the skill in

  $SKILL_INSTALL_DIR/

  is still uninstalled on exit, so the next run sees a clean install state.

references/output-schema-spec.md

• Complete Tier 1 before entering Tier 2 to ensure skills are individually functional before orchestration
• Confirm write operations one by one in Phase 4 and Phase 5/6; do not batch-confirm to avoid misoperations
• With only 1 skill, Phase 5/6 automatically downgrade to single-skill closed loop; no need to manually skip
• When using

  --fresh

  to reset and rerun, confirm there are no uncleaned test resources
• Review orchestration scenarios before execution to ensure resource dependency order is correct

• Three-track eight-phase strictly follows sequential order; chain verification prevents skipping
• API endpoints are strictly prohibited from being inferred; only obtain from SDK

  _http_info

  or API Explorer
• Credentials are read from environment variables; hardcoding is prohibited
• If AK/SK is missing, the framework emits the env-var setup template to stderr and exits 77. The Agent MUST output that template to the user verbatim and instruct them to set env vars out-of-band (in their shell / PowerShell $PROFILE). The Agent MUST NEVER ask the user to type or paste AK/SK in chat. Strictly prohibited from silently skipping any step that requires credentials.（完整协议见

  references/agent-protocol.md

  ）
• Resources created during testing must be tracked; if any are left behind, output manual cleanup instructions
• Orchestration scenarios are auto-derived; user should review and confirm before execution
• Write operations in orchestration scenarios require per-step user confirmation

__HUAWEI_SKILL_TESTER_CRED_REQUEST_v1__

77

1. 检测到 exit 77 或 sentinel → 暂停流水线,不得跳过 Phase 5/7
2. 将 stderr 中的 env-var 设置模板原样输出给用户,引导用户在 shell profile / PowerShell $PROFILE 中带外配置(占位符

   <your-access-key-id>

   /

   <your-secret-access-key>

   )
3. 禁止在对话中索要 AK/SK 明文(ask_user / read -p / 剪贴板回传 / 读取 ~/.hcloud/config.json 等路径)
4. 用户带外设置后,重跑失败 phase;用户拒绝则明确标记"live phases skipped — no credentials",不得标记 pass

完整协议(6 步响应流程、直接终端模式、示例行为)见

references/agent-protocol.md

。

• Chain Verification — Each Phase checks the previous phase's JSON to prevent skipping
• Agent-proof — Write operations must be confirmed by the user; fake confirmations are not allowed
• Data-Driven — All phases output in JSON format; Phase 7 merges
• Batch Repeatable — The same set of skills can be tested repeatedly; --fresh resets
• Real-Environment First — All orchestrations and E2E flows execute against real Huawei Cloud; no mocks
• Degrade Without Losing Value — Single skill does not run empty orchestration phases; degrades to meaningful single-skill lifecycle tests
• Resource Safety — Resources created during testing must be tracked; if any remain, output clear manual cleanup instructions
• Credentials Mandatory — If AK/SK is missing, the framework emits the env-var setup template to stderr and exits 77. The Agent MUST output that template to the user and instruct them to set env vars out-of-band. The Agent MUST NEVER ask the user to type or paste AK/SK in chat. Strictly prohibited from silently skipping any step that requires credentials.