huawei-cloud-cce-alarm-correlation-engine

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Huawei Cloud CCE Alarm Correlation Engine

华为云CCE告警关联引擎

Overview

概述

Correlate Huawei Cloud AOM active and historical alarms for CCE operations, then convert raw alarm streams into prioritized investigation leads. This skill also manages AOM alarm rules and notification action rules through a strict preview-and-confirm workflow.
Architecture:
python3 scripts/huawei-cloud.py
dispatcher -> local
hcloud
(KooCLI) -> AOM/CCE/IAM cloud service operations -> alarm query, alarm correlation, alarm-rule management, notification-rule management, and CCE alarm health inspection.
Execution Method: All Huawei Cloud operations must go through the bundled Python dispatcher. The dispatcher invokes the local
hcloud
CLI. Direct SDK imports, hand-written API signing, curl IAM flows, openstack commands, or out-of-band cloud access paths are prohibited.
Related Skills:
  • huawei-cloud-cce-metric-analyzer
    - CCE, AOM, and cloud-resource metric checks
  • huawei-cloud-cce-kubernetes-event-analyzer
    - Kubernetes warning event analysis
  • huawei-cloud-cce-pod-failure-diagnoser
    - Pod failure diagnosis
  • huawei-cloud-cce-node-failure-diagnoser
    - Node failure diagnosis
  • huawei-cloud-cce-auto-remediation-runner
    - Remediation preview and execution
Capabilities:
  • Query active AOM alarms, historical AOM alarms, and merged active+historical views
  • Analyze alarm deduplication, severity grouping, burst alarms, attention alarms, and chronic alarms
  • Inspect CCE alarm health and produce risk items for follow-up diagnosis
  • Query AOM alarm rules with optional
    cluster_id
    filtering
  • Resolve the AOM Prometheus instance bound to a CCE cluster
  • Create metric alarm rules and event alarm rules
  • Create notification action rules from a user-provided SMN topic
  • Batch configure CCE recommended alarm rules from the cloud-side CCE alarm template
  • Batch clean CCE alarm rules matching the cloud-side CCE alarm template
  • Update, delete, enable, and disable AOM alarm rules
  • Query AOM action rules and mute rules
Typical Use Cases:
  • "List AOM alarms for this CCE cluster"
  • "Analyze active and historical alarms for alarm storms"
  • "Show current alarm rules for cluster
    <cluster_id>
    "
  • "Create CCE recommended alarm rules with this notification rule"
  • "Clean CCE template alarm rules for this cluster"
  • "Create a notification action rule using this SMN topic"
  • "Disable this noisy alarm rule after I confirm"
为CCE运维关联华为云AOM当前告警与历史告警,将原始告警流转换为优先级明确的排查线索。本技能还通过严格的预览确认流程管理AOM告警规则与通知动作规则。
架构
python3 scripts/huawei-cloud.py
调度器 -> 本地
hcloud
(KooCLI)-> AOM/CCE/IAM云服务操作 -> 告警查询、告警关联、告警规则管理、通知规则管理及CCE告警健康检查。
执行方式:所有华为云操作必须通过捆绑的Python调度器执行。调度器调用本地
hcloud
CLI。禁止直接导入SDK、手动编写API签名、curl IAM流程、openstack命令或带外云访问路径。
相关技能:
  • huawei-cloud-cce-metric-analyzer
    - CCE、AOM及云资源指标检查
  • huawei-cloud-cce-kubernetes-event-analyzer
    - Kubernetes警告事件分析
  • huawei-cloud-cce-pod-failure-diagnoser
    - Pod故障诊断
  • huawei-cloud-cce-node-failure-diagnoser
    - Node故障诊断
  • huawei-cloud-cce-auto-remediation-runner
    - 修复预览与执行
功能:
  • 查询AOM当前告警、历史告警及合并的当前+历史视图
  • 分析告警去重、按严重度分组、突发告警、重点告警及持续告警
  • 检查CCE告警健康状态并生成后续诊断的风险项
  • 查询AOM告警规则,支持可选的
    cluster_id
    过滤
  • 解析与CCE集群绑定的AOM Prometheus实例
  • 创建指标告警规则与事件告警规则
  • 根据用户提供的SMN主题创建通知动作规则
  • 从云端CCE告警模板批量配置CCE推荐的告警规则
  • 批量清理与云端CCE告警模板匹配的CCE告警规则
  • 更新、删除、启用及禁用AOM告警规则
  • 查询AOM动作规则与静音规则
典型用例:
  • "列出该CCE集群的AOM告警"
  • "分析当前与历史告警中的告警风暴"
  • "显示集群
    <cluster_id>
    的当前告警规则"
  • "使用此通知规则创建CCE推荐的告警规则"
  • "清理该集群的CCE模板告警规则"
  • "使用此SMN主题创建通知动作规则"
  • "我确认后禁用这个频繁触发的告警规则"

Prerequisites

前置条件

1. Runtime Dependencies

1. 运行时依赖

  • Python 3.8+ for the dispatcher and result processing
  • hcloud (KooCLI) 7.2.2+ in
    PATH
  • A local hcloud profile configured with
    hcloud configure
  • Run environment checks before first use when available in the skill package
  • 调度器与结果处理需Python 3.8+
  • PATH
    中需包含hcloud(KooCLI)7.2.2+
  • 已通过
    hcloud configure
    配置本地hcloud配置文件
  • 首次使用前,若技能包中包含环境检查工具,请运行该工具

2. Credential Configuration

2. 凭证配置

  • Valid Huawei Cloud credentials via explicit tool parameters, local hcloud profile, or environment variable fallback
  • Credential priority for hcloud calls is: explicit tool parameters > local hcloud profile > environment variables
  • Tools that need
    project_id
    resolve it internally where possible: explicit
    project_id
    parameter first, then active hcloud profile/IAM project lookup for the target region, then environment fallback
Security Rules:
  • Never expose AK/SK, tokens, or credential-derived secrets in code, commands, logs, or responses
  • Never run
    echo $HUAWEI_AK
    or
    echo $HUAWEI_SK
  • Never write credentials to files
  • Prefer hcloud profile for normal use
  • Use IAM users with least privilege
Optional Environment Fallback:
bash
export HUAWEI_AK=<your-ak>
export HUAWEI_SK=<your-sk>
export HUAWEI_REGION=cn-north-4
export HUAWEI_PROJECT_ID=<project-id>
export HUAWEI_SECURITY_TOKEN=<security-token>
  • 可通过明确的工具参数、本地hcloud配置文件或环境变量回退获取有效的华为云凭证
  • hcloud调用的凭证优先级:明确的工具参数 > 本地hcloud配置文件 > 环境变量
  • 需要
    project_id
    的工具会尽可能内部解析:优先使用明确的
    project_id
    参数,然后是目标区域的当前hcloud配置文件/IAM项目查询,最后是环境变量回退
安全规则:
  • 切勿在代码、命令、日志或响应中暴露AK/SK、令牌或凭证衍生的机密信息
  • 切勿运行
    echo $HUAWEI_AK
    echo $HUAWEI_SK
  • 切勿将凭证写入文件
  • 正常使用时优先使用hcloud配置文件
  • 使用权限最小化的IAM用户
可选环境变量回退:
bash
export HUAWEI_AK=<your-ak>
export HUAWEI_SK=<your-sk>
export HUAWEI_REGION=cn-north-4
export HUAWEI_PROJECT_ID=<project-id>
export HUAWEI_SECURITY_TOKEN=<security-token>

3. IAM Permission Requirements

3. IAM权限要求

PermissionPurpose
aom:event:list
Query active and historical AOM alarms
aom:alarmRule:list
Query AOM alarm rules
aom:alarmRule:create
Create AOM alarm rules
aom:alarmRule:update
Update, enable, or disable AOM alarm rules
aom:alarmRule:delete
Delete AOM alarm rules
aom:actionRule:list
Query AOM action/notification rules
aom:actionRule:create
Create AOM notification action rules
aom:actionRule:delete
Delete AOM notification action rules
aom:muteRule:list
Query AOM mute rules
cce:cluster:get
Resolve CCE cluster and AOM Prometheus binding
Permission Failure Handling:
  1. Show the failed operation and required permission.
  2. Ask the user to grant the missing IAM permission.
  3. Pause mutation work until the user confirms permissions are ready.
权限用途
aom:event:list
查询AOM当前与历史告警
aom:alarmRule:list
查询AOM告警规则
aom:alarmRule:create
创建AOM告警规则
aom:alarmRule:update
更新、启用或禁用AOM告警规则
aom:alarmRule:delete
删除AOM告警规则
aom:actionRule:list
查询AOM动作/通知规则
aom:actionRule:create
创建AOM通知动作规则
aom:actionRule:delete
删除AOM通知动作规则
aom:muteRule:list
查询AOM静音规则
cce:cluster:get
解析CCE集群与AOM Prometheus的绑定关系
权限失败处理:
  1. 显示失败的操作及所需权限。
  2. 请求用户授予缺失的IAM权限。
  3. 在用户确认权限就绪前暂停变更操作。

Core Commands

核心命令

All commands use the Python dispatcher script:
bash
python3 scripts/huawei-cloud.py <action> <key=value>...
所有命令均使用Python调度器脚本:
bash
python3 scripts/huawei-cloud.py <action> <key=value>...

KooCLI命令格式标准

KooCLI命令格式标准

This skill does not ask users to run raw
hcloud
commands directly. All cloud operations must use the dispatcher command format:
bash
python3 scripts/huawei-cloud.py <tool-name> key=value key=value
The dispatcher converts tool parameters into standard KooCLI calls:
bash
hcloud <service> <operation> --cli-region=<region> --cli-output=json [--cli-jsonInput=<file>]
Follow these rules for every command:
  • Use
    key=value
    parameters; quote values that contain spaces,
    >
    ,
    <
    ,
    |
    , JSON, or PromQL.
  • Do not print or persist AK/SK, security tokens, or generated JSON input files.
  • Use dispatcher preview output first for R2/R1/R0 tools; add
    confirm=true
    only after explicit user confirmation.
  • Prefer hcloud profile credentials; explicit tool parameters override profile and environment fallback.
  • See CLI Installation Guide for setup and validation.
本技能不要求用户直接运行原始
hcloud
命令。所有云操作必须使用调度器命令格式:
bash
python3 scripts/huawei-cloud.py <tool-name> key=value key=value
调度器会将工具参数转换为标准KooCLI调用:
bash
hcloud <service> <operation> --cli-region=<region> --cli-output=json [--cli-jsonInput=<file>]
所有命令需遵循以下规则:
  • 使用
    key=value
    参数;若值包含空格、
    >
    <
    |
    、JSON或PromQL,请添加引号。
  • 切勿打印或持久化AK/SK、安全令牌或生成的JSON输入文件。
  • 对于R2/R1/R0工具,优先使用调度器预览输出;仅在用户明确确认后添加
    confirm=true
  • 优先使用hcloud配置文件凭证;明确的工具参数会覆盖配置文件与环境变量回退。
  • 安装与验证请参考CLI安装指南

1. Alarm Query And Correlation

1. 告警查询与关联

bash
undefined
bash
undefined

Query active + historical alarms in a region

查询某区域的当前+历史告警

python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4

Query active + historical alarms for a cluster

查询某集群的当前+历史告警

python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>

Query current active alarms only

仅查询当前活跃告警

python3 scripts/huawei-cloud.py huawei_list_aom_current_alarms
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_list_aom_current_alarms
region=cn-north-4 cluster_id=<cluster-id>

Analyze deduplicated, burst, attention, and chronic alarm groups

分析去重、突发、重点及持续告警组

python3 scripts/huawei-cloud.py huawei_analyze_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>

> Do not conclude "no issue" from absence of active alarms alone. Always consider historical alarms when diagnosing a recent or recovered problem.
python3 scripts/huawei-cloud.py huawei_analyze_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>

> 仅根据无活跃告警不能得出“无问题”的结论。诊断近期或已恢复的问题时,务必考虑历史告警。

2. Alarm Rule Query

2. 告警规则查询

bash
undefined
bash
undefined

Query all alarm rules in a region

查询某区域的所有告警规则

python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4

Query alarm rules related to one CCE cluster

查询与某CCE集群相关的告警规则

python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>

Resolve the cluster AOM Prometheus instance

解析集群绑定的AOM Prometheus实例

python3 scripts/huawei-cloud.py huawei_resolve_cce_aom_prom_instance
region=cn-north-4 cluster_id=<cluster-id>

`huawei_list_aom_alarm_rules` supports cluster filtering by `cluster_id` only. Do not use `cluster_name` as a filter.
python3 scripts/huawei-cloud.py huawei_resolve_cce_aom_prom_instance
region=cn-north-4 cluster_id=<cluster-id>

`huawei_list_aom_alarm_rules`仅支持通过`cluster_id`进行集群过滤,请勿使用`cluster_name`作为过滤条件。

3. Notification Rules

3. 通知规则

bash
undefined
bash
undefined

List existing action/notification rules

列出已有的动作/通知规则

python3 scripts/huawei-cloud.py huawei_list_aom_action_rules
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules
region=cn-north-4

Preview creating a notification action rule from an SMN topic

预览从SMN主题创建通知动作规则

python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>

Confirm creation

确认创建

python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
confirm=true

Never choose a notification rule automatically. If `bind_notification_rule_id` is missing for batch alarm creation, list available action rules and wait for explicit user selection, or ask the user to provide an SMN topic and create a notification rule first.
python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
confirm=true

切勿自动选择通知规则。若批量告警创建时缺少`bind_notification_rule_id`,请列出可用的动作规则并等待用户明确选择,或请求用户提供SMN主题并先创建通知规则。

4. Alarm Rule Creation

4. 告警规则创建

bash
undefined
bash
undefined

Preview creating a CCE template metric alarm rule

预览创建CCE模板指标告警规则

python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent

Confirm creating a CCE template metric alarm rule

确认创建CCE模板指标告警规则

python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
bind_notification_rule_id=<action-rule-id>
confirm=true
python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
bind_notification_rule_id=<action-rule-id>
confirm=true

Preview creating an event alarm rule

预览创建事件告警规则

python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"

Confirm creating an event alarm rule

确认创建事件告警规则

python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
bind_notification_rule_id=<action-rule-id>
confirm=true

Metric rules with `cluster_id` + `alarm_item` and event rules both use the same CCE template payload path and default naming as batch creation. Pass `rule_name` only when the user explicitly requests a custom name. Single-rule creation requires explicit `bind_notification_rule_id` during confirmed execution, same as batch CCE alarm rule configuration. Event alarm rule `event_name` should reference `references/cce-event-list.md`; metric `alarm_item` should use the CCE template alias or rule name.
python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
bind_notification_rule_id=<action-rule-id>
confirm=true

带`cluster_id`+`alarm_item`的指标规则与事件规则均使用相同的CCE模板负载路径及批量创建的默认命名。仅当用户明确请求自定义名称时才传入`rule_name`。单规则创建在确认执行时需要明确的`bind_notification_rule_id`,与批量CCE告警规则配置要求一致。事件告警规则的`event_name`应参考`references/cce-event-list.md`;指标`alarm_item`应使用CCE模板别名或规则名称。

5. CCE Template Alarm Rules

5. CCE模板告警规则

bash
undefined
bash
undefined

Preview batch creating CCE recommended alarm rules

预览批量创建CCE推荐告警规则

python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id>
python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id>

Confirm batch creation

确认批量创建

python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id> confirm=true
python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id> confirm=true

Preview cleanup of CCE template alarm rules for a cluster

预览清理某集群的CCE模板告警规则

python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>

Confirm cleanup

确认清理

python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
delete_auto_notification_rule=true confirm=true

`huawei_configure_cce_aom_alarm_rules` requires explicit `bind_notification_rule_id`. If not provided, do not return or choose `available_notification_rules` from this tool; call `huawei_list_aom_action_rules` separately and present choices to the user.
python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
delete_auto_notification_rule=true confirm=true

`huawei_configure_cce_aom_alarm_rules`需要明确的`bind_notification_rule_id`。若未提供,请勿从此工具返回或选择`available_notification_rules`;请单独调用`huawei_list_aom_action_rules`并向用户展示选项。

6. Alarm Rule Mutation

6. 告警规则变更

bash
undefined
bash
undefined

Preview update

预览更新

python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80
python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80

Confirm update

确认更新

python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80 confirm=true
python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80 confirm=true

Preview delete

预览删除

python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule
python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule

Confirm delete

确认删除

python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule confirm=true
python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule confirm=true

Preview disable or enable

预览禁用或启用

python3 scripts/huawei-cloud.py huawei_disable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
python3 scripts/huawei-cloud.py huawei_enable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
undefined
python3 scripts/huawei-cloud.py huawei_disable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
python3 scripts/huawei-cloud.py huawei_enable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
undefined

7. Action And Mute Rule Visibility

7. 动作规则与静音规则查看

bash
undefined
bash
undefined

Query mute rules

查询静音规则

python3 scripts/huawei-cloud.py huawei_list_aom_mute_rules
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_mute_rules
region=cn-north-4

Preview deleting an action rule

预览删除动作规则

python3 scripts/huawei-cloud.py huawei_delete_aom_action_rule
region=cn-north-4 rule_name=<rule-name>
undefined
python3 scripts/huawei-cloud.py huawei_delete_aom_action_rule
region=cn-north-4 rule_name=<rule-name>
undefined

8. Cluster Alarm Inspection

8. 集群告警检查

bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
  region=cn-north-4 cluster_id=<cluster-id>
bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
  region=cn-north-4 cluster_id=<cluster-id>

Risk Levels

风险等级

This skill includes read-only query tools and mutation tools. Mutation tools must be previewed first and must wait for explicit user confirmation before using
confirm=true
.
LevelMeaningExecution Guidance
R3No-risk read-only query or local analysisMay run automatically
R2Low-risk monitoring configuration change, such as creating alarm or notification configuration without deleting resources or increasing service capacity/costPreview first; execute only after explicit user confirmation
R1Risky monitoring change, such as updating or disabling a rule in a way that may reduce observabilityPreview first; require explicit user confirmation and
confirm=true
R0Critical monitoring protection removal, such as deleting alarm rules, action rules, or broad cleanupRequire explicit confirmation, impact review, and rollback plan before
confirm=true
ToolOperation TypeRisk LevelDescription
huawei_list_aom_alarms
QueryR3Query active + historical alarms, merged and deduplicated
huawei_list_aom_current_alarms
QueryR3Query current active alarms only
huawei_analyze_aom_alarms
Query + local analysisR3Analyze alarm groups, bursts, attention alarms, and chronic alarms
huawei_aom_alarm_inspection
Query + local analysisR3Inspect cluster alarm health
huawei_list_aom_alarm_rules
QueryR3Query AOM alarm rules; supports optional
cluster_id
huawei_resolve_cce_aom_prom_instance
QueryR3Resolve the cluster AOM Prometheus instance
huawei_list_aom_action_rules
QueryR3Query action/notification rules
huawei_list_aom_mute_rules
QueryR3Query mute rules
huawei_create_aom_alarm_rule
CreateR2Create an AOM metric alarm rule
huawei_create_aom_event_alarm_rule
CreateR2Create an AOM event alarm rule
huawei_create_aom_notification_action_rule
CreateR2Create a notification action rule from a user-provided SMN topic
huawei_configure_cce_aom_alarm_rules
Batch createR2Create CCE recommended alarm rules using an explicit existing notification action rule
huawei_enable_aom_alarm_rule
EnableR2Enable an AOM alarm rule
huawei_update_aom_alarm_rule
UpdateR1Update an AOM alarm rule
huawei_disable_aom_alarm_rule
DisableR1Disable an AOM alarm rule
huawei_delete_aom_alarm_rule
DeleteR0Delete an AOM alarm rule
huawei_cleanup_cce_aom_alarm_rules
Batch deleteR0Delete CCE template alarm rules for the target cluster
huawei_delete_aom_action_rule
DeleteR0Delete a notification action rule
本技能包含只读查询工具与变更工具。变更工具必须先进行预览,且必须等待用户明确确认后才能使用
confirm=true
等级含义执行指引
R3无风险的只读查询或本地分析可自动运行
R2低风险的监控配置变更,例如创建告警或通知配置,不涉及删除资源或增加服务容量/成本先预览;仅在用户明确确认后执行
R1有风险的监控变更,例如更新或禁用规则可能降低可观测性先预览;需要用户明确确认并添加
confirm=true
R0关键监控保护移除,例如删除告警规则、动作规则或批量清理需要明确确认、影响评估及回滚计划后才能使用
confirm=true
工具操作类型风险等级描述
huawei_list_aom_alarms
查询R3查询当前+历史告警,已合并去重
huawei_list_aom_current_alarms
查询R3仅查询当前活跃告警
huawei_analyze_aom_alarms
查询+本地分析R3分析告警组、突发告警、重点告警及持续告警
huawei_aom_alarm_inspection
查询+本地分析R3检查集群告警健康状态
huawei_list_aom_alarm_rules
查询R3查询AOM告警规则;支持可选的
cluster_id
huawei_resolve_cce_aom_prom_instance
查询R3解析集群绑定的AOM Prometheus实例
huawei_list_aom_action_rules
查询R3查询动作/通知规则
huawei_list_aom_mute_rules
查询R3查询静音规则
huawei_create_aom_alarm_rule
创建R2创建AOM指标告警规则
huawei_create_aom_event_alarm_rule
创建R2创建AOM事件告警规则
huawei_create_aom_notification_action_rule
创建R2根据用户提供的SMN主题创建通知动作规则
huawei_configure_cce_aom_alarm_rules
批量创建R2使用明确的现有通知动作规则创建CCE推荐告警规则
huawei_enable_aom_alarm_rule
启用R2启用AOM告警规则
huawei_update_aom_alarm_rule
更新R1更新AOM告警规则
huawei_disable_aom_alarm_rule
禁用R1禁用AOM告警规则
huawei_delete_aom_alarm_rule
删除R0删除AOM告警规则
huawei_cleanup_cce_aom_alarm_rules
批量删除R0删除目标集群的CCE模板告警规则
huawei_delete_aom_action_rule
删除R0删除通知动作规则

Parameter Reference

参数参考

Common Parameters

通用参数

ParameterRequired/OptionalDescriptionDefault
region
RequiredHuawei Cloud region
HUAWEI_REGION
cluster_id
Tool-specificCCE cluster ID; required for cluster-scoped operationsN/A
ak
OptionalExplicit AK; highest priority for hcloud callsprofile/env fallback
sk
OptionalExplicit SK; highest priority for hcloud callsprofile/env fallback
project_id
OptionalExplicit Project IDhcloud/IAM/env fallback
confirm
Mutation onlyExecute a previewed mutation when set to
true
false
参数必填/可选描述默认值
region
必填华为云区域
HUAWEI_REGION
cluster_id
工具特定CCE集群ID;集群范围操作必填N/A
ak
可选明确的AK;hcloud调用优先级最高配置文件/环境变量回退
sk
可选明确的SK;hcloud调用优先级最高配置文件/环境变量回退
project_id
可选明确的项目IDhcloud/IAM/环境变量回退
confirm
仅变更工具设置为
true
时执行已预览的变更
false

Alarm Query Parameters

告警查询参数

ToolRequiredOptional
huawei_list_aom_alarms
region
cluster_id
, time-window/filter params
huawei_list_aom_current_alarms
region
cluster_id
, filter params
huawei_analyze_aom_alarms
region
cluster_id
, time-window/filter params
huawei_aom_alarm_inspection
region
,
cluster_id
filter params
工具必填可选
huawei_list_aom_alarms
region
cluster_id
、时间窗口/过滤参数
huawei_list_aom_current_alarms
region
cluster_id
、过滤参数
huawei_analyze_aom_alarms
region
cluster_id
、时间窗口/过滤参数
huawei_aom_alarm_inspection
region
,
cluster_id
过滤参数

Alarm Rule Parameters

告警规则参数

ToolRequiredNotes
huawei_list_aom_alarm_rules
region
Optional
cluster_id
; no
cluster_name
filtering
huawei_create_aom_alarm_rule
region
Template mode:
cluster_id
,
alarm_item
; manual mode: metric fields. Confirmed execution requires
bind_notification_rule_id
huawei_create_aom_event_alarm_rule
region
,
cluster_id
,
event_name
R2; confirmed execution requires
bind_notification_rule_id
; optional
rule_name
overrides template naming
huawei_create_aom_notification_action_rule
region
,
rule_name
,
notification_topic_urn
,
notification_topic_name
R2; user must provide the topic
huawei_configure_cce_aom_alarm_rules
region
,
cluster_id
,
bind_notification_rule_id
R2; user must explicitly choose the notification rule
huawei_cleanup_cce_aom_alarm_rules
region
,
cluster_id
R0; optional
delete_auto_notification_rule=true
huawei_update_aom_alarm_rule
region
,
rule_name
R1; update fields are tool-specific
huawei_delete_aom_alarm_rule
region
,
rule_name
R0
huawei_disable_aom_alarm_rule
region
,
rule_id
R1
huawei_enable_aom_alarm_rule
region
,
rule_id
R2
工具必填说明
huawei_list_aom_alarm_rules
region
可选
cluster_id
;不支持
cluster_name
过滤
huawei_create_aom_alarm_rule
region
模板模式:
cluster_id
,
alarm_item
;手动模式:指标字段。确认执行需要
bind_notification_rule_id
huawei_create_aom_event_alarm_rule
region
,
cluster_id
,
event_name
R2;确认执行需要
bind_notification_rule_id
;可选
rule_name
覆盖模板命名
huawei_create_aom_notification_action_rule
region
,
rule_name
,
notification_topic_urn
,
notification_topic_name
R2;用户必须提供主题
huawei_configure_cce_aom_alarm_rules
region
,
cluster_id
,
bind_notification_rule_id
R2;用户必须明确选择通知规则
huawei_cleanup_cce_aom_alarm_rules
region
,
cluster_id
R0;可选
delete_auto_notification_rule=true
huawei_update_aom_alarm_rule
region
,
rule_name
R1;更新字段为工具特定
huawei_delete_aom_alarm_rule
region
,
rule_name
R0
huawei_disable_aom_alarm_rule
region
,
rule_id
R1
huawei_enable_aom_alarm_rule
region
,
rule_id
R2

Output Format

输出格式

See Output Schema for the complete JSON response structure.
Key output fields:
  • success
    : command success status
  • error
    : failure reason when
    success=false
  • preview
    : mutation preview when
    confirm=true
    is not provided
  • requires_confirmation
    : whether user confirmation is required
  • report
    : alarm-correlation summary
  • issues
    : inspection risk items
  • alarms
    ,
    rules
    ,
    action_rules
    ,
    mute_rules
    : queried resources
完整的JSON响应结构请参考输出Schema
关键输出字段:
  • success
    : 命令执行成功状态
  • error
    :
    success=false
    时的失败原因
  • preview
    : 未提供
    confirm=true
    时的变更预览
  • requires_confirmation
    : 是否需要用户确认
  • report
    : 告警关联摘要
  • issues
    : 检查风险项
  • alarms
    ,
    rules
    ,
    action_rules
    ,
    mute_rules
    : 查询到的资源

Workflow

工作流程

See Workflow for the detailed alarm-correlation flow.
Recommended workflow:
  1. Read the alarm name, resource, time window, and severity level from the user.
  2. Default to the last 1 hour unless the user gives a different incident window.
  3. Query active + historical alarms with
    huawei_list_aom_alarms
    .
  4. Analyze alarm groups with
    huawei_analyze_aom_alarms
    .
  5. Query alarm rules, action rules, and mute rules when notification gaps or suppression are suspected.
  6. Group findings by resource, namespace, node, workload, and alarm type.
  7. Hand off to diagnosis skills for Pod, Node, Network, Storage, or Workload root cause analysis.
详细的告警关联流程请参考工作流程
推荐工作流程:
  1. 从用户处获取告警名称、资源、时间窗口及严重度级别。
  2. 若用户未指定事件窗口,默认使用最近1小时。
  3. 使用
    huawei_list_aom_alarms
    查询当前+历史告警。
  4. 使用
    huawei_analyze_aom_alarms
    分析告警组。
  5. 若怀疑存在通知缺口或抑制,查询告警规则、动作规则及静音规则。
  6. 按资源、命名空间、节点、工作负载及告警类型对结果进行分组。
  7. 将任务移交至Pod、Node、网络、存储或工作负载根因分析技能。

Verification

验证

Run read-only checks first:
bash
python3 scripts/huawei-cloud.py huawei_list_aom_alarms region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules region=cn-north-4
For cluster-scoped checks:
bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
  region=cn-north-4 cluster_id=<cluster-id>
Mutation verification:
  1. Run the command without
    confirm=true
    .
  2. Review
    preview
    , affected resources, and expected impact.
  3. Wait for explicit user confirmation.
  4. Re-run with
    confirm=true
    .
  5. Verify with the corresponding list/query command.
先运行只读检查:
bash
python3 scripts/huawei-cloud.py huawei_list_aom_alarms region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules region=cn-north-4
针对集群范围的检查:
bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
  region=cn-north-4 cluster_id=<cluster-id>
变更验证:
  1. 不带
    confirm=true
    运行命令。
  2. 查看
    preview
    、受影响资源及预期影响。
  3. 等待用户明确确认。
  4. confirm=true
    重新运行。
  5. 使用对应的列表/查询命令进行验证。

Best Practices

最佳实践

  1. Use active + historical alarms for diagnosis - active-only views can miss recently recovered issues.
  2. Filter by
    cluster_id
    - use exact cluster ID for cluster-scoped alarm rule and alarm analysis tasks.
  3. Preview every mutation - never skip the preview step for R2/R1/R0 tools.
  4. Do not auto-select notification rules - always wait for user selection of
    bind_notification_rule_id
    .
  5. Keep notification setup explicit - create notification action rules only from a user-provided SMN topic.
  6. Separate analysis from remediation - recommend remediation, then hand off to remediation skills.
  7. Protect observability - avoid disabling or deleting rules unless impact is understood and confirmed.
  1. 使用当前+历史告警进行诊断 - 仅查看当前告警可能遗漏近期已恢复的问题。
  2. cluster_id
    过滤
    - 针对集群范围的告警规则与告警分析任务,使用精确的集群ID。
  3. 所有变更均需预览 - 切勿跳过R2/R1/R0工具的预览步骤。
  4. 勿自动选择通知规则 - 始终等待用户选择
    bind_notification_rule_id
  5. 通知设置需明确 - 仅根据用户提供的SMN主题创建通知动作规则。
  6. 分析与修复分离 - 推荐修复方案后,移交至修复技能处理。
  7. 保护可观测性 - 除非已理解并确认影响,否则避免禁用或删除规则。

Notes

注意事项

  • Alarm diagnosis must consider both active and historical alarms; do not conclude the cluster is healthy only because there are no active alarms.
  • R2/R1/R0 tools must use preview-first execution and wait for explicit user confirmation before applying changes.
  • Do not automatically select notification rules, SMN topics, clusters, or templates for the user.
  • Do not modify alarm resources outside the tools provided by this skill.
  • Use
    cluster_id
    for cluster-scoped filtering; cluster names are not accepted by
    huawei_list_aom_alarm_rules
    .
  • 告警诊断必须同时考虑当前与历史告警;不能仅因无活跃告警就判定集群健康。
  • R2/R1/R0工具必须采用先预览后执行的方式,在应用变更前等待用户明确确认。
  • 切勿自动为用户选择通知规则、SMN主题、集群或模板。
  • 切勿通过本技能提供的工具以外的方式修改告警资源。
  • 针对集群范围的过滤使用
    cluster_id
    huawei_list_aom_alarm_rules
    不接受集群名称。

Troubleshooting

故障排除

SymptomLikely CauseAction
hcloud command failsMissing profile, region, or IAM permissionCheck
hcloud configure list
and IAM policy
No active alarmsIssue may have recovered or only exists in historyQuery active + historical alarms
Batch configure fails without notification rule
bind_notification_rule_id
is missing
List action rules or create one from an SMN topic, then retry
Cluster alarm rule query returns too many rulesMissing
cluster_id
filter
Re-run with exact
cluster_id
Project ID errorsProfile/project resolution failedProvide
project_id
explicitly or fix hcloud profile
症状可能原因操作
hcloud命令执行失败缺少配置文件、区域或IAM权限检查
hcloud configure list
及IAM策略
无活跃告警问题可能已恢复或仅存在于历史记录中查询当前+历史告警
无通知规则时批量配置失败缺少
bind_notification_rule_id
列出动作规则或从SMN主题创建一个,然后重试
集群告警规则查询返回过多规则缺少
cluster_id
过滤
使用精确的
cluster_id
重新运行
项目ID错误配置文件/项目解析失败明确提供
project_id
或修复hcloud配置文件

Limitations

限制

  • This skill does not modify CCE, ECS, ELB, EIP, VPC, security group, node, workload, or Kubernetes resources.
  • This skill must not create, update, or delete mute rules.
  • This skill must not use out-of-band cloud commands to modify alarm resources.
  • This skill cannot safely infer a notification rule for the user.
  • huawei_list_aom_alarm_rules
    filters by
    cluster_id
    , not cluster name.
  • 本技能不修改CCE、ECS、ELB、EIP、VPC、安全组、节点、工作负载或Kubernetes资源。
  • 本技能不得创建、更新或删除静音规则。
  • 本技能不得使用带外云命令修改告警资源。
  • 本技能无法安全地为用户推断通知规则。
  • huawei_list_aom_alarm_rules
    cluster_id
    过滤,而非集群名称。

References

参考文档

DocumentUse
Operation GuideDetailed parameters, output expectations, verification, diagnosis hand-off, and pitfalls
WorkflowAlarm correlation workflow
Output SchemaOutput JSON schema
Risk RulesRisk boundaries and confirmation rules
CLI Installation Guidehcloud/KooCLI installation and profile checks
IAM PoliciesRequired AOM, CCE, and IAM permissions
Verification MethodFunctional and mutation verification checklist
Acceptance CriteriaSkill acceptance criteria and test cases
CCE Event ListCCE event names for event alarm rules
Prometheus Metric AlarmsPrometheus metric alarm references
文档用途
操作指南详细参数、输出预期、验证、诊断移交及注意事项
工作流程告警关联工作流程
输出Schema输出JSON Schema
风险规则风险边界与确认规则
CLI安装指南hcloud/KooCLI安装与配置文件检查
IAM策略所需的AOM、CCE及IAM权限
验证方法功能与变更验证清单
验收标准技能验收标准与测试用例
CCE事件列表事件告警规则使用的CCE事件名称
Prometheus指标告警Prometheus指标告警参考