huawei-cloud-cce-alarm-correlation-engine
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseHuawei Cloud CCE Alarm Correlation Engine
华为云CCE告警关联引擎
Overview
概述
Correlate Huawei Cloud AOM active and historical alarms for CCE operations, then convert raw alarm streams into prioritized investigation leads. This skill also manages AOM alarm rules and notification action rules through a strict preview-and-confirm workflow.
Architecture: dispatcher -> local (KooCLI) -> AOM/CCE/IAM cloud service operations -> alarm query, alarm correlation, alarm-rule management, notification-rule management, and CCE alarm health inspection.
python3 scripts/huawei-cloud.pyhcloudExecution Method: All Huawei Cloud operations must go through the bundled Python dispatcher. The dispatcher invokes the localCLI. Direct SDK imports, hand-written API signing, curl IAM flows, openstack commands, or out-of-band cloud access paths are prohibited.hcloud
Related Skills:
- - CCE, AOM, and cloud-resource metric checks
huawei-cloud-cce-metric-analyzer - - Kubernetes warning event analysis
huawei-cloud-cce-kubernetes-event-analyzer - - Pod failure diagnosis
huawei-cloud-cce-pod-failure-diagnoser - - Node failure diagnosis
huawei-cloud-cce-node-failure-diagnoser - - Remediation preview and execution
huawei-cloud-cce-auto-remediation-runner
Capabilities:
- Query active AOM alarms, historical AOM alarms, and merged active+historical views
- Analyze alarm deduplication, severity grouping, burst alarms, attention alarms, and chronic alarms
- Inspect CCE alarm health and produce risk items for follow-up diagnosis
- Query AOM alarm rules with optional filtering
cluster_id - Resolve the AOM Prometheus instance bound to a CCE cluster
- Create metric alarm rules and event alarm rules
- Create notification action rules from a user-provided SMN topic
- Batch configure CCE recommended alarm rules from the cloud-side CCE alarm template
- Batch clean CCE alarm rules matching the cloud-side CCE alarm template
- Update, delete, enable, and disable AOM alarm rules
- Query AOM action rules and mute rules
Typical Use Cases:
- "List AOM alarms for this CCE cluster"
- "Analyze active and historical alarms for alarm storms"
- "Show current alarm rules for cluster "
<cluster_id> - "Create CCE recommended alarm rules with this notification rule"
- "Clean CCE template alarm rules for this cluster"
- "Create a notification action rule using this SMN topic"
- "Disable this noisy alarm rule after I confirm"
为CCE运维关联华为云AOM当前告警与历史告警,将原始告警流转换为优先级明确的排查线索。本技能还通过严格的预览确认流程管理AOM告警规则与通知动作规则。
架构: 调度器 -> 本地(KooCLI)-> AOM/CCE/IAM云服务操作 -> 告警查询、告警关联、告警规则管理、通知规则管理及CCE告警健康检查。
python3 scripts/huawei-cloud.pyhcloud执行方式:所有华为云操作必须通过捆绑的Python调度器执行。调度器调用本地CLI。禁止直接导入SDK、手动编写API签名、curl IAM流程、openstack命令或带外云访问路径。hcloud
相关技能:
- - CCE、AOM及云资源指标检查
huawei-cloud-cce-metric-analyzer - - Kubernetes警告事件分析
huawei-cloud-cce-kubernetes-event-analyzer - - Pod故障诊断
huawei-cloud-cce-pod-failure-diagnoser - - Node故障诊断
huawei-cloud-cce-node-failure-diagnoser - - 修复预览与执行
huawei-cloud-cce-auto-remediation-runner
功能:
- 查询AOM当前告警、历史告警及合并的当前+历史视图
- 分析告警去重、按严重度分组、突发告警、重点告警及持续告警
- 检查CCE告警健康状态并生成后续诊断的风险项
- 查询AOM告警规则,支持可选的过滤
cluster_id - 解析与CCE集群绑定的AOM Prometheus实例
- 创建指标告警规则与事件告警规则
- 根据用户提供的SMN主题创建通知动作规则
- 从云端CCE告警模板批量配置CCE推荐的告警规则
- 批量清理与云端CCE告警模板匹配的CCE告警规则
- 更新、删除、启用及禁用AOM告警规则
- 查询AOM动作规则与静音规则
典型用例:
- "列出该CCE集群的AOM告警"
- "分析当前与历史告警中的告警风暴"
- "显示集群的当前告警规则"
<cluster_id> - "使用此通知规则创建CCE推荐的告警规则"
- "清理该集群的CCE模板告警规则"
- "使用此SMN主题创建通知动作规则"
- "我确认后禁用这个频繁触发的告警规则"
Prerequisites
前置条件
1. Runtime Dependencies
1. 运行时依赖
- Python 3.8+ for the dispatcher and result processing
- hcloud (KooCLI) 7.2.2+ in
PATH - A local hcloud profile configured with
hcloud configure - Run environment checks before first use when available in the skill package
- 调度器与结果处理需Python 3.8+
- 中需包含hcloud(KooCLI)7.2.2+
PATH - 已通过配置本地hcloud配置文件
hcloud configure - 首次使用前,若技能包中包含环境检查工具,请运行该工具
2. Credential Configuration
2. 凭证配置
- Valid Huawei Cloud credentials via explicit tool parameters, local hcloud profile, or environment variable fallback
- Credential priority for hcloud calls is: explicit tool parameters > local hcloud profile > environment variables
- Tools that need resolve it internally where possible: explicit
project_idparameter first, then active hcloud profile/IAM project lookup for the target region, then environment fallbackproject_id
Security Rules:
- Never expose AK/SK, tokens, or credential-derived secrets in code, commands, logs, or responses
- Never run or
echo $HUAWEI_AKecho $HUAWEI_SK - Never write credentials to files
- Prefer hcloud profile for normal use
- Use IAM users with least privilege
Optional Environment Fallback:
bash
export HUAWEI_AK=<your-ak>
export HUAWEI_SK=<your-sk>
export HUAWEI_REGION=cn-north-4
export HUAWEI_PROJECT_ID=<project-id>
export HUAWEI_SECURITY_TOKEN=<security-token>- 可通过明确的工具参数、本地hcloud配置文件或环境变量回退获取有效的华为云凭证
- hcloud调用的凭证优先级:明确的工具参数 > 本地hcloud配置文件 > 环境变量
- 需要的工具会尽可能内部解析:优先使用明确的
project_id参数,然后是目标区域的当前hcloud配置文件/IAM项目查询,最后是环境变量回退project_id
安全规则:
- 切勿在代码、命令、日志或响应中暴露AK/SK、令牌或凭证衍生的机密信息
- 切勿运行或
echo $HUAWEI_AKecho $HUAWEI_SK - 切勿将凭证写入文件
- 正常使用时优先使用hcloud配置文件
- 使用权限最小化的IAM用户
可选环境变量回退:
bash
export HUAWEI_AK=<your-ak>
export HUAWEI_SK=<your-sk>
export HUAWEI_REGION=cn-north-4
export HUAWEI_PROJECT_ID=<project-id>
export HUAWEI_SECURITY_TOKEN=<security-token>3. IAM Permission Requirements
3. IAM权限要求
| Permission | Purpose |
|---|---|
| Query active and historical AOM alarms |
| Query AOM alarm rules |
| Create AOM alarm rules |
| Update, enable, or disable AOM alarm rules |
| Delete AOM alarm rules |
| Query AOM action/notification rules |
| Create AOM notification action rules |
| Delete AOM notification action rules |
| Query AOM mute rules |
| Resolve CCE cluster and AOM Prometheus binding |
Permission Failure Handling:
- Show the failed operation and required permission.
- Ask the user to grant the missing IAM permission.
- Pause mutation work until the user confirms permissions are ready.
| 权限 | 用途 |
|---|---|
| 查询AOM当前与历史告警 |
| 查询AOM告警规则 |
| 创建AOM告警规则 |
| 更新、启用或禁用AOM告警规则 |
| 删除AOM告警规则 |
| 查询AOM动作/通知规则 |
| 创建AOM通知动作规则 |
| 删除AOM通知动作规则 |
| 查询AOM静音规则 |
| 解析CCE集群与AOM Prometheus的绑定关系 |
权限失败处理:
- 显示失败的操作及所需权限。
- 请求用户授予缺失的IAM权限。
- 在用户确认权限就绪前暂停变更操作。
Core Commands
核心命令
All commands use the Python dispatcher script:
bash
python3 scripts/huawei-cloud.py <action> <key=value>...所有命令均使用Python调度器脚本:
bash
python3 scripts/huawei-cloud.py <action> <key=value>...KooCLI命令格式标准
KooCLI命令格式标准
This skill does not ask users to run raw commands directly. All cloud operations must use the dispatcher command format:
hcloudbash
python3 scripts/huawei-cloud.py <tool-name> key=value key=valueThe dispatcher converts tool parameters into standard KooCLI calls:
bash
hcloud <service> <operation> --cli-region=<region> --cli-output=json [--cli-jsonInput=<file>]Follow these rules for every command:
- Use parameters; quote values that contain spaces,
key=value,>,<, JSON, or PromQL.| - Do not print or persist AK/SK, security tokens, or generated JSON input files.
- Use dispatcher preview output first for R2/R1/R0 tools; add only after explicit user confirmation.
confirm=true - Prefer hcloud profile credentials; explicit tool parameters override profile and environment fallback.
- See CLI Installation Guide for setup and validation.
本技能不要求用户直接运行原始命令。所有云操作必须使用调度器命令格式:
hcloudbash
python3 scripts/huawei-cloud.py <tool-name> key=value key=value调度器会将工具参数转换为标准KooCLI调用:
bash
hcloud <service> <operation> --cli-region=<region> --cli-output=json [--cli-jsonInput=<file>]所有命令需遵循以下规则:
- 使用参数;若值包含空格、
key=value、>、<、JSON或PromQL,请添加引号。| - 切勿打印或持久化AK/SK、安全令牌或生成的JSON输入文件。
- 对于R2/R1/R0工具,优先使用调度器预览输出;仅在用户明确确认后添加。
confirm=true - 优先使用hcloud配置文件凭证;明确的工具参数会覆盖配置文件与环境变量回退。
- 安装与验证请参考CLI安装指南。
1. Alarm Query And Correlation
1. 告警查询与关联
bash
undefinedbash
undefinedQuery active + historical alarms in a region
查询某区域的当前+历史告警
python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4
region=cn-north-4
Query active + historical alarms for a cluster
查询某集群的当前+历史告警
python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_list_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
Query current active alarms only
仅查询当前活跃告警
python3 scripts/huawei-cloud.py huawei_list_aom_current_alarms
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_list_aom_current_alarms
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
Analyze deduplicated, burst, attention, and chronic alarm groups
分析去重、突发、重点及持续告警组
python3 scripts/huawei-cloud.py huawei_analyze_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
> Do not conclude "no issue" from absence of active alarms alone. Always consider historical alarms when diagnosing a recent or recovered problem.python3 scripts/huawei-cloud.py huawei_analyze_aom_alarms
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
> 仅根据无活跃告警不能得出“无问题”的结论。诊断近期或已恢复的问题时,务必考虑历史告警。2. Alarm Rule Query
2. 告警规则查询
bash
undefinedbash
undefinedQuery all alarm rules in a region
查询某区域的所有告警规则
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4
region=cn-north-4
Query alarm rules related to one CCE cluster
查询与某CCE集群相关的告警规则
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
Resolve the cluster AOM Prometheus instance
解析集群绑定的AOM Prometheus实例
python3 scripts/huawei-cloud.py huawei_resolve_cce_aom_prom_instance
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
`huawei_list_aom_alarm_rules` supports cluster filtering by `cluster_id` only. Do not use `cluster_name` as a filter.python3 scripts/huawei-cloud.py huawei_resolve_cce_aom_prom_instance
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
`huawei_list_aom_alarm_rules`仅支持通过`cluster_id`进行集群过滤,请勿使用`cluster_name`作为过滤条件。3. Notification Rules
3. 通知规则
bash
undefinedbash
undefinedList existing action/notification rules
列出已有的动作/通知规则
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules
region=cn-north-4
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules
region=cn-north-4
region=cn-north-4
Preview creating a notification action rule from an SMN topic
预览从SMN主题创建通知动作规则
python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
Confirm creation
确认创建
python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
confirm=true
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
confirm=true
Never choose a notification rule automatically. If `bind_notification_rule_id` is missing for batch alarm creation, list available action rules and wait for explicit user selection, or ask the user to provide an SMN topic and create a notification rule first.python3 scripts/huawei-cloud.py huawei_create_aom_notification_action_rule
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
confirm=true
region=cn-north-4 rule_name=auto-cluster-xxx
notification_topic_name=<smn-topic-name>
notification_topic_urn=<smn-topic-urn>
confirm=true
切勿自动选择通知规则。若批量告警创建时缺少`bind_notification_rule_id`,请列出可用的动作规则并等待用户明确选择,或请求用户提供SMN主题并先创建通知规则。4. Alarm Rule Creation
4. 告警规则创建
bash
undefinedbash
undefinedPreview creating a CCE template metric alarm rule
预览创建CCE模板指标告警规则
python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
Confirm creating a CCE template metric alarm rule
确认创建CCE模板指标告警规则
python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
bind_notification_rule_id=<action-rule-id>
confirm=true
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
bind_notification_rule_id=<action-rule-id>
confirm=true
python3 scripts/huawei-cloud.py huawei_create_aom_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
bind_notification_rule_id=<action-rule-id>
confirm=true
region=cn-north-4 cluster_id=<cluster-id>
alarm_item=NodeCPUUsageHigherThanEightyPercent
bind_notification_rule_id=<action-rule-id>
confirm=true
Preview creating an event alarm rule
预览创建事件告警规则
python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
Confirm creating an event alarm rule
确认创建事件告警规则
python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
bind_notification_rule_id=<action-rule-id>
confirm=true
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
bind_notification_rule_id=<action-rule-id>
confirm=true
Metric rules with `cluster_id` + `alarm_item` and event rules both use the same CCE template payload path and default naming as batch creation. Pass `rule_name` only when the user explicitly requests a custom name. Single-rule creation requires explicit `bind_notification_rule_id` during confirmed execution, same as batch CCE alarm rule configuration. Event alarm rule `event_name` should reference `references/cce-event-list.md`; metric `alarm_item` should use the CCE template alias or rule name.python3 scripts/huawei-cloud.py huawei_create_aom_event_alarm_rule
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
bind_notification_rule_id=<action-rule-id>
confirm=true
region=cn-north-4 cluster_id=<cluster-id>
event_name="<event-name>"
bind_notification_rule_id=<action-rule-id>
confirm=true
带`cluster_id`+`alarm_item`的指标规则与事件规则均使用相同的CCE模板负载路径及批量创建的默认命名。仅当用户明确请求自定义名称时才传入`rule_name`。单规则创建在确认执行时需要明确的`bind_notification_rule_id`,与批量CCE告警规则配置要求一致。事件告警规则的`event_name`应参考`references/cce-event-list.md`;指标`alarm_item`应使用CCE模板别名或规则名称。5. CCE Template Alarm Rules
5. CCE模板告警规则
bash
undefinedbash
undefinedPreview batch creating CCE recommended alarm rules
预览批量创建CCE推荐告警规则
python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id>
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id>
python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id>
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id>
Confirm batch creation
确认批量创建
python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id> confirm=true
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id> confirm=true
python3 scripts/huawei-cloud.py huawei_configure_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id> confirm=true
region=cn-north-4 cluster_id=<cluster-id>
bind_notification_rule_id=<action-rule-id> confirm=true
Preview cleanup of CCE template alarm rules for a cluster
预览清理某集群的CCE模板告警规则
python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
region=cn-north-4 cluster_id=<cluster-id>
Confirm cleanup
确认清理
python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
delete_auto_notification_rule=true confirm=true
region=cn-north-4 cluster_id=<cluster-id>
delete_auto_notification_rule=true confirm=true
`huawei_configure_cce_aom_alarm_rules` requires explicit `bind_notification_rule_id`. If not provided, do not return or choose `available_notification_rules` from this tool; call `huawei_list_aom_action_rules` separately and present choices to the user.python3 scripts/huawei-cloud.py huawei_cleanup_cce_aom_alarm_rules
region=cn-north-4 cluster_id=<cluster-id>
delete_auto_notification_rule=true confirm=true
region=cn-north-4 cluster_id=<cluster-id>
delete_auto_notification_rule=true confirm=true
`huawei_configure_cce_aom_alarm_rules`需要明确的`bind_notification_rule_id`。若未提供,请勿从此工具返回或选择`available_notification_rules`;请单独调用`huawei_list_aom_action_rules`并向用户展示选项。6. Alarm Rule Mutation
6. 告警规则变更
bash
undefinedbash
undefinedPreview update
预览更新
python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80
region=cn-north-4 rule_name=my-rule threshold=80
python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80
region=cn-north-4 rule_name=my-rule threshold=80
Confirm update
确认更新
python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80 confirm=true
region=cn-north-4 rule_name=my-rule threshold=80 confirm=true
python3 scripts/huawei-cloud.py huawei_update_aom_alarm_rule
region=cn-north-4 rule_name=my-rule threshold=80 confirm=true
region=cn-north-4 rule_name=my-rule threshold=80 confirm=true
Preview delete
预览删除
python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule
region=cn-north-4 rule_name=my-rule
python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule
region=cn-north-4 rule_name=my-rule
Confirm delete
确认删除
python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule confirm=true
region=cn-north-4 rule_name=my-rule confirm=true
python3 scripts/huawei-cloud.py huawei_delete_aom_alarm_rule
region=cn-north-4 rule_name=my-rule confirm=true
region=cn-north-4 rule_name=my-rule confirm=true
Preview disable or enable
预览禁用或启用
python3 scripts/huawei-cloud.py huawei_disable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
region=cn-north-4 rule_id=<rule-id>
python3 scripts/huawei-cloud.py huawei_enable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
region=cn-north-4 rule_id=<rule-id>
undefinedpython3 scripts/huawei-cloud.py huawei_disable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
region=cn-north-4 rule_id=<rule-id>
python3 scripts/huawei-cloud.py huawei_enable_aom_alarm_rule
region=cn-north-4 rule_id=<rule-id>
region=cn-north-4 rule_id=<rule-id>
undefined7. Action And Mute Rule Visibility
7. 动作规则与静音规则查看
bash
undefinedbash
undefinedQuery mute rules
查询静音规则
python3 scripts/huawei-cloud.py huawei_list_aom_mute_rules
region=cn-north-4
region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_mute_rules
region=cn-north-4
region=cn-north-4
Preview deleting an action rule
预览删除动作规则
python3 scripts/huawei-cloud.py huawei_delete_aom_action_rule
region=cn-north-4 rule_name=<rule-name>
region=cn-north-4 rule_name=<rule-name>
undefinedpython3 scripts/huawei-cloud.py huawei_delete_aom_action_rule
region=cn-north-4 rule_name=<rule-name>
region=cn-north-4 rule_name=<rule-name>
undefined8. Cluster Alarm Inspection
8. 集群告警检查
bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
region=cn-north-4 cluster_id=<cluster-id>bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
region=cn-north-4 cluster_id=<cluster-id>Risk Levels
风险等级
This skill includes read-only query tools and mutation tools. Mutation tools must be previewed first and must wait for explicit user confirmation before using .
confirm=true| Level | Meaning | Execution Guidance |
|---|---|---|
| R3 | No-risk read-only query or local analysis | May run automatically |
| R2 | Low-risk monitoring configuration change, such as creating alarm or notification configuration without deleting resources or increasing service capacity/cost | Preview first; execute only after explicit user confirmation |
| R1 | Risky monitoring change, such as updating or disabling a rule in a way that may reduce observability | Preview first; require explicit user confirmation and |
| R0 | Critical monitoring protection removal, such as deleting alarm rules, action rules, or broad cleanup | Require explicit confirmation, impact review, and rollback plan before |
| Tool | Operation Type | Risk Level | Description |
|---|---|---|---|
| Query | R3 | Query active + historical alarms, merged and deduplicated |
| Query | R3 | Query current active alarms only |
| Query + local analysis | R3 | Analyze alarm groups, bursts, attention alarms, and chronic alarms |
| Query + local analysis | R3 | Inspect cluster alarm health |
| Query | R3 | Query AOM alarm rules; supports optional |
| Query | R3 | Resolve the cluster AOM Prometheus instance |
| Query | R3 | Query action/notification rules |
| Query | R3 | Query mute rules |
| Create | R2 | Create an AOM metric alarm rule |
| Create | R2 | Create an AOM event alarm rule |
| Create | R2 | Create a notification action rule from a user-provided SMN topic |
| Batch create | R2 | Create CCE recommended alarm rules using an explicit existing notification action rule |
| Enable | R2 | Enable an AOM alarm rule |
| Update | R1 | Update an AOM alarm rule |
| Disable | R1 | Disable an AOM alarm rule |
| Delete | R0 | Delete an AOM alarm rule |
| Batch delete | R0 | Delete CCE template alarm rules for the target cluster |
| Delete | R0 | Delete a notification action rule |
本技能包含只读查询工具与变更工具。变更工具必须先进行预览,且必须等待用户明确确认后才能使用。
confirm=true| 等级 | 含义 | 执行指引 |
|---|---|---|
| R3 | 无风险的只读查询或本地分析 | 可自动运行 |
| R2 | 低风险的监控配置变更,例如创建告警或通知配置,不涉及删除资源或增加服务容量/成本 | 先预览;仅在用户明确确认后执行 |
| R1 | 有风险的监控变更,例如更新或禁用规则可能降低可观测性 | 先预览;需要用户明确确认并添加 |
| R0 | 关键监控保护移除,例如删除告警规则、动作规则或批量清理 | 需要明确确认、影响评估及回滚计划后才能使用 |
| 工具 | 操作类型 | 风险等级 | 描述 |
|---|---|---|---|
| 查询 | R3 | 查询当前+历史告警,已合并去重 |
| 查询 | R3 | 仅查询当前活跃告警 |
| 查询+本地分析 | R3 | 分析告警组、突发告警、重点告警及持续告警 |
| 查询+本地分析 | R3 | 检查集群告警健康状态 |
| 查询 | R3 | 查询AOM告警规则;支持可选的 |
| 查询 | R3 | 解析集群绑定的AOM Prometheus实例 |
| 查询 | R3 | 查询动作/通知规则 |
| 查询 | R3 | 查询静音规则 |
| 创建 | R2 | 创建AOM指标告警规则 |
| 创建 | R2 | 创建AOM事件告警规则 |
| 创建 | R2 | 根据用户提供的SMN主题创建通知动作规则 |
| 批量创建 | R2 | 使用明确的现有通知动作规则创建CCE推荐告警规则 |
| 启用 | R2 | 启用AOM告警规则 |
| 更新 | R1 | 更新AOM告警规则 |
| 禁用 | R1 | 禁用AOM告警规则 |
| 删除 | R0 | 删除AOM告警规则 |
| 批量删除 | R0 | 删除目标集群的CCE模板告警规则 |
| 删除 | R0 | 删除通知动作规则 |
Parameter Reference
参数参考
Common Parameters
通用参数
| Parameter | Required/Optional | Description | Default |
|---|---|---|---|
| Required | Huawei Cloud region | |
| Tool-specific | CCE cluster ID; required for cluster-scoped operations | N/A |
| Optional | Explicit AK; highest priority for hcloud calls | profile/env fallback |
| Optional | Explicit SK; highest priority for hcloud calls | profile/env fallback |
| Optional | Explicit Project ID | hcloud/IAM/env fallback |
| Mutation only | Execute a previewed mutation when set to | |
| 参数 | 必填/可选 | 描述 | 默认值 |
|---|---|---|---|
| 必填 | 华为云区域 | |
| 工具特定 | CCE集群ID;集群范围操作必填 | N/A |
| 可选 | 明确的AK;hcloud调用优先级最高 | 配置文件/环境变量回退 |
| 可选 | 明确的SK;hcloud调用优先级最高 | 配置文件/环境变量回退 |
| 可选 | 明确的项目ID | hcloud/IAM/环境变量回退 |
| 仅变更工具 | 设置为 | |
Alarm Query Parameters
告警查询参数
| Tool | Required | Optional |
|---|---|---|
| | |
| | |
| | |
| | filter params |
| 工具 | 必填 | 可选 |
|---|---|---|
| | |
| | |
| | |
| | 过滤参数 |
Alarm Rule Parameters
告警规则参数
| Tool | Required | Notes |
|---|---|---|
| | Optional |
| | Template mode: |
| | R2; confirmed execution requires |
| | R2; user must provide the topic |
| | R2; user must explicitly choose the notification rule |
| | R0; optional |
| | R1; update fields are tool-specific |
| | R0 |
| | R1 |
| | R2 |
| 工具 | 必填 | 说明 |
|---|---|---|
| | 可选 |
| | 模板模式: |
| | R2;确认执行需要 |
| | R2;用户必须提供主题 |
| | R2;用户必须明确选择通知规则 |
| | R0;可选 |
| | R1;更新字段为工具特定 |
| | R0 |
| | R1 |
| | R2 |
Output Format
输出格式
See Output Schema for the complete JSON response structure.
Key output fields:
- : command success status
success - : failure reason when
errorsuccess=false - : mutation preview when
previewis not providedconfirm=true - : whether user confirmation is required
requires_confirmation - : alarm-correlation summary
report - : inspection risk items
issues - ,
alarms,rules,action_rules: queried resourcesmute_rules
完整的JSON响应结构请参考输出Schema。
关键输出字段:
- : 命令执行成功状态
success - :
error时的失败原因success=false - : 未提供
preview时的变更预览confirm=true - : 是否需要用户确认
requires_confirmation - : 告警关联摘要
report - : 检查风险项
issues - ,
alarms,rules,action_rules: 查询到的资源mute_rules
Workflow
工作流程
See Workflow for the detailed alarm-correlation flow.
Recommended workflow:
- Read the alarm name, resource, time window, and severity level from the user.
- Default to the last 1 hour unless the user gives a different incident window.
- Query active + historical alarms with .
huawei_list_aom_alarms - Analyze alarm groups with .
huawei_analyze_aom_alarms - Query alarm rules, action rules, and mute rules when notification gaps or suppression are suspected.
- Group findings by resource, namespace, node, workload, and alarm type.
- Hand off to diagnosis skills for Pod, Node, Network, Storage, or Workload root cause analysis.
详细的告警关联流程请参考工作流程。
推荐工作流程:
- 从用户处获取告警名称、资源、时间窗口及严重度级别。
- 若用户未指定事件窗口,默认使用最近1小时。
- 使用查询当前+历史告警。
huawei_list_aom_alarms - 使用分析告警组。
huawei_analyze_aom_alarms - 若怀疑存在通知缺口或抑制,查询告警规则、动作规则及静音规则。
- 按资源、命名空间、节点、工作负载及告警类型对结果进行分组。
- 将任务移交至Pod、Node、网络、存储或工作负载根因分析技能。
Verification
验证
Run read-only checks first:
bash
python3 scripts/huawei-cloud.py huawei_list_aom_alarms region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules region=cn-north-4For cluster-scoped checks:
bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
region=cn-north-4 cluster_id=<cluster-id>Mutation verification:
- Run the command without .
confirm=true - Review , affected resources, and expected impact.
preview - Wait for explicit user confirmation.
- Re-run with .
confirm=true - Verify with the corresponding list/query command.
先运行只读检查:
bash
python3 scripts/huawei-cloud.py huawei_list_aom_alarms region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_alarm_rules region=cn-north-4
python3 scripts/huawei-cloud.py huawei_list_aom_action_rules region=cn-north-4针对集群范围的检查:
bash
python3 scripts/huawei-cloud.py huawei_aom_alarm_inspection \
region=cn-north-4 cluster_id=<cluster-id>变更验证:
- 不带运行命令。
confirm=true - 查看、受影响资源及预期影响。
preview - 等待用户明确确认。
- 带重新运行。
confirm=true - 使用对应的列表/查询命令进行验证。
Best Practices
最佳实践
- Use active + historical alarms for diagnosis - active-only views can miss recently recovered issues.
- Filter by - use exact cluster ID for cluster-scoped alarm rule and alarm analysis tasks.
cluster_id - Preview every mutation - never skip the preview step for R2/R1/R0 tools.
- Do not auto-select notification rules - always wait for user selection of .
bind_notification_rule_id - Keep notification setup explicit - create notification action rules only from a user-provided SMN topic.
- Separate analysis from remediation - recommend remediation, then hand off to remediation skills.
- Protect observability - avoid disabling or deleting rules unless impact is understood and confirmed.
- 使用当前+历史告警进行诊断 - 仅查看当前告警可能遗漏近期已恢复的问题。
- 按过滤 - 针对集群范围的告警规则与告警分析任务,使用精确的集群ID。
cluster_id - 所有变更均需预览 - 切勿跳过R2/R1/R0工具的预览步骤。
- 勿自动选择通知规则 - 始终等待用户选择。
bind_notification_rule_id - 通知设置需明确 - 仅根据用户提供的SMN主题创建通知动作规则。
- 分析与修复分离 - 推荐修复方案后,移交至修复技能处理。
- 保护可观测性 - 除非已理解并确认影响,否则避免禁用或删除规则。
Notes
注意事项
- Alarm diagnosis must consider both active and historical alarms; do not conclude the cluster is healthy only because there are no active alarms.
- R2/R1/R0 tools must use preview-first execution and wait for explicit user confirmation before applying changes.
- Do not automatically select notification rules, SMN topics, clusters, or templates for the user.
- Do not modify alarm resources outside the tools provided by this skill.
- Use for cluster-scoped filtering; cluster names are not accepted by
cluster_id.huawei_list_aom_alarm_rules
- 告警诊断必须同时考虑当前与历史告警;不能仅因无活跃告警就判定集群健康。
- R2/R1/R0工具必须采用先预览后执行的方式,在应用变更前等待用户明确确认。
- 切勿自动为用户选择通知规则、SMN主题、集群或模板。
- 切勿通过本技能提供的工具以外的方式修改告警资源。
- 针对集群范围的过滤使用;
cluster_id不接受集群名称。huawei_list_aom_alarm_rules
Troubleshooting
故障排除
| Symptom | Likely Cause | Action |
|---|---|---|
| hcloud command fails | Missing profile, region, or IAM permission | Check |
| No active alarms | Issue may have recovered or only exists in history | Query active + historical alarms |
| Batch configure fails without notification rule | | List action rules or create one from an SMN topic, then retry |
| Cluster alarm rule query returns too many rules | Missing | Re-run with exact |
| Project ID errors | Profile/project resolution failed | Provide |
| 症状 | 可能原因 | 操作 |
|---|---|---|
| hcloud命令执行失败 | 缺少配置文件、区域或IAM权限 | 检查 |
| 无活跃告警 | 问题可能已恢复或仅存在于历史记录中 | 查询当前+历史告警 |
| 无通知规则时批量配置失败 | 缺少 | 列出动作规则或从SMN主题创建一个,然后重试 |
| 集群告警规则查询返回过多规则 | 缺少 | 使用精确的 |
| 项目ID错误 | 配置文件/项目解析失败 | 明确提供 |
Limitations
限制
- This skill does not modify CCE, ECS, ELB, EIP, VPC, security group, node, workload, or Kubernetes resources.
- This skill must not create, update, or delete mute rules.
- This skill must not use out-of-band cloud commands to modify alarm resources.
- This skill cannot safely infer a notification rule for the user.
- filters by
huawei_list_aom_alarm_rules, not cluster name.cluster_id
- 本技能不修改CCE、ECS、ELB、EIP、VPC、安全组、节点、工作负载或Kubernetes资源。
- 本技能不得创建、更新或删除静音规则。
- 本技能不得使用带外云命令修改告警资源。
- 本技能无法安全地为用户推断通知规则。
- 按
huawei_list_aom_alarm_rules过滤,而非集群名称。cluster_id
References
参考文档
| Document | Use |
|---|---|
| Operation Guide | Detailed parameters, output expectations, verification, diagnosis hand-off, and pitfalls |
| Workflow | Alarm correlation workflow |
| Output Schema | Output JSON schema |
| Risk Rules | Risk boundaries and confirmation rules |
| CLI Installation Guide | hcloud/KooCLI installation and profile checks |
| IAM Policies | Required AOM, CCE, and IAM permissions |
| Verification Method | Functional and mutation verification checklist |
| Acceptance Criteria | Skill acceptance criteria and test cases |
| CCE Event List | CCE event names for event alarm rules |
| Prometheus Metric Alarms | Prometheus metric alarm references |
| 文档 | 用途 |
|---|---|
| 操作指南 | 详细参数、输出预期、验证、诊断移交及注意事项 |
| 工作流程 | 告警关联工作流程 |
| 输出Schema | 输出JSON Schema |
| 风险规则 | 风险边界与确认规则 |
| CLI安装指南 | hcloud/KooCLI安装与配置文件检查 |
| IAM策略 | 所需的AOM、CCE及IAM权限 |
| 验证方法 | 功能与变更验证清单 |
| 验收标准 | 技能验收标准与测试用例 |
| CCE事件列表 | 事件告警规则使用的CCE事件名称 |
| Prometheus指标告警 | Prometheus指标告警参考 |