ae-dataops

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

ae-dataops

CRITICAL - This skill is self-contained. Use the Global AE CLI Rules below; do not require a separate shared skill for DataOps-side tasks.

The AE Data Development and Operations domain provides capabilities for data warehouse management, flow orchestration, IDE SQL queries, data integration, operations and backfill management, including the following subcommands:

Subcommand	Responsibility	Corresponding Scenario Skill
`dataops_repo`	Space discovery	—
`dataops_datatable`	Data table and view management	`dataops-table`
`dataops_flow`	Flow creation, node deletion, and orchestration	`dataops-flow-create`
`dataops_flow`	Flow execution and monitoring	`dataops-flow-monitor`
`dataops_operations`	Operations instance search, details, and task logs	`dataops-flow-monitor`
`dataops_ide`	Data exploration and SQL queries	`dataops-query`
`dataops_integration`	Datasource and data integration	`dataops-integration`

重要提示 - 本技能为独立技能。请遵循下方的全局AE CLI规则；数据运维相关任务无需依赖单独的共享技能。

AE数据开发与运维领域提供数据仓库管理、流程编排、IDE SQL查询、数据集成、运维及回填管理等能力，包含以下子命令：

Subcommand	职责	对应场景技能
`dataops_repo`	空间发现	—
`dataops_datatable`	数据表与视图管理	`dataops-table`
`dataops_flow`	流程创建、节点删除与编排	`dataops-flow-create`
`dataops_flow`	流程执行与监控	`dataops-flow-monitor`
`dataops_operations`	运维实例搜索、详情及任务日志	`dataops-flow-monitor`
`dataops_ide`	数据探索与SQL查询	`dataops-query`
`dataops_integration`	数据源与数据集成	`dataops-integration`

Global AE CLI Rules

全局AE CLI规则

AE CLI (

ae-cli

) is the command-line tool for the AE / TE / ThinkingEngine analysis platform. For AE analysis-side requests, prefer

ae-cli

and this skill's reference docs over model memory.

Global parameters:

Parameter	Description
`--format <json\|table>`	Output format. Default is JSON.
`--jq <expr>`	jq filter expression for JSON output.
`--host <url>`	Override the active AE host. Available on every command and may be placed after the subcommand, e.g. `ae-cli dataops_ide +<command> --host <url>` .

Output and errors:

Successful commands return machine-readable JSON by default. Envelope may include optional
```
_notice.host_compat
```
.

Failed commands return

{ "ok": false, "error": { "type": "...", "message": "...", "hint": "..." } }

and exit non-zero.

CRITICAL — Host compat (do this first): After each
```
ae-cli
```
run, check stderr and
```
_notice.host_compat
```
. If either is present, open the user reply with a short ⚠️ version warning and quote the
npm i -g
/
npx skills add
(or update-cluster) lines verbatim, then present the business result. Soft tip;
```
ok: true
```
can still carry the notice.

Safety constraints:

Read commands can execute directly after required IDs and references are verified.
Ordinary
```
write
```
commands execute without
```
--yes
```
; use
```
--yes
```
only for a
```
high-risk-write
```
command after explicit user confirmation.
Never invent command names, flags, JSON payloads,
```
project_id
```
, resource IDs, field names, event names, property names, metric definitions, or date formats. Read the matching command reference and discover real project metadata first.
NEVER fabricate or guess resource names (reports, dashboards, events, properties, metrics, clusters, tags, alerts). Always use list commands to discover real resources first. If a resource is not found after fuzzy search and full list fallback, explicitly tell the user "resource not found" and stop - do not proceed with fabricated names.

Domains for DataOps:

dataops_repo

dataops_datatable

dataops_flow

dataops_operations

dataops_ide

dataops_integration

AE CLI (

ae-cli

) 是AE / TE / ThinkingEngine分析平台的命令行工具。对于AE分析侧的请求，优先使用

ae-cli

及本技能的参考文档，而非模型记忆。

全局参数：

参数	描述
`--format <json\|table>`	输出格式，默认值为JSON
`--jq <expr>`	用于JSON输出的jq过滤表达式
`--host <url>`	覆盖当前活跃的AE主机地址。所有命令均支持该参数，可放在子命令之后，例如 `ae-cli dataops_ide +<command> --host <url>`

输出与错误处理：

成功执行的命令默认返回机器可读的JSON格式。返回结果可能包含可选的
```
_notice.host_compat
```
字段。

执行失败的命令返回

{ "ok": false, "error": { "type": "...", "message": "...", "hint": "..." } }

并返回非零退出码。

重要提示 — 主机兼容性检查（优先执行）：每次运行
```
ae-cli
```
后，检查标准错误输出和
```
_notice.host_compat
```
字段。若存在其中任意一项，在回复用户时先以简短的⚠️版本警告开头，并逐字引用
npm i -g
/
npx skills add
（或update-cluster）的命令行内容，再展示业务结果。即使
```
ok: true
```
，也可能携带该提示信息。

安全约束：

读取类命令在验证所需ID及引用信息无误后可直接执行。
普通写入类命令无需
```
--yes
```
参数；仅在用户明确确认后，对高风险写入类命令使用
```
--yes
```
参数。
禁止编造命令名称、标志、JSON负载、
```
project_id
```
、资源ID、字段名、事件名、属性名、指标定义或日期格式。请先查阅匹配的命令参考文档并获取真实的项目元数据。
绝对禁止编造或猜测资源名称（报表、仪表盘、事件、属性、指标、集群、标签、告警）。必须先使用列表命令发现真实资源。若经过模糊搜索和全量列表查询后仍未找到资源，需明确告知用户“资源未找到”并停止操作，不得使用编造的名称继续执行。

数据运维领域的子命令：

dataops_repo

dataops_datatable

dataops_flow

dataops_operations

dataops_ide

dataops_integration

Core Concepts and Rules

核心概念与规则

You must understand the following key concepts before use, otherwise errors are highly likely.

使用本技能前必须理解以下关键概念，否则极易出错。

ID System

ID系统

ID	Source	Usage Scope
executeId	Returned by `dataops_flow +execute_flow`	Early stop handle before the scheduler `flowInstanceId` is available
flowInstanceId	Returned by `dataops_operations +search_flow_instances`	Operations perspective instance inspection and stop

ID	来源	使用范围
executeId	由 `dataops_flow +execute_flow` 返回	在调度器 `flowInstanceId` 生成前用于提前终止流程
flowInstanceId	由 `dataops_operations +search_flow_instances` 返回	从运维视角检查实例状态及终止实例

Environment and Defaults

环境与默认值

Scenario	Default Environment	Description
Most flow/ide/datatable commands	`DEV`	Development environment
`dataops_operations +search_flow_instances`	Operations instance search	Filter by keyword, execution date, status, and paging
`dataops_operations +get_flow_instance_detail`	Instance detail	Inspect one instance DAG and task statuses
`dataops_operations +get_task_instance_detail`	Task detail/logs	Inspect one task and include logs only when needed
`dataops_operations +stop_flow_instance`	Instance stop	Stop by exactly one of `executeId` or `flowInstanceId`

场景	默认环境	描述
大多数流程/IDE/数据表命令	`DEV`	开发环境
`dataops_operations +search_flow_instances`	运维实例搜索	按关键词、执行日期、状态及分页过滤
`dataops_operations +get_flow_instance_detail`	实例详情	查看单个实例的DAG及任务状态
`dataops_operations +get_task_instance_detail`	任务详情/日志	查看单个任务，仅在需要时返回日志
`dataops_operations +stop_flow_instance`	实例终止	必须通过 `executeId` 或 `flowInstanceId` 其中之一精确终止

Schema Naming Rules

Schema命名规则

DEV environment:
```
ws_${spaceCode}_dev
```
PROD environment:
```
ws_${spaceCode}_product
```

DEV环境：
```
ws_${spaceCode}_dev
```
PROD环境：
```
ws_${spaceCode}_product
```

Responsibility Boundaries

职责边界

Operation	Correct Tool	Prohibited
Execute SELECT queries	`dataops_ide`	—
Create/modify/delete data tables (DDL)	`dataops_datatable`	`dataops_ide`

操作	正确工具	禁止使用
执行SELECT查询	`dataops_ide`	—
创建/修改/删除数据表（DDL）	`dataops_datatable`	`dataops_ide`

Flow Lifecycle

流程生命周期

Create DEV Flow → Create/Update DEV SQL, Integration, Workflow Instance Check, or Task Instance Check Tasks → Configure Dependencies/Schedule → Preview Release → Release to PROD → PROD Manual Execution / Operations Troubleshooting

创建DEV流程 → 创建/更新DEV SQL、集成、工作流实例检查或任务实例检查任务 → 配置依赖/调度计划 → 预览发布 → 发布至PROD → PROD手动执行 / 运维故障排查

CRON Format (6 fields)

CRON格式（6字段）

second minute hour day month weekday

— Note: one more "second" field than standard 5-field format.

```
0 0 2 * * ?
```
— Daily at 2 AM
```
0 0 */4 * * ?
```
— Every 4 hours
```
0 30 8 * * 1-5
```
— Weekdays at 8:30

second minute hour day month weekday

— 注意：比标准的5字段格式多一个“秒”字段。

```
0 0 2 * * ?
```
— 每日凌晨2点执行
```
0 0 */4 * * ?
```
— 每4小时执行一次
```
0 30 8 * * 1-5
```
— 工作日早8:30执行

Preset Repository vs Non-Preset Repository

预设仓库 vs 非预设仓库

Preset Repository (te_etl):
```
datasourceId
```
is
```
te_etl@TASK_ENGINE_TRINO
```
, database field is empty, requires
```
gatewayConfig
```
Non-Preset Repository:
```
datasourceId
```
is specific datasource ID, database field is required

预设仓库（te_etl）：
```
datasourceId
```
为
```
te_etl@TASK_ENGINE_TRINO
```
，数据库字段为空，需配置
```
gatewayConfig
```
非预设仓库：
```
datasourceId
```
为具体的数据源ID，必须填写数据库字段

Scenario Routing

场景路由

Choose the appropriate scenario skill based on user intent to get complete step-by-step workflow guidance.

User Intent	Trigger Skill	Keywords
Create flow, add or delete nodes, configure schedule, release	`dataops-flow-create`	create flow, new workflow, configure schedule, add task node, delete task node, release, cron, scheduled execution
View execution status, troubleshoot failures, view logs	`dataops-flow-monitor`	execute flow, running instance, monitor, logs, stop, DAG, troubleshoot
Search operation instances across a space	`dataops-flow-monitor`	operations instance, flow instance search, status statistics, owner statistics
Create datasource, configure sync solution, execute sync	`dataops-integration`	datasource, sync, integration, field mapping, data ingestion, MySQL, ClickHouse, DatabricksJdbc
Browse metadata, search tables, execute SQL queries	`dataops-query`	query, SQL, data exploration, search tables, view table structure, IDE, catalog, select
Create tables and views	`dataops-table`	create table, table creation, view, data dictionary, table details, DDL

根据用户意图选择合适的场景技能，获取完整的分步工作流指导。

用户意图	触发技能	关键词
创建流程、添加/删除节点、配置调度计划、发布	`dataops-flow-create`	创建流程、新工作流、配置调度计划、添加任务节点、删除任务节点、发布、cron、定时执行
查看执行状态、排查故障、查看日志	`dataops-flow-monitor`	执行流程、运行中实例、监控、日志、终止、DAG、故障排查
跨空间搜索运维实例	`dataops-flow-monitor`	运维实例、流程实例搜索、状态统计、所有者统计
创建数据源、配置同步方案、执行同步	`dataops-integration`	数据源、同步、集成、字段映射、数据导入、MySQL、ClickHouse、DatabricksJdbc
浏览元数据、搜索表、执行SQL查询	`dataops-query`	查询、SQL、数据探索、搜索表、查看表结构、IDE、目录、select
创建表与视图	`dataops-table`	创建表、建表、视图、数据字典、表详情、DDL

1. Space Discovery

1. 空间发现

dataops_repo

exposes only one read command. Use it to discover a valid

spaceCode

before calling DataOps commands that require one. It returns

createTime

spaceCode

, and

spaceDisplayName

If the user already provided a trusted
```
spaceCode
```
, reuse it.
If
```
spaceCode
```
is unknown, run
```
+list_spaces
```
first.
If exactly one space is returned, use its
```
spaceCode
```
.
If multiple spaces are returned and the user intent does not identify one, ask the user which space to use. Do not guess.

bash

undefined

dataops_repo

仅提供一个读取类命令。在调用需要

spaceCode

的数据运维命令前，使用该命令获取有效的

spaceCode

。该命令返回

createTime

、

spaceCode

和

spaceDisplayName

。

若用户已提供可信的
```
spaceCode
```
，直接复用。
若
```
spaceCode
```
未知，先执行
```
+list_spaces
```
命令。
若仅返回一个空间，使用其
```
spaceCode
```
。
若返回多个空间且用户意图未明确指定，询问用户使用哪个空间，禁止猜测。

bash

undefined

List spaces accessible to the current user

列出当前用户可访问的空间

ae-cli dataops_repo +list_spaces

---

ae-cli dataops_repo +list_spaces

---

2. Data Table and View Management

2. 数据表与视图管理

Detailed workflow, command flags, examples, and parameter notes live in

references/dataops-table.md

Key constraints:

Start with
```
dataops_datatable +dict_search_tables
```
for visible DataOps catalog discovery.
Use
```
dataops_ide +search_tables
```
only for raw engine metadata, and
```
dataops_ide +ide_list_tables
```
only for known catalog/schema browsing.
Create tables/views with
```
dataops_datatable
```
, not
```
dataops_ide
```
; creation is DEV-only and must be published with
```
+publish_entity
```
.
DDL follows Trino syntax; current-space view DDL should keep the literal
```
${env}
```
placeholder.

详细的工作流、命令标志、示例及参数说明请查阅

references/dataops-table.md

。

关键约束：

从
```
dataops_datatable +dict_search_tables
```
开始，发现可见的数据运维目录。
仅在需要获取原始引擎元数据时使用
```
dataops_ide +search_tables
```
，仅在浏览已知目录/schema时使用
```
dataops_ide +ide_list_tables
```
。
使用
```
dataops_datatable
```
创建表/视图，禁止使用
```
dataops_ide
```
；创建操作仅支持DEV环境，且必须通过
```
+publish_entity
```
发布。
DDL遵循Trino语法；当前空间的视图DDL需保留字面量
```
${env}
```
占位符。

3. Flow Orchestration

3. 流程编排

Flow orchestration is divided into two scenario skills: creation and configuration and execution and monitoring.

Lifecycle: DEV configuration and preview → Release to PROD → PROD manual execution and operations troubleshooting

Detailed creation/configuration commands live in

references/dataops-flow-create.md

. Detailed execution, monitoring, operation instance, task log, and stop commands live in

references/dataops-flow-monitor.md

Key constraints:

Create and update tasks in DEV, preview/release before PROD execution.
Treat
```
+delete_task
```
as high-risk: verify the target with
```
+get_flow_overview
```
, preview with
```
--dry-run
```
, and use
```
--yes
```
only after explicit user confirmation. Deletion affects DEV; release the flow to apply it to PROD.
```
+execute_flow
```
always runs PROD; it returns
```
executeId
```
for early stop.
Prefer
```
flowInstanceId
```
from operations search for stable inspection and troubleshooting.
Reference workspace parameters in task SQL as
```
${paramKey}
```
.

流程编排分为两个场景技能：创建与配置和执行与监控。

生命周期：DEV环境配置与预览 → 发布至PROD → PROD环境手动执行与运维故障排查

详细的创建/配置命令请查阅

references/dataops-flow-create.md

。详细的执行、监控、运维实例、任务日志及终止命令请查阅

references/dataops-flow-monitor.md

。

关键约束：

在DEV环境创建和更新任务，预览/发布后再在PROD环境执行。
将
```
+delete_task
```
视为高风险操作：先通过
```
+get_flow_overview
```
验证目标，使用
```
--dry-run
```
预览，仅在用户明确确认后使用
```
--yes
```
参数。删除操作仅影响DEV环境；需发布流程才能将变更应用至PROD环境。
```
+execute_flow
```
始终在PROD环境执行；返回
```
executeId
```
用于提前终止流程。
优先使用运维搜索返回的
```
flowInstanceId
```
进行稳定的检查与故障排查。
在任务SQL中引用工作区参数时使用
```
${paramKey}
```
格式。

4. IDE SQL Queries

4. IDE SQL查询

Detailed metadata browsing, SQL query, async download, and cancel workflows live in

references/dataops-query.md

Key constraints:

IDE is query-only; create/modify/delete tables with
```
dataops_datatable
```
.
Prefer
```
dataops_datatable +dict_search_tables
```
for table discovery unless raw engine metadata or schema browsing is required.
SQL query submission creates a download task; rows are not returned inline.

详细的元数据浏览、SQL查询、异步下载及取消工作流请查阅

references/dataops-query.md

。

关键约束：

IDE仅用于查询；创建/修改/删除表请使用
```
dataops_datatable
```
。
优先使用
```
dataops_datatable +dict_search_tables
```
发现表，除非需要获取原始引擎元数据或浏览schema。
提交SQL查询会创建一个下载任务；查询结果不会直接返回。

5. Data Integration

5. 数据集成

Detailed datasource, metadata browsing, sync solution, execution, and monitoring workflows live in

references/dataops-integration.md

Key constraints:

Generate
```
sourceConfig
```
,
```
sinkConfig
```
,
```
channelConfig
```
, and
```
fieldsMapping
```
from the reference templates; do not invent keys.
MySQL Source read partitioning uses
```
sourceConfig.splitColumn
```
;
```
fieldsMapping.shardingKey
```
is column metadata and must not be used for it.
```
+save_sync_solution
```
is not a partial patch: call
```
+get_sync_detail --withParams true
```
first, then submit complete configs.
```
syncName
```
is accepted for compatibility but ignored.
Preset repository sync uses
```
te_etl@TASK_ENGINE_TRINO
```
and requires gateway configuration.
Use
```
+list_sync_runs
```
to get
```
taskId
```
before stopping a running sync.

详细的数据源、元数据浏览、同步方案、执行及监控工作流请查阅

references/dataops-integration.md

。

关键约束：

从参考模板生成
```
sourceConfig
```
、
```
sinkConfig
```
、
```
channelConfig
```
和
```
fieldsMapping
```
，禁止自行编造键名。
MySQL源读取分区使用
```
sourceConfig.splitColumn
```
；
```
fieldsMapping.shardingKey
```
为列元数据，不得用于分区。
```
+save_sync_solution
```
不支持部分更新：需先调用
```
+get_sync_detail --withParams true
```
获取完整配置，再提交完整的配置内容。
```
syncName
```
仅为兼容保留，实际会被忽略。
预设仓库同步使用
```
te_etl@TASK_ENGINE_TRINO
```
，并需配置网关。
终止运行中的同步任务前，需使用
```
+list_sync_runs
```
获取
```
taskId
```
。

Reference Documentation

参考文档

For detailed command flags and usage, please refer to the command documentation in the

references/

directory.

如需详细的命令标志及使用说明，请查阅

references/

目录下的命令文档。