ae-dataops

CRITICAL - This skill is self-contained. Use the Global AE CLI Rules below; do not require a separate shared skill for DataOps-side tasks.

The AE Data Development and Operations domain provides capabilities for data warehouse management, flow orchestration, IDE SQL queries, data integration, operations and backfill management, including the following subcommands:

Subcommand	Responsibility	Corresponding Scenario Skill
`dataops_repo`	Space discovery	—
`dataops_datatable`	Data table and view management	`dataops-table`
`dataops_flow`	Flow creation, node deletion, and orchestration	`dataops-flow-create`
`dataops_flow`	Flow execution and monitoring	`dataops-flow-monitor`
`dataops_operations`	Operations instance search, details, and task logs	`dataops-flow-monitor`
`dataops_ide`	Data exploration and SQL queries	`dataops-query`
`dataops_integration`	Datasource and data integration	`dataops-integration`

Global AE CLI Rules

AE CLI (

ae-cli

) is the command-line tool for the AE / TE / ThinkingEngine analysis platform. For AE analysis-side requests, prefer

ae-cli

and this skill's reference docs over model memory.

Global parameters:

Parameter	Description
`--format <json\|table>`	Output format. Default is JSON.
`--jq <expr>`	jq filter expression for JSON output.
`--host <url>`	Override the active AE host. Available on every command and may be placed after the subcommand, e.g. `ae-cli dataops_ide +<command> --host <url>` .

Output and errors:

Successful commands return machine-readable JSON by default. Envelope may include optional
```
_notice.host_compat
```
.

Failed commands return

{ "ok": false, "error": { "type": "...", "message": "...", "hint": "..." } }

and exit non-zero.

CRITICAL — Host compat (do this first): After each
```
ae-cli
```
run, check stderr and
```
_notice.host_compat
```
. If either is present, open the user reply with a short ⚠️ version warning and quote the
npm i -g
/
npx skills add
(or update-cluster) lines verbatim, then present the business result. Soft tip;
```
ok: true
```
can still carry the notice.

Safety constraints:

Read commands can execute directly after required IDs and references are verified.
Ordinary
```
write
```
commands execute without
```
--yes
```
; use
```
--yes
```
only for a
```
high-risk-write
```
command after explicit user confirmation.
Never invent command names, flags, JSON payloads,
```
project_id
```
, resource IDs, field names, event names, property names, metric definitions, or date formats. Read the matching command reference and discover real project metadata first.
NEVER fabricate or guess resource names (reports, dashboards, events, properties, metrics, clusters, tags, alerts). Always use list commands to discover real resources first. If a resource is not found after fuzzy search and full list fallback, explicitly tell the user "resource not found" and stop - do not proceed with fabricated names.

Domains for DataOps:

dataops_repo

dataops_datatable

dataops_flow

dataops_operations

dataops_ide

dataops_integration

Core Concepts and Rules

You must understand the following key concepts before use, otherwise errors are highly likely.

ID System

ID	Source	Usage Scope
executeId	Returned by `dataops_flow +execute_flow`	Early stop handle before the scheduler `flowInstanceId` is available
flowInstanceId	Returned by `dataops_operations +search_flow_instances`	Operations perspective instance inspection and stop

Environment and Defaults

Scenario	Default Environment	Description
Most flow/ide/datatable commands	`DEV`	Development environment
`dataops_operations +search_flow_instances`	Operations instance search	Filter by keyword, execution date, status, and paging
`dataops_operations +get_flow_instance_detail`	Instance detail	Inspect one instance DAG and task statuses
`dataops_operations +get_task_instance_detail`	Task detail/logs	Inspect one task and include logs only when needed
`dataops_operations +stop_flow_instance`	Instance stop	Stop by exactly one of `executeId` or `flowInstanceId`

Schema Naming Rules

DEV environment:
```
ws_${spaceCode}_dev
```
PROD environment:
```
ws_${spaceCode}_product
```

Responsibility Boundaries

Operation	Correct Tool	Prohibited
Execute SELECT queries	`dataops_ide`	—
Create/modify/delete data tables (DDL)	`dataops_datatable`	`dataops_ide`

Flow Lifecycle

Create DEV Flow → Create/Update DEV SQL, Integration, Workflow Instance Check, or Task Instance Check Tasks → Configure Dependencies/Schedule → Preview Release → Release to PROD → PROD Manual Execution / Operations Troubleshooting

CRON Format (6 fields)

second minute hour day month weekday

— Note: one more "second" field than standard 5-field format.

```
0 0 2 * * ?
```
— Daily at 2 AM
```
0 0 */4 * * ?
```
— Every 4 hours
```
0 30 8 * * 1-5
```
— Weekdays at 8:30

Preset Repository vs Non-Preset Repository

Preset Repository (te_etl):
```
datasourceId
```
is
```
te_etl@TASK_ENGINE_TRINO
```
, database field is empty, requires
```
gatewayConfig
```
Non-Preset Repository:
```
datasourceId
```
is specific datasource ID, database field is required

Scenario Routing

Choose the appropriate scenario skill based on user intent to get complete step-by-step workflow guidance.

User Intent	Trigger Skill	Keywords
Create flow, add or delete nodes, configure schedule, release	`dataops-flow-create`	create flow, new workflow, configure schedule, add task node, delete task node, release, cron, scheduled execution
View execution status, troubleshoot failures, view logs	`dataops-flow-monitor`	execute flow, running instance, monitor, logs, stop, DAG, troubleshoot
Search operation instances across a space	`dataops-flow-monitor`	operations instance, flow instance search, status statistics, owner statistics
Create datasource, configure sync solution, execute sync	`dataops-integration`	datasource, sync, integration, field mapping, data ingestion, MySQL, ClickHouse, DatabricksJdbc
Browse metadata, search tables, execute SQL queries	`dataops-query`	query, SQL, data exploration, search tables, view table structure, IDE, catalog, select
Create tables and views	`dataops-table`	create table, table creation, view, data dictionary, table details, DDL

1. Space Discovery

dataops_repo

exposes only one read command. Use it to discover a valid

spaceCode

before calling DataOps commands that require one. It returns

createTime

spaceCode

, and

spaceDisplayName

If the user already provided a trusted
```
spaceCode
```
, reuse it.
If
```
spaceCode
```
is unknown, run
```
+list_spaces
```
first.
If exactly one space is returned, use its
```
spaceCode
```
.
If multiple spaces are returned and the user intent does not identify one, ask the user which space to use. Do not guess.

bash

# List spaces accessible to the current user
ae-cli dataops_repo +list_spaces

2. Data Table and View Management

Detailed workflow, command flags, examples, and parameter notes live in

references/dataops-table.md

Key constraints:

Start with
```
dataops_datatable +dict_search_tables
```
for visible DataOps catalog discovery.
Use
```
dataops_ide +search_tables
```
only for raw engine metadata, and
```
dataops_ide +ide_list_tables
```
only for known catalog/schema browsing.
Create tables/views with
```
dataops_datatable
```
, not
```
dataops_ide
```
; creation is DEV-only and must be published with
```
+publish_entity
```
.
DDL follows Trino syntax; current-space view DDL should keep the literal
```
${env}
```
placeholder.

3. Flow Orchestration

Flow orchestration is divided into two scenario skills: creation and configuration and execution and monitoring.

Lifecycle: DEV configuration and preview → Release to PROD → PROD manual execution and operations troubleshooting

Detailed creation/configuration commands live in

references/dataops-flow-create.md

. Detailed execution, monitoring, operation instance, task log, and stop commands live in

references/dataops-flow-monitor.md

Key constraints:

Create and update tasks in DEV, preview/release before PROD execution.
Treat
```
+delete_task
```
as high-risk: verify the target with
```
+get_flow_overview
```
, preview with
```
--dry-run
```
, and use
```
--yes
```
only after explicit user confirmation. Deletion affects DEV; release the flow to apply it to PROD.
```
+execute_flow
```
always runs PROD; it returns
```
executeId
```
for early stop.
Prefer
```
flowInstanceId
```
from operations search for stable inspection and troubleshooting.
Reference workspace parameters in task SQL as
```
${paramKey}
```
.

4. IDE SQL Queries

Detailed metadata browsing, SQL query, async download, and cancel workflows live in

references/dataops-query.md

Key constraints:

IDE is query-only; create/modify/delete tables with
```
dataops_datatable
```
.
Prefer
```
dataops_datatable +dict_search_tables
```
for table discovery unless raw engine metadata or schema browsing is required.
SQL query submission creates a download task; rows are not returned inline.

5. Data Integration

Detailed datasource, metadata browsing, sync solution, execution, and monitoring workflows live in

references/dataops-integration.md

Key constraints:

Generate
```
sourceConfig
```
,
```
sinkConfig
```
,
```
channelConfig
```
, and
```
fieldsMapping
```
from the reference templates; do not invent keys.
MySQL Source read partitioning uses
```
sourceConfig.splitColumn
```
;
```
fieldsMapping.shardingKey
```
is column metadata and must not be used for it.
```
+save_sync_solution
```
is not a partial patch: call
```
+get_sync_detail --withParams true
```
first, then submit complete configs.
```
syncName
```
is accepted for compatibility but ignored.
Preset repository sync uses
```
te_etl@TASK_ENGINE_TRINO
```
and requires gateway configuration.
Use
```
+list_sync_runs
```
to get
```
taskId
```
before stopping a running sync.

Reference Documentation

For detailed command flags and usage, please refer to the command documentation in the

references/

directory.

ae-dataops

NPX Install

Tags

SKILL.md Content

ae-dataops

Global AE CLI Rules

Core Concepts and Rules

ID System

Environment and Defaults

Schema Naming Rules

Responsibility Boundaries

Flow Lifecycle

CRON Format (6 fields)

Preset Repository vs Non-Preset Repository

Scenario Routing

1. Space Discovery

2. Data Table and View Management

3. Flow Orchestration

4. IDE SQL Queries

5. Data Integration

Reference Documentation