troubleshooting-flows

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Troubleshooting Flows

Flow故障排查

A flow is broken when it fails to move data correctly. This skill covers systematic diagnosis: identifying the problem type, isolating the failing step, inspecting errors, and resolving them.

Troubleshooting concerns:

Job status -- understanding what
```
completed
```
,
```
failed
```
,
```
canceled
```
, and
```
retrying
```
mean for the flow
Error analysis -- grouping errors by pattern to find root causes instead of reading them one-by-one
Request/response inspection -- seeing exactly what was sent and returned at each step
Execution logs -- record-level tracing through every stage of the pipeline
Retry and resolution -- fixing error data and retrying vs bulk resolving
Delta/state issues --
```
lastExportDateTime
```
drift, stuck deltas, re-processing windows

当flow无法正确传输数据时，即表示其已出现故障。本指南涵盖系统性诊断方法：识别问题类型、定位故障步骤、检查错误并解决问题。

故障排查涉及以下方面：

任务状态——理解flow的
```
completed
```
、
```
failed
```
、
```
canceled
```
和
```
retrying
```
状态含义
错误分析——按模式分组错误以找到根本原因，而非逐个查看错误
请求/响应检查——查看每个步骤中发送和返回的具体内容
执行日志——记录管道每个阶段的逐层级追踪信息
重试与修复——修复错误数据并重试，或批量解决错误
增量/状态问题——
```
lastExportDateTime
```
偏移、停滞的增量、重复处理窗口

Problem Categories

问题分类

Total Failure

完全失败

Job status is

failed

with 0 successful records. The entire run collapsed before processing any data. Typically

numPagesGenerated: 0

(export-level failure) or pages generated but

numPagesProcessed: 0

(import-level failure on the first page).

Common causes: connection failure (credentials expired, endpoint down), export query error (invalid SQL, bad saved search ID), missing/deleted resource, permission denied.

任务状态为

failed

，成功记录数为0。整个运行在处理任何数据前就崩溃了。通常表现为

numPagesGenerated: 0

（导出级失败），或已生成页面但

numPagesProcessed: 0

（第一页导入级失败）。

常见原因：连接失败（凭证过期、端点宕机）、导出查询错误（无效SQL、错误的已保存搜索ID）、资源缺失/删除、权限被拒绝。

Partial Failure

部分失败

Job status is

completed

but

numError > 0

alongside successful records. Some records failed while others processed normally. Real-world data shows wide variance -- from 2 errors in 8000 successes to 500+ errors in 8000 successes.

Common causes: validation errors on the destination (required fields missing, type mismatches), duplicate key violations, record-level lookup failures, rate limiting on specific batches, data-dependent issues (specific records have bad data).

任务状态为

completed

，但

numError > 0

，同时存在成功记录。部分记录处理失败，其余记录正常处理。实际数据情况差异很大——从8000条成功记录中出现2个错误，到8000条成功记录中出现500+个错误。

常见原因：目标端验证错误（必填字段缺失、类型不匹配）、重复键冲突、记录级查找失败、特定批次速率限制、数据相关问题（特定记录存在错误数据）。

Empty Run

空运行

Job completes successfully with 0 errors AND 0 records processed.

Common causes: wrong

resourcePath

on the export (extracts from wrong JSON path), delta export with no changes since last run (legitimate), output filter too restrictive (all records filtered out), source query returns no results, webhook export with no inbound events.

任务成功完成，但错误数和处理记录数均为0。

常见原因：导出的

resourcePath

错误（从错误的JSON路径提取数据）、增量导出自上次运行后无变更（正常情况）、输出过滤过于严格（所有记录被过滤）、源查询无结果、Webhook导出无入站事件。

Stuck or Long-Running

停滞或长时间运行

Job stays in

running

retrying

status longer than expected.

Common causes: large dataset with no pagination limits, destination system slow to respond, script hook with long-running logic, on-premise agent connectivity issues, rate limiting causing backoff.

任务长时间处于

running

或

retrying

状态，超出预期时长。

常见原因：无分页限制的大型数据集、目标系统响应缓慢、包含长时间运行逻辑的脚本钩子、本地Agent连接问题、速率限制导致退避。

Intermittent Failures

间歇性失败

Flow sometimes succeeds and sometimes fails with the same configuration.

Common causes: token/session expiry mid-run (long-running flows), rate limiting (varies with concurrent flows), transient network errors, source system maintenance windows.

相同配置下，flow有时成功有时失败。

常见原因：运行过程中令牌/会话过期（长时间运行的flow）、速率限制（随并发flow变化）、临时网络错误、源系统维护窗口。

Error Diagnosis Framework

错误诊断框架

Classification

分类

When an error occurs, classify it into one of three categories to determine the right action:

Category	HTTP status codes	Meaning	Action
Needs investigation	400, 401, 403, 404, 405, 409, 422	Missing info, wrong IDs, permission denied, validation errors	Stop and investigate -- check resource config, connection status, permissions
Transient	408, 429, 500, 502, 503, 504	Timeouts, rate limits, server errors	Retry once. If it fails again, escalate -- the external system may be down
Configuration error	varies	Preconditions not met but fixable	Follow the error message guidance to fix the config, then retry

A 5xx error not in the transient list (e.g., 501) is still likely transient. A 4xx error not in the investigation list warrants manual review.

发生错误时，将其分为以下三类，以确定正确的处理措施：

类别	HTTP状态码	含义	操作
需要调查	400, 401, 403, 404, 405, 409, 422	信息缺失、ID错误、权限被拒绝、验证错误	停止并调查——检查资源配置、连接状态、权限
临时错误	408, 429, 500, 502, 503, 504	超时、速率限制、服务器错误	重试一次。若再次失败，升级处理——外部系统可能宕机
配置错误	不固定	未满足前置条件但可修复	根据错误消息指引修复配置，然后重试

不在临时错误列表中的5xx错误（如501）仍可能是临时错误。不在调查列表中的4xx错误需要手动审核。

Root Cause: Configuration vs Data

根本原因：配置 vs 数据

Every flow error has one of two root causes:

Static configuration -- a hardcoded value in the step config is wrong (mapping expression, filter rule, hardcoded field, URI, query, SQL statement). Fix: change the resource configuration via
```
celigo <type> set
```
Dynamic data -- the upstream source sent unexpected data (missing required field, wrong type, null where a value is expected, unexpected array/object shape). Fix: add input filtering or validation upstream, or fix the source system

To distinguish: check if the error reproduces with different input records. If the same error occurs for every record, it's configuration. If only some records fail, it's data.

bash

undefined

每个flow错误都有以下两种根本原因之一：

静态配置——步骤配置中的硬编码值错误（映射表达式、过滤规则、硬编码字段、URI、查询、SQL语句）。修复方式：通过
```
celigo <type> set
```
修改资源配置
动态数据——上游源发送了意外数据（缺失必填字段、类型错误、预期有值但为null、意外的数组/对象结构）。修复方式：在上游添加输入过滤或验证，或修复源系统

**区分方法：**检查错误是否在不同输入记录中重现。如果每条记录都出现相同错误，则是配置问题。如果只有部分记录失败，则是数据问题。

bash

undefined

Check if all records fail (configuration) or only some (data)

检查是所有记录都失败（配置问题）还是仅部分记录失败（数据问题）

celigo flows error-summary <flowId> # Compare error count vs total records celigo flows errors <flowId> <stepId> # Sample specific errors to compare

undefined

celigo flows error-summary <flowId> # 比较错误数与总记录数 celigo flows errors <flowId> <stepId> # 采样特定错误进行比较

undefined

Which Step Failed?

哪个步骤失败了？

Error location determines which resource and skill to investigate:

Error location	Resource to check	Skill
Export / page generator	Export config (connection, query, resourcePath)	configuring-exports
Import / page processor	Import config (mapping, destination fields, operation)	configuring-imports
Script hook	Script code (preSavePage, preMap, postMap, postSubmit)	writing-scripts
Mapping	Mapping expression (field paths, lookups, hardcoded values)	writing-mappings
Filter	Filter expression (s-expression syntax, field references)	configuring-filters
Connection	Connection config (auth, URL, credentials)	configuring-connections

错误位置决定了需要检查的资源和技能：

错误位置	需要检查的资源	技能
导出 / 页面生成器	导出配置（连接、查询、resourcePath）	configuring-exports
导入 / 页面处理器	导入配置（映射、目标字段、操作）	configuring-imports
脚本钩子	脚本代码（preSavePage、preMap、postMap、postSubmit）	writing-scripts
映射	映射表达式（字段路径、查找、硬编码值）	writing-mappings
过滤	过滤表达式（s-expression语法、字段引用）	configuring-filters
连接	连接配置（认证、URL、凭证）	configuring-connections

Quick Reference

快速参考

Symptom --> First Command

症状 --> 优先执行的命令

Symptom	Run first	Then
Flow totally failed	`celigo jobs list --flow <flowId> --limit 1`	Check `numPagesGenerated` -- if 0, export failed; check connection and query
Partial errors	`celigo flows error-summary <flowId>`	`celigo flows error-analysis <flowId> <stepId>` to find root cause pattern
Empty run (0 records)	`celigo jobs list --flow <flowId> --limit 1`	Check export config ( `resourcePath` , delta state, output filter)
Stuck / long-running	`celigo jobs current --flow <flowId>`	Check job status; if `retrying` , inspect rate limiting or connection issues
Intermittent failures	`celigo jobs run-stats --flow <flowId>`	Compare failing vs passing runs; check token expiry and rate limits
Silent logic bug (no errors, wrong output)	`celigo flows test-run <flowId> --export <genId>`	If test-run can't reach it, enable execution logging (§6) and run for real
Production incident (real traffic matters)	`celigo flows enable-execution-logs <flowId>` then run	Read per-record I/O with `query-execution-logs` / `execution-log-detail` ; the failing stage names the problem

症状	优先执行	后续操作
Flow完全失败	`celigo jobs list --flow <flowId> --limit 1`	检查 `numPagesGenerated` ——如果为0，说明导出失败；检查连接和查询
部分错误	`celigo flows error-summary <flowId>`	执行 `celigo flows error-analysis <flowId> <stepId>` 查找根本原因模式
空运行（0条记录）	`celigo jobs list --flow <flowId> --limit 1`	检查导出配置（ `resourcePath` 、增量状态、输出过滤）
停滞 / 长时间运行	`celigo jobs current --flow <flowId>`	检查任务状态；如果是 `retrying` ，检查速率限制或连接问题
间歇性失败	`celigo jobs run-stats --flow <flowId>`	比较失败与成功运行；检查令牌过期和速率限制
静默逻辑错误（无错误，但输出错误）	`celigo flows test-run <flowId> --export <genId>`	如果测试运行无法复现，启用执行日志（第6节）并实际运行
生产事件（真实流量至关重要）	先执行 `celigo flows enable-execution-logs <flowId>` 再运行	使用 `query-execution-logs` / `execution-log-detail` 查看每条记录的I/O；失败阶段会指明问题

Key Diagnostic Commands

关键诊断命令

bash

undefined

bash

undefined

Job status

任务状态

celigo jobs list --flow <flowId> --limit 1 # Most recent job celigo jobs current --flow <flowId> # Currently running job celigo jobs diagnostics <jobId> # Full diagnostic bundle

celigo jobs list --flow <flowId> --limit 1 # 最新任务 celigo jobs current --flow <flowId> # 当前运行的任务 celigo jobs diagnostics <jobId> # 完整诊断包

Error investigation

错误调查

celigo flows error-summary <flowId> # Per-step error counts celigo flows error-analysis <flowId> <id> # Group errors by pattern celigo flows errors <flowId> <id> # List individual errors (each has an errorId) celigo flows error <flowId> <id> <errorId> --request-detail # Raw HTTP request/response for one error

celigo flows error-summary <flowId> # 按步骤统计错误数 celigo flows error-analysis <flowId> <id> # 按模式分组错误 celigo flows errors <flowId> <id> # 列出单个错误（每个错误都有errorId） celigo flows error <flowId> <id> <errorId> --request-detail # 单个错误的原始HTTP请求/响应

Safe iteration first

优先进行安全迭代

celigo flows test-run <flowId> --export <genId> # safe, fast, try this first

celigo flows test-run <flowId> --export <genId> # 安全、快速，优先尝试此命令

End-to-end execution logging (real run, full per-record I/O)

端到端执行日志（实际运行，完整的每条记录I/O）

celigo flows enable-execution-logs <flowId> # arm debug logging, then run the flow celigo flows execution-logs <flowId> <jobId> # list captured per-record logs celigo flows debug-requests <flowId> <id> # per-bubble HTTP request/response

undefined

celigo flows enable-execution-logs <flowId> # 启用调试日志，然后运行flow celigo flows execution-logs <flowId> <jobId> # 列出捕获的每条记录日志 celigo flows debug-requests <flowId> <id> # 每个环节的HTTP请求/响应

undefined

Related Skills

Diagnostic Workflow

诊断工作流

1. Check the job status

1. 检查任务状态

Start with the most recent job to understand what happened.

bash

celigo jobs list --flow <flowId> --limit 1
celigo jobs get <jobId>
celigo jobs current --flow <flowId>

Key fields:

status

numError

numSuccess

numIgnore

numPagesGenerated

numPagesProcessed

startedAt

endedAt

. A

failed

status with

numPagesGenerated: 0

means the export itself failed -- don't look at import errors.

completed

with

numError > 0

means partial failure at the record level.

从最新任务开始，了解发生了什么。

bash

celigo jobs list --flow <flowId> --limit 1
celigo jobs get <jobId>
celigo jobs current --flow <flowId>

关键字段：

status

、

numError

、

numSuccess

、

numIgnore

、

numPagesGenerated

、

numPagesProcessed

、

startedAt

、

endedAt

。状态为

failed

且

numPagesGenerated: 0

意味着导出本身失败——无需查看导入错误。状态为

completed

且

numError > 0

意味着记录级部分失败。

2. Get the error summary

2. 获取错误摘要

See which steps have errors and how many.

bash

celigo flows error-summary <flowId>

This returns per-step error counts. Focus on the step with the most errors first.

查看哪些步骤存在错误以及错误数量。

bash

celigo flows error-summary <flowId>

此命令返回按步骤统计的错误数。优先关注错误数最多的步骤。

3. Analyze error patterns

3. 分析错误模式

Group errors by message pattern to find the root cause instead of reading them one-by-one.

bash

celigo flows error-analysis <flowId> <exportOrImportId> [--limit 200]

If most errors share the same message, that's your root cause. Multiple distinct patterns may indicate multiple issues.

按消息模式分组错误以找到根本原因，而非逐个查看错误。

bash

celigo flows error-analysis <flowId> <exportOrImportId> [--limit 200]

如果大多数错误共享相同消息，那就是根本原因。多个不同模式可能意味着存在多个问题。

4. Inspect individual errors

4. 检查单个错误

Once you know the pattern, look at specific records and the HTTP request/response that produced the error.

bash

celigo flows errors <flowId> <exportOrImportId>                       # list open errors (note the errorId of each)
celigo flows error <flowId> <exportOrImportId> <errorId> --retry-data       # inspect one error + its editable retry data
celigo flows error <flowId> <exportOrImportId> <errorId> --request-detail   # + the captured HTTP request/response

flows error --request-detail

is the most powerful diagnostic -- it resolves the error's

reqAndResKey

for you and shows exactly what HTTP request was sent and what the destination responded with. (If you already hold a

reqAndResKey

from

debug-requests

, use

debug-request-detail

instead.)

确定模式后，查看特定记录以及产生错误的HTTP请求/响应。

bash

celigo flows errors <flowId> <exportOrImportId>                       # 列出未解决的错误（记录每个错误的errorId）
celigo flows error <flowId> <exportOrImportId> <errorId> --retry-data       # 检查单个错误及其可编辑的重试数据
celigo flows error <flowId> <exportOrImportId> <errorId> --request-detail   # + 捕获的HTTP请求/响应

flows error --request-detail

是最强大的诊断工具——它会自动解析错误的

reqAndResKey

，并显示发送的具体HTTP请求和目标端的响应内容。（如果已从

debug-requests

获取

reqAndResKey

，请改用

debug-request-detail

。）

5. Use test runs for safe iteration (try this first)

5. 使用测试运行进行安全迭代（优先尝试此方法）

Test runs process a single page without affecting production data or delta state. Fast, safe, no arming, no side effects -- answers most logic questions.

bash

celigo flows test-run <flowId> --export <exportId>
celigo flows test-run-step-results <flowId> <runId> <exportOrImportId>
celigo flows test-run-step-logs <flowId> <runId> <exportOrImportId>

test-run

returns

{metadata, flowJob, childJobs}

-- metadata lists stage names per bubble. Follow with

test-run-step-results

to get

stages[] = [{name, input, output, errors}]

per bubble. Errors include

retryData

inline.

Test runs don't advance the delta timestamp -- you can repeat them safely against the same data.

Limitations: imports don't actually submit, mock data is shared across lookups/imports, and some adaptors don't run in test mode. When these bite, escalate to §6.

测试运行仅处理单个页面，不会影响生产数据或增量状态。快速、安全、无需启用调试、无副作用——可解决大多数逻辑问题。

bash

celigo flows test-run <flowId> --export <exportId>
celigo flows test-run-step-results <flowId> <runId> <exportOrImportId>
celigo flows test-run-step-logs <flowId> <runId> <exportOrImportId>

test-run

{metadata, flowJob, childJobs}

——metadata列出每个环节的阶段名称。后续执行

test-run-step-results

可获取每个环节的

stages[] = [{name, input, output, errors}]

。错误中包含内联的

retryData

。

测试运行不会推进增量时间戳——可以针对相同数据重复测试。

**局限性：**导入不会实际提交，查找/导入使用共享模拟数据，部分适配器不支持测试模式。遇到这些情况时，升级到第6节的方法。

6. End-to-end debugging with execution logs (silent/logic bugs, production incidents)

6. 使用执行日志进行端到端调试（静默/逻辑错误、生产事件）

When test-run can't answer it -- imports must actually submit, destination behavior matters, or it's a production incident -- arm execution logging, run the flow for real, then read the per-record I/O the run captured. Disable debug when you're done.

bash

undefined

当测试运行无法解决问题——导入必须实际提交、目标端行为至关重要，或发生生产事件——启用执行日志，实际运行flow，然后查看运行捕获的每条记录I/O。调试完成后禁用日志。

bash

undefined

1. Arm debug logging on the flow (optionally bound the window)

1. 为flow启用调试日志（可选设置时长）

celigo flows enable-execution-logs <flowId> [--duration <minutes>]

2. Trigger the run (or wait for the next scheduled run)

2. 触发运行（或等待下一次计划运行）

celigo flows run <flowId> -y

3. After the run, list the captured per-record logs for the job

3. 运行完成后，列出任务捕获的每条记录日志

celigo flows execution-logs <flowId> <jobId>

4. Drill into one record's stages and stage data

4. 深入查看一条记录的阶段和阶段数据

celigo flows query-execution-logs <flowId> <jobId> --export-or-import-id <id> --group-id <gid> --record-id <rid> celigo flows execution-log-detail <flowId> <jobId> --export-or-import-id <id> --stage <stage> --group-id <gid> --record-id <rid>

5. Disarm debug logging

5. 禁用调试日志

celigo flows disable-execution-logs <flowId>


Each per-record log entry names the stage that produced it (matching the `test-run-step-results` stage shape: `{ name, input, output, errors }`), so the failing stage tells you where the record broke. For raw HTTP at a bubble, use `debug-requests` / `debug-request-detail` (§7). Errors carry `retryData` inline -- see §8 to fix and retry.

celigo flows disable-execution-logs <flowId>


每条记录日志条目都会指明生成它的阶段（与`test-run-step-results`的阶段结构匹配：`{ name, input, output, errors }`），因此失败阶段会告诉你记录在哪里出现问题。如需查看环节的原始HTTP请求/响应，使用`debug-requests` / `debug-request-detail`（第7节）。错误中包含内联的`retryData`——请查看第8节了解修复和重试方法。

7. Low-level debug primitives (surgical control)

7. 底层调试原语（精准控制）

These are the debug primitives the §6 workflow builds on -- use them directly when you want manual control over arming, clearing, or probing:

bash

undefined

这些是第6节工作流所基于的调试原语——当你想要手动控制启用、清除或探查时，直接使用它们：

bash

undefined

Flow-level execution logging

Flow级执行日志

celigo flows enable-execution-logs <flowId> [--duration <minutes>] celigo flows disable-execution-logs <flowId> celigo flows execution-logs <flowId> <jobId> celigo flows query-execution-logs <flowId> <jobId> --export-or-import-id <id> --group-id <gid> --record-id <rid> celigo flows execution-log-detail <flowId> <jobId> --export-or-import-id <id> --stage <stage> --group-id <gid> --record-id <rid>

Per-bubble HTTP request/response

每个环节的HTTP请求/响应

celigo flows debug-requests <flowId> <exportOrImportId> [--since 60] celigo flows debug-request-detail <flowId> <exportOrImportId> <key>

Per-resource debug toggles (capture raw HTTP at a specific bubble)

按资源启用调试开关（捕获特定环节的原始HTTP请求/响应）

celigo exports enable-debug <id> celigo imports enable-debug <id> celigo scripts enable-debug <id> celigo connections enable-debug <id>


**Stage names** (for `execution-log-detail --stage`):
- Built-in: `apiCall`, `transformation`, `mapping`, `inputFilter`, `outputFilter`, `responseMapping`, `responseTransformation`, `routing`
- Script hooks: the **function name** wired on the bubble (e.g., `preMapHook`, `postSubmitHook`, `branchingHook`, `preSavePageHook`, `postResponseMapHook`)

**Test-run uses a different stage vocabulary** than live `/logs/data/query`: `request`/`response`/`parse` (three stages) instead of live's merged `apiCall`; `transformTwoDotZero` instead of `transformation`; `responseMap` instead of `responseMapping`; `router` instead of `routing`. Everything else matches. The live execution-log commands (§6) use the live vocabulary.

celigo exports enable-debug <id> celigo imports enable-debug <id> celigo scripts enable-debug <id> celigo connections enable-debug <id>


**阶段名称**（用于`execution-log-detail --stage`）：
- 内置阶段：`apiCall`、`transformation`、`mapping`、`inputFilter`、`outputFilter`、`responseMapping`、`responseTransformation`、`routing`
- 脚本钩子：环节上绑定的**函数名称**（如`preMapHook`、`postSubmitHook`、`branchingHook`、`preSavePageHook`、`postResponseMapHook`）

**测试运行使用的阶段术语与实时`/logs/data/query`不同**：使用`request`/`response`/`parse`（三个阶段）代替实时环境中合并的`apiCall`；使用`transformTwoDotZero`代替`transformation`；使用`responseMap`代替`responseMapping`；使用`router`代替`routing`。其他术语一致。实时执行日志命令（第6节）使用实时术语。

8. Fix and retry (or resolve)

8. 修复并重试（或解决）

Fix the configuration, then retry:

bash

celigo flows retry-errors <flowId> <exportOrImportId> -y
celigo flows retry-errors <flowId> <exportOrImportId> key1,key2,key3

Fix the data when specific records have bad values:

bash

celigo flows error <flowId> <exportOrImportId> <errorId> --retry-data > data.json

修复配置，然后重试：

bash

celigo flows retry-errors <flowId> <exportOrImportId> -y
celigo flows retry-errors <flowId> <exportOrImportId> key1,key2,key3

修复数据（当特定记录存在错误值时）：

bash

celigo flows error <flowId> <exportOrImportId> <errorId> --retry-data > data.json

Edit data.json (the retryData object), then push it back by errorId:

编辑data.json（retryData对象），然后通过errorId推送回去：

celigo flows update-error-data <flowId> <exportOrImportId> <errorId> < data.json celigo flows retry-errors <flowId> <exportOrImportId> <retryDataKey>


**Resolve without retry** when errors are expected or not worth reprocessing:

```bash
celigo flows resolve-errors <flowId> <exportOrImportId> errorId1,errorId2
celigo flows resolve-errors <flowId> <exportOrImportId> -y

celigo flows update-error-data <flowId> <exportOrImportId> <errorId> < data.json celigo flows retry-errors <flowId> <exportOrImportId> <retryDataKey>


**无需重试直接解决**（当错误是预期的或不值得重新处理时）：

```bash
celigo flows resolve-errors <flowId> <exportOrImportId> errorId1,errorId2
celigo flows resolve-errors <flowId> <exportOrImportId> -y

9. Verify the fix

9. 验证修复

Run the flow again and confirm clean execution.

bash

celigo flows run <flowId> -y
celigo jobs list --flow <flowId> --limit 1
celigo flows error-summary <flowId>

重新运行flow并确认执行无错误。

bash

celigo flows run <flowId> -y
celigo jobs list --flow <flowId> --limit 1
celigo flows error-summary <flowId>

Monitoring Lenses and Execution Metrics

监控视角与执行指标

Before diagnosing, pick the lens that matches the question -- all execution state comes through one of two:

Running -- jobs executing right now. The live view: in-progress jobs with real-time progress (records processed so far, errors accumulating, pages generated, whether the export phase is done). Use it for "what's happening this moment" -- a long-running job you're watching, or a flow you just kicked off. Reach it with
```
celigo jobs current --flow <flowId>
```
.
Completed -- historical aggregate per flow. The rear-view: run counts, average runtime, success / error / ignore totals, open errors, and when it last executed or errored. Use it for "how have things been going" -- slow flows, error-prone flows, "did the order sync run today." Reach it with
```
celigo jobs list --flow <flowId>
```
and
```
celigo jobs run-stats --flow <flowId>
```
.

Quick test: now --> running; recently / over time --> completed.

诊断前，选择与问题匹配的视角——所有执行状态都来自以下两种视角之一：

运行中——当前正在执行的任务。实时视图：进行中的任务，包含实时进度（已处理记录数、累计错误数、已生成页面数、导出阶段是否完成）。用于了解“当前正在发生什么”——正在监控的长时间运行任务，或刚启动的flow。通过
```
celigo jobs current --flow <flowId>
```
访问。
已完成——每个flow的历史汇总。回顾视图：运行次数、平均运行时间、成功/错误/忽略总数、未解决错误，以及上次执行或出错时间。用于了解“近期情况如何”——运行缓慢的flow、易出错的flow、“订单同步今天运行了吗”。通过
```
celigo jobs list --flow <flowId>
```
和
```
celigo jobs run-stats --flow <flowId>
```
访问。

快速判断：当前时刻→运行中；近期/一段时间内→已完成。

Reading the Execution Metrics

解读执行指标

Keep the record-level counts straight so you don't misread a run:

Success, error, and ignore are three distinct outcomes per record.
```
numSuccess
```
processed cleanly;
```
numError
```
failed; numIgnore
is not an error -- it's a record the flow intentionally skipped (filtered out, or a no-op upsert). Don't fold ignores into error counts: a run with a high
```
numIgnore
```
and
```
numError: 0
```
is healthy, not broken.
Open vs resolved errors. Open errors are the failures still unresolved and needing attention -- the number that matters for triage (
```
celigo flows errors <flowId> <exportOrImportId>
```
). Resolved errors -- cleared by auto-retry or by a user -- are no longer open (
```
celigo flows resolved-errors <flowId> <exportOrImportId>
```
). A flow with many total errors but zero open errors has already recovered; don't chase it.

理清记录级计数，避免误读运行结果：

成功、错误和忽略是每条记录的三种不同结果。
```
numSuccess
```
表示处理成功；
```
numError
```
表示处理失败；numIgnore
不是错误——是flow有意跳过的记录（被过滤掉，或无操作的更新）。不要将忽略计入错误数：
```
numIgnore
```
高但
```
numError: 0
```
的运行是健康的，而非故障。
**未解决错误 vs 已解决错误。**未解决错误是仍未处理、需要关注的失败——这是分类排查的关键数值（
```
celigo flows errors <flowId> <exportOrImportId>
```
）。已解决错误——通过自动重试或用户操作清除——不再处于未解决状态（
```
celigo flows resolved-errors <flowId> <exportOrImportId>
```
）。总错误数多但未解决错误数为0的flow已经恢复；无需再排查。

Stuck-Flow Triangulation

停滞Flow三角定位

Every connection is backed by its own FIFO message queue, shared by every flow (and API or Tool step) that uses that connection and drained up to the connection's

concurrencyLevel

in parallel. When a run seems stuck --

queued

, "not moving", "started but nothing is happening" -- the cause is almost never visible on the resource's configuration. Triangulate three live facts before offering any explanation:

The run's actual state -- the latest job:
```
queued
```
,
```
running
```
(with or without progress), or already finished. Get it with
```
celigo jobs current --flow <flowId>
```
and
```
celigo jobs list --flow <flowId> --limit 1
```
.
Queue depth on each connection the steps drain through -- how many messages are already in line ahead of this run.
What else is running or queued right now on those same connections -- other flows and integrations contending for the same drain.

A bare

queued

status is not a diagnosis on its own. Calibrate every conclusion to the legs you actually verified:

Verified blocker -- runtime evidence links it to this run: a queue this run's steps drain through currently holds messages ahead of it.
Likely hypothesis -- consistent with the evidence but not linked at runtime. Another flow's job running or queued right now on a shared connection is a real observation whose blocking role is still an inference (per-slot occupancy isn't visible) -- present it as the likely cause, never confirmed. A flow that merely shares the connection but isn't currently active is a candidate at most.
Unknown -- a leg couldn't be fetched. Name exactly which evidence is missing and keep the diagnosis unconfirmed.

A shared connection alone is never proof. "Another flow also uses this connection" becomes a diagnosis only when that flow's work is running or sitting in the queue right now.

Speak in flows and integrations, not queues and exports. Walk the chain up and name the flows (and their integrations) producing the load: "your run is waiting behind ~1,200 messages on the NetSuite connection, mostly from 'Inventory Sync', which is running right now" is a diagnosis; "the connection queue has 1,200 messages" is not.

A backlog is an explanation, not a defect -- but verify it drains. Work queued ahead means the run is waiting its turn, not broken; it starts when the queue drains. One snapshot can't tell a draining queue from a wedged one, so re-check in ~10-15 minutes. If the second look shows the same depth with the job still waiting -- especially with nothing else running or queued on those connections (idle capacity next to a standing line is itself an anomaly) -- stop advising patience and escalate. Raising

concurrencyLevel

can lift throughput, but that's a connection change, not the fix for a one-time backlog.

Truly stuck = empty queues + nothing else active + still not moving. That combination rules out the account and points at the platform -- capture a

celigo jobs diagnostics <jobId>

bundle and escalate to Celigo support.

A fix is a hypothesis until a later run proves it. After any corrective action -- re-enabling the flow, a config change, or canceling a wedged job (

celigo jobs cancel <jobId>

) -- re-run and compare before/after job state before calling the incident resolved. A config-only change with no new run is not runtime-validated.

每个连接都有自己的FIFO消息队列，供使用该连接的所有flow（以及API或Tool步骤）共享，并按连接的

concurrencyLevel

并行处理。当运行似乎停滞——

queued

、“无进展”、“已启动但无任何动作”——原因几乎无法从资源配置中直接看到。在给出任何解释前，先确认三个实时事实：

运行的实际状态——最新任务：
```
queued
```
、
```
running
```
（有/无进度），或已完成。通过
```
celigo jobs current --flow <flowId>
```
和
```
celigo jobs list --flow <flowId> --limit 1
```
获取。
步骤所使用的每个连接的队列深度——此运行之前已有多少消息在队列中等待。
这些相同连接上当前正在运行或排队的其他任务——其他flow和集成正在争夺相同的处理资源。

单纯的

queued

状态本身不是诊断结论。所有结论都必须基于已验证的事实：

已验证的阻塞因素——运行时证据表明其与此运行相关：此运行步骤所使用的队列当前有消息在其之前等待。
可能的假设——与证据一致但未在运行时关联。当前在共享连接上运行或排队的其他flow是真实观察结果，但其阻塞作用仍为推断（无法看到每个槽位的占用情况）——应表述为可能原因，而非已确认原因。仅共享连接但当前未活跃的flow最多只是候选因素。
未知——无法获取某一事实。明确指出缺失的证据，并保持诊断未确认状态。

仅共享连接绝非证据。“另一个flow也使用此连接”只有在该flow的任务当前正在运行或排队时才成为诊断结论；否则只是候选因素。

**用flow和集成表述，而非队列和导出。**梳理流程并指出产生负载的flow（及其集成）：“你的运行在NetSuite连接上等待约1200条消息，主要来自当前正在运行的‘库存同步’flow”是诊断结论；“连接队列有1200条消息”不是。

**积压是解释，而非缺陷——但需验证是否在处理。**队列中有任务等待意味着运行在排队等候，而非故障；队列处理完后就会启动。单次快照无法区分正在处理的队列和停滞的队列，因此约10-15分钟后重新检查。如果第二次检查显示队列深度相同且任务仍在等待——尤其是在这些连接上没有其他运行或排队任务（空闲容量与排队任务并存本身就是异常）——停止建议等待并升级处理。提高

concurrencyLevel

可以提升吞吐量，但这是连接配置变更，不是一次性积压的解决方案。

**真正的停滞=空队列+无其他活跃任务+仍无进展。**这种组合排除了账户层面的问题，指向平台层面——捕获

celigo jobs diagnostics <jobId>

包并升级到Celigo支持团队。

**修复方案在后续运行验证前只是假设。**采取任何纠正措施后——重新启用flow、配置变更、取消停滞任务（

celigo jobs cancel <jobId>

）——重新运行并比较前后任务状态，再宣布事件解决。仅变更配置但未重新运行无法在运行时验证。

CLI Commands

CLI命令

All commands shown in the Diagnostic Workflow above, plus these additional commands:

bash

undefined

上述诊断工作流中显示的所有命令，加上以下附加命令：

bash

undefined

Job inspection (additional)

任务检查（附加）

celigo jobs cancel <jobId> [-y] celigo jobs diagnostics <jobId> celigo jobs download-files <jobId> celigo jobs get <jobId> celigo jobs errors <jobId> celigo jobs run-stats [--flow <flowId>] [--status <status>]

Error investigation (additional)

错误调查（附加）

celigo flows resolved-errors <flowId> <exportOrImportId>

Error resolution (additional)

错误解决（附加）

celigo flows assign-errors <flowId> <exportOrImportId> <email> [errorIds] [-y] celigo flows delete-resolved-errors <flowId> <exportOrImportId> [errorIds] [-y] celigo flows tag-errors <flowId> <exportOrImportId>

Debug logging (additional)

调试日志（附加）

celigo flows query-execution-logs <flowId> <jobId> --export-or-import-id <id> --group-id <gid> --record-id <rid>

Flow state

Flow状态

celigo flows last-export-date <flowId> celigo flows run <flowId> [--start-date <ISO8601>] [--end-date <ISO8601>] [-y]


<!-- TIER:3 -->

celigo flows last-export-date <flowId> celigo flows run <flowId> [--start-date <ISO8601>] [--end-date <ISO8601>] [-y]


<!-- TIER:3 -->

Diagnostic Checklist

诊断检查清单

Before escalating or concluding investigation:

Checked job status via

celigo jobs list --flow <flowId> --limit 1

-- confirmed

status

numError

numSuccess

numPagesGenerated

Ran
```
celigo flows error-summary
```
to identify which step(s) have errors
Ran
```
celigo flows error-analysis
```
to group errors by pattern and identify root cause

Inspected individual errors via

celigo flows errors

and

celigo flows error <errorId> --retry-data

Reviewed raw HTTP request/response via

celigo flows error <errorId> --request-detail

If errors are unclear: enabled execution logs, re-ran flow, and inspected record-level trace
If HTTP-level detail needed: used
```
celigo flows debug-requests
```
on the failing export/import
Verified fix by re-running the flow and confirming clean execution

升级处理或结束调查前：

通过

celigo jobs list --flow <flowId> --limit 1

检查任务状态——确认

status

、

numError

、

numSuccess

、

numPagesGenerated

执行
```
celigo flows error-summary
```
确定哪些步骤存在错误
执行
```
celigo flows error-analysis
```
按模式分组错误并识别根本原因

通过

celigo flows errors

和

celigo flows error <errorId> --retry-data

检查单个错误

通过

celigo flows error <errorId> --request-detail

查看原始HTTP请求/响应

如果错误不明确：启用执行日志，重新运行flow，检查记录级追踪信息
如果需要HTTP级细节：对失败的导出/导入使用
```
celigo flows debug-requests
```
通过重新运行flow并确认执行无错误来验证修复

Gotchas

注意事项

A
completed
job can still have errors.
```
completed
```
means the job finished, not that every record succeeded. Always check
```
numError
```
alongside status.
error-analysis
only samples up to
--limit
errors. Default is 100. For flows with thousands of errors, increase the limit to get an accurate pattern distribution.

flows error
takes the
errorId
(the
_id
from
flows errors
), not a
reqAndResKey
or
retryDataKey
. It resolves those internal keys for you:

--request-detail

follows the error's

reqAndResKey

--retry-data

follows its

retryDataKey

. Use

debug-request-detail

only when you already hold a raw

reqAndResKey

from

debug-requests

Debug execution logs auto-disable after
--duration
minutes. Default is 60. If your flow runs after the window expires, you get no logs. Enable, then run promptly.
Test runs don't advance delta state. This is intentional -- you can test repeatedly with the same data. But it means test runs always re-fetch the same records.
Retrying resolved errors is not possible. Once resolved, an error cannot be retried. Only resolve errors you're certain don't need reprocessing.
lastExportDateTime
drift causes re-processing or gaps. If a delta export's timestamp is wrong, use
```
flows run --start-date
```
to override and reprocess a specific window.
Empty
numPagesGenerated: 0
on a failed job means the export itself failed. The problem is the source step -- check the connection, query, or endpoint, not the import.
Execution log data is ephemeral. Logs are retained for a limited time. Enable logging and run the flow promptly.
jobs diagnostics
produces a diagnostic bundle. Use this when Celigo support asks for details -- it includes internal execution context not visible through other commands.
numIgnore
is not
numError
. An ignored record was intentionally skipped (filtered out, or a no-op upsert), not failed. A run with a high
```
numIgnore
```
and zero
```
numError
```
is healthy -- don't triage it as a failure.
Open errors are the triage number, not total errors. A flow with thousands of total errors but zero open errors has already recovered via auto-retry or manual resolve. Compare
```
flows errors
```
(open) against
```
flows resolved-errors
```
before investigating.
A
queued
status is not a diagnosis, and a shared connection is not proof. Triangulate job state, connection queue depth, and what else is draining that connection first. Another flow blocks yours only when its work is running or queued right now -- merely sharing the connection makes it a candidate at most. A backlog means the run is waiting its turn; re-check in ~10-15 minutes before escalating.

completed
状态的任务仍可能存在错误。
```
completed
```
表示任务已完成，而非每条记录都成功。始终同时检查
```
numError
```
和状态。
**
```
error-analysis
```
仅采样最多
```
--limit
```
个错误。**默认值为100。对于有数千个错误的flow，增加限制以获取准确的模式分布。

flows error

需要

errorId

（来自

flows errors

的

_id

），而非

reqAndResKey

或

retryDataKey

。**它会自动解析这些内部键：

--request-detail

会跟踪错误的

reqAndResKey

，

--retry-data

会跟踪其

retryDataKey

。仅当已从

debug-requests

获取原始

reqAndResKey

时，才使用

debug-request-detail

。

**调试执行日志会在
```
--duration
```
分钟后自动禁用。**默认值为60。如果flow在窗口过期后运行，将无法获取日志。启用日志后立即运行flow。
**测试运行不会推进增量状态。**这是有意设计的——可以针对相同数据重复测试。但这意味着测试运行总是重新获取相同记录。
**无法重试已解决的错误。**错误解决后，无法再重试。仅解决确定无需重新处理的错误。
**
```
lastExportDateTime
```
偏移会导致重复处理或数据遗漏。**如果增量导出的时间戳错误，使用
```
flows run --start-date
```
覆盖并重新处理特定窗口。
**失败任务的
```
numPagesGenerated: 0
```
表示导出本身失败。**问题出在源步骤——检查连接、查询或端点，而非导入。
**执行日志数据是临时的。**日志仅保留有限时间。启用日志后立即运行flow。
**
```
jobs diagnostics
```
生成诊断包。**当Celigo支持团队要求提供详细信息时使用——它包含其他命令无法查看的内部执行上下文。
**
```
numIgnore
```
不是
```
numError
```
。**被忽略的记录是有意跳过的（被过滤掉，或无操作的更新），而非处理失败。
```
numIgnore
```
高但
```
numError
```
为0的运行是健康的——不要将其归类为故障。
**未解决错误是分类排查的关键数值，而非总错误数。**总错误数多但未解决错误数为0的flow已通过自动重试或手动解决恢复。排查前比较
```
flows errors
```
（未解决）和
```
flows resolved-errors
```
（已解决）。
**
```
queued
```
状态不是诊断结论，共享连接也不是证据。**先三角定位任务状态、连接队列深度以及正在使用该连接的其他flow。另一个flow仅在其任务当前正在运行或排队时才会阻塞你的flow——仅共享连接最多只是候选因素。积压意味着运行在排队等候；升级处理前约10-15分钟重新检查。

Common Errors

常见错误

Symptom	Likely Cause	Diagnostic Steps
`failed` with `numPagesGenerated: 0`	Export-level failure (connection, query, endpoint)	Check connection status ( `celigo connections ping` ); review export config
`completed` with high `numError`	Destination validation or data issues	`error-analysis` to find pattern; `flows error --request-detail` for HTTP detail
`completed` with 0 records, 0 errors	Wrong `resourcePath` , empty delta, or filter too restrictive	Verify `resourcePath` ; check `lastExportDateTime` ; review output filter
`retrying` for extended period	Rate limiting, slow destination, or large dataset	Check destination rate limits; review concurrency on connection
Errors only on specific records	Data-dependent issue (missing fields, bad types, duplicates)	`flows error --retry-data` to inspect failing records; compare with successful records
Intermittent `failed` on same flow	Token expiry mid-run, transient network, or rate limits	Compare timestamps of failures; check connection token refresh config
401/403 errors in `flows error --request-detail`	Expired credentials or insufficient permissions	`celigo connections ping` ; re-authorize if OAuth; check API permissions
Timeout errors	Slow destination or oversized payload	Reduce batch size; check destination system performance
`queued` / `running` but not advancing	Waiting behind queued work on a shared connection, or a platform-side stall	Triangulate job state ( `jobs current` ), connection queue depth, and other flows draining that connection; re-check in 10-15 min. If empty queues + nothing active + still stuck, capture `jobs diagnostics` and escalate

症状	可能原因	诊断步骤
`failed` 且 `numPagesGenerated: 0`	导出级失败（连接、查询、端点）	检查连接状态（ `celigo connections ping` ）；查看导出配置
`completed` 且 `numError` 高	目标端验证或数据问题	`error-analysis` 查找模式； `flows error --request-detail` 查看HTTP细节
`completed` 且0条记录、0个错误	`resourcePath` 错误、增量为空或过滤过于严格	验证 `resourcePath` ；检查 `lastExportDateTime` ；查看输出过滤
长时间处于 `retrying` 状态	速率限制、目标端缓慢或大型数据集	检查目标端速率限制；查看连接的并发设置
仅特定记录出现错误	数据相关问题（缺失字段、类型错误、重复数据）	`flows error --retry-data` 检查失败记录；与成功记录对比
同一flow间歇性 `failed`	运行过程中令牌过期、临时网络问题或速率限制	比较失败时间戳；检查连接令牌刷新配置
`flows error --request-detail` 中出现401/403错误	凭证过期或权限不足	`celigo connections ping` ；如果是OAuth，重新授权；检查API权限
超时错误	目标端缓慢或负载过大	减小批量大小；检查目标系统性能
`queued` / `running` 但无进展	在共享连接上等待排队任务，或平台端停滞	三角定位任务状态（ `jobs current` ）、连接队列深度以及使用该连接的其他flow；10-15分钟后重新检查。如果空队列+无其他活跃任务+仍停滞，捕获 `jobs diagnostics` 并升级处理

troubleshooting-flows

Original

Translation

Troubleshooting Flows

Flow故障排查

Problem Categories

问题分类

Total Failure

完全失败

Partial Failure

部分失败

Empty Run

空运行

Stuck or Long-Running

停滞或长时间运行

Intermittent Failures

间歇性失败

Error Diagnosis Framework

错误诊断框架

Classification

分类

Root Cause: Configuration vs Data

根本原因：配置 vs 数据

Check if all records fail (configuration) or only some (data)

检查是所有记录都失败（配置问题）还是仅部分记录失败（数据问题）

Which Step Failed?

哪个步骤失败了？

Quick Reference

快速参考

Symptom --> First Command

症状 --> 优先执行的命令

Key Diagnostic Commands

关键诊断命令

Job status

任务状态

Error investigation

错误调查

Safe iteration first

优先进行安全迭代

End-to-end execution logging (real run, full per-record I/O)

端到端执行日志（实际运行，完整的每条记录I/O）

Related Skills

相关技能

Diagnostic Workflow

诊断工作流

1. Check the job status

1. 检查任务状态

2. Get the error summary

2. 获取错误摘要

3. Analyze error patterns

3. 分析错误模式

4. Inspect individual errors

4. 检查单个错误

5. Use test runs for safe iteration (try this first)

5. 使用测试运行进行安全迭代（优先尝试此方法）

6. End-to-end debugging with execution logs (silent/logic bugs, production incidents)

6. 使用执行日志进行端到端调试（静默/逻辑错误、生产事件）

1. Arm debug logging on the flow (optionally bound the window)

1. 为flow启用调试日志（可选设置时长）

2. Trigger the run (or wait for the next scheduled run)

2. 触发运行（或等待下一次计划运行）

3. After the run, list the captured per-record logs for the job

3. 运行完成后，列出任务捕获的每条记录日志

4. Drill into one record's stages and stage data

4. 深入查看一条记录的阶段和阶段数据

5. Disarm debug logging

5. 禁用调试日志

7. Low-level debug primitives (surgical control)

7. 底层调试原语（精准控制）

Flow-level execution logging

Flow级执行日志

Per-bubble HTTP request/response

每个环节的HTTP请求/响应

Per-resource debug toggles (capture raw HTTP at a specific bubble)

按资源启用调试开关（捕获特定环节的原始HTTP请求/响应）

8. Fix and retry (or resolve)

8. 修复并重试（或解决）

Edit data.json (the retryData object), then push it back by errorId:

编辑data.json（retryData对象），然后通过errorId推送回去：

9. Verify the fix