workflow-engine-mapper

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Workflow Engine Mapper

工作流引擎映射工具

Goal

目标

Choose the smallest workflow structure that preserves reproducibility, restartability, and provenance for a materials simulation task.

为材料模拟任务选择最小化的工作流结构，同时确保可复现性、可重启性和溯源性。

Requirements

要求

Python 3.10+
No external dependencies
Works on Linux, macOS, and Windows

Python 3.10+
无外部依赖
支持Linux、macOS和Windows系统

Inputs to Gather

需要收集的输入信息

Input	Description	Example
Task	Workflow purpose	`VASP relax-static-DOS for 200 structures`
Code	Main simulation engine	`vasp` , `qe` , `lammps` , `ase`
Runs	Approximate number of calculations	`200`
Provenance	Whether audit trail matters	`true`
Restart	Whether jobs may resume after failure	`true`
HPC	Whether remote scheduler is required	`true`

输入项	描述	示例
任务	工作流用途	`VASP relax-static-DOS for 200 structures`
代码	主模拟引擎	`vasp` , `qe` , `lammps` , `ase`
运行次数	估算的计算次数	`200`
溯源性	是否需要审计追踪	`true`
重启功能	任务失败后是否需要恢复	`true`
HPC	是否需要远程调度器	`true`

Decision Guidance

决策指南

Use one-off scripts for fewer than 5 local exploratory runs (no provenance, no HPC).
Use jobflow/atomate2 when the workflow is Python-native and Materials Project style input sets are useful.
Use AiiDA when provenance-critical work is also remote (HPC) or large (>= 50 runs) — i.e. long-lived, database-backed campaigns. For smaller local provenance needs, atomate2 (Materials Project codes, >= 10 runs) or jobflow stores already capture inputs, outputs, code version, and environment, so the mapper recommends those instead of the heavier AiiDA stack.
Use pyiron when interactive atomistic workflows, notebooks, and job management are the primary user surface (ASE/LAMMPS without strict provenance).

The recommendations are emitted in a fixed precedence so the prose and the implemented thresholds agree: an explicit

--preferred

engine overrides everything; otherwise one-off (small local, no provenance/HPC) -> AiiDA (provenance AND remote/large) -> atomate2 (VASP/QE/CP2K/force-field, >= 10 runs) -> pyiron (ASE/LAMMPS, no provenance) -> jobflow (fallback).

针对少于5次的本地探索性运行（无需溯源、无需HPC），使用一次性脚本。
当工作流为Python原生，且Materials Project风格的输入集有用时，使用jobflow/atomate2。
当需要严格溯源的工作同时涉及远程（HPC）或大规模（≥50次运行）场景——即长期运行、基于数据库的计算任务时，使用AiiDA。对于规模较小的本地溯源需求，atomate2（Materials Project支持的代码、≥10次运行）或jobflow的存储已能捕获输入、输出、代码版本和环境信息，因此映射工具会推荐这些方案，而非更重型的AiiDA栈。
当以交互式原子工作流、笔记本和任务管理为主要使用场景（无需严格溯源的ASE/LAMMPS任务）时，使用pyiron。

推荐结果遵循固定优先级，以确保文字说明与实现的阈值一致：显式指定的

--preferred

引擎会覆盖所有规则；否则优先级顺序为：一次性脚本（小规模本地任务、无需溯源/HPC）→ AiiDA（需要溯源且为远程/大规模任务）→ atomate2（代码为vasp/qe/CP2K/力场、≥10次运行）→ pyiron（ASE/LAMMPS、无需溯源）→ jobflow（兜底方案）。

Script Outputs

脚本输出

scripts/workflow_engine_mapper.py

emits:

```
recommended_engine
```
```
dag_pattern
```
```
provenance_requirements
```
```
restart_strategy
```
```
storage_layout
```
```
migration_triggers
```
```
notes
```

scripts/workflow_engine_mapper.py

会输出以下内容：

```
recommended_engine
```
```
dag_pattern
```
```
provenance_requirements
```
```
restart_strategy
```
```
storage_layout
```
```
migration_triggers
```
```
notes
```

Workflow

使用流程

bash

python3 skills/simulation-workflow/workflow-engine-mapper/scripts/workflow_engine_mapper.py \
  --task "relax static dos for 200 oxides" \
  --code vasp \
  --runs 200 \
  --needs-provenance \
  --needs-restart \
  --hpc \
  --json

Use the output to scaffold the workflow before writing engine-specific code.

bash

python3 skills/simulation-workflow/workflow-engine-mapper/scripts/workflow_engine_mapper.py \
  --task "relax static dos for 200 oxides" \
  --code vasp \
  --runs 200 \
  --needs-provenance \
  --needs-restart \
  --hpc \
  --json

在编写引擎专属代码前，可使用输出内容搭建工作流框架。

Error Handling

错误处理

If the task has too few details, choose the conservative pattern and ask for engine, run count, and restart needs before implementation.

若任务信息不足，选择保守模式，并在实现前询问引擎类型、运行次数和重启需求。

Limitations

局限性

The skill does not replace the official APIs of atomate2, jobflow, AiiDA, or pyiron; it selects and explains the workflow shape.

本工具无法替代atomate2、jobflow、AiiDA或pyiron的官方API；它仅负责选择并解释工作流的架构。

Verification checklist

验证清单

Recorded the full
```
--json
```
payload from
```
workflow_engine_mapper.py
```
(including
```
inputs
```
) and confirmed the echoed
```
runs
```
,
```
code
```
,
```
needs_provenance
```
,
```
needs_restart
```
, and
```
hpc
```
match the task you actually intend, not a guessed default.
Confirmed
```
recommended_engine
```
follows the documented precedence for these inputs:
```
--preferred
```
override -> one-off (
```
runs<5
```
, no provenance, no HPC) -> aiida (provenance AND (
```
hpc
```
or
```
runs>=50
```
)) -> atomate2 (
```
code
```
in vasp/qe/cp2k/forcefield AND
```
runs>=10
```
) -> pyiron (ase/lammps, no provenance) -> jobflow fallback; if the result surprises you, re-check which branch the inputs hit rather than overriding blindly.
Verified
```
dag_pattern
```
reflects the task keywords: branch terms (
```
dos
```
/
```
band
```
/
```
phonon
```
/
```
static
```
) and sweep terms (
```
screen
```
/
```
sweep
```
/
```
campaign
```
/
```
many
```
/
```
batch
```
) compose into the map+branch pattern, and
```
with restart checkpoints
```
is appended only when
```
--needs-restart
```
was passed.
Checked
```
provenance_requirements
```
and
```
restart_strategy
```
against intent:
```
store_code_version
```
/
```
store_environment
```
,
```
checkpoint_jobs
```
(also auto-true at
```
runs>=20
```
), and
```
resume_by_job_id_or_name
```
(false for one-off) are consistent with how the campaign will actually be audited and resumed.
Read
```
migration_triggers
```
; for a one-off recommendation, confirmed the forward-looking "migrate once results are compared/published/screened/resumed or runs reach ~5+" entry is present and planned for, rather than treating one-off as permanent.
Used
```
storage_layout
```
(
```
inputs/
```
,
```
runs/<job-id>/
```
,
```
outputs/
```
,
```
metadata/workflow.json
```
,
```
reports/
```
) as the on-disk scaffold and confirmed it maps onto the chosen engine's native store before writing engine-specific code.
Confirmed the mapper exit code was
```
0
```
(a
```
2
```
means input validation rejected the args and no recommendation was produced) before trusting any output.

记录
```
workflow_engine_mapper.py
```
输出的完整
```
--json
```
负载（包括
```
inputs
```
），并确认回显的
```
runs
```
、
```
code
```
、
```
needs_provenance
```
、
```
needs_restart
```
和
```
hpc
```
与实际任务需求一致，而非默认值。
确认
```
recommended_engine
```
符合文档中定义的输入优先级：
```
--preferred
```
覆盖规则 → 一次性脚本（
```
runs<5
```
、无需溯源、无需HPC）→ aiida（需要溯源且（使用HPC或
```
runs>=50
```
））→ atomate2（
```
code
```
属于vasp/qe/cp2k/力场且
```
runs>=10
```
）→ pyiron（ase/lammps、无需溯源）→ jobflow兜底；若结果不符合预期，先检查输入触发了哪个分支，而非盲目覆盖。
验证
```
dag_pattern
```
反映了任务关键字：分支术语（
```
dos
```
/
```
band
```
/
```
phonon
```
/
```
static
```
）和扫描术语（
```
screen
```
/
```
sweep
```
/
```
campaign
```
/
```
many
```
/
```
batch
```
）组合成映射+分支模式，且仅当传递了
```
--needs-restart
```
时才会追加
```
with restart checkpoints
```
。
检查
```
provenance_requirements
```
和
```
restart_strategy
```
是否与预期一致：
```
store_code_version
```
/
```
store_environment
```
、
```
checkpoint_jobs
```
（当
```
runs>=20
```
时自动设为true）和
```
resume_by_job_id_or_name
```
（一次性脚本为false）需与任务实际的审计和恢复方式一致。
阅读
```
migration_triggers
```
；若推荐为一次性脚本，确认其中包含前瞻性的“当结果完成对比/发布/筛选/恢复或运行次数达到~5+时进行迁移”条目，并做好规划，不要将一次性脚本视为永久方案。
使用
```
storage_layout
```
（
```
inputs/
```
、
```
runs/<job-id>/
```
、
```
outputs/
```
、
```
metadata/workflow.json
```
、
```
reports/
```
）作为磁盘架构，并确认其与所选引擎的原生存储结构匹配后，再编写引擎专属代码。
确认映射工具的退出码为
```
0
```
（退出码
```
2
```
表示输入验证未通过，未生成推荐结果），再信任输出内容。

Common pitfalls & rationalizations

常见误区与合理化建议

Tempting shortcut	Why it's wrong / what to do
"It's only a few runs now, so a one-off script is fine forever."	The mapper emits a `migration_triggers` entry precisely because one-off is exploratory-only; once results are compared, published, screened, or resumed (or runs reach ~5+), promote to an engine. Treat one-off as a starting point, not a destination.
"I need provenance, so the answer must be AiiDA."	AiiDA is gated on provenance AND (HPC or `runs>=50` ). For smaller/local provenance needs the mapper deliberately picks atomate2 or jobflow, whose stores already capture inputs, outputs, code version, and environment. Do not reach for the heavier stack when the lighter store suffices.
"The engine ran and printed a recommendation, so the inputs were right."	The script validates bounds, not intent. A forgotten `--needs-provenance` or `--hpc` flag, or a defaulted `--code general` , silently changes the branch taken. Re-read the echoed `inputs` block in the JSON and confirm each flag matches the real task.
"It's a screening campaign, so the DAG is just a map over structures."	If the task also names a property (dos/band/phonon/static), the correct pattern composes both: `map over structures -> (relax -> static -> property branches) -> collect -> rank` . Check that branch and sweep keywords both surfaced in `dag_pattern` .
"Restart isn't critical, so I can skip checkpointing."	`checkpoint_jobs` is also forced true at `runs>=20` independent of `--needs-restart` , because large sweeps fail partway. Honor the emitted `restart_strategy` rather than your gut feel about restart importance.
"The recommended engine doesn't match what I'd have picked, so I'll just use `--preferred` ."	`--preferred` overrides everything before any heuristic runs, so it can mask a genuine mismatch in your inputs. First confirm `runs` / `code` /provenance/HPC are stated correctly; only override when you have a specific reason, and record it.

诱人的捷径	错误原因及正确做法
"现在只是几次运行，所以一次性脚本可以一直用下去。"	映射工具输出 `migration_triggers` 条目正是因为一次性脚本仅适用于探索阶段；一旦结果完成对比、发布、筛选或恢复（或运行次数达到~5+），就应升级到正式引擎。将一次性脚本视为起点，而非终点。
"我需要溯源功能，所以必须选AiiDA。"	AiiDA的使用条件是需要溯源且（使用HPC或 `runs>=50` ）。对于规模较小的本地溯源需求，映射工具会刻意选择atomate2或jobflow，因为它们的存储已能捕获输入、输出、代码版本和环境信息。不要在轻量级存储足够的情况下使用重型栈。
"工具运行并给出了推荐，所以输入肯定是对的。"	脚本仅验证输入的合法性，而非合理性。遗漏的 `--needs-provenance` 或 `--hpc` 标志，或默认的 `--code general` ，都会悄无声息地改变分支走向。重新阅读JSON中的 `inputs` 块，确认每个标志都与实际任务匹配。
"这是一个筛选任务，所以DAG只是对结构的映射。"	如果任务还指定了属性（dos/band/phonon/static），正确的模式应同时包含两者： `遍历结构 -> (弛豫 -> 静态计算 -> 属性分支) -> 收集 -> 排序` 。检查 `dag_pattern` 中是否同时包含分支和扫描关键字。
"重启不重要，所以我可以跳过 checkpointing。"	当 `runs>=20` 时， `checkpoint_jobs` 会被强制设为true，与 `--needs-restart` 无关，因为大规模扫描任务中途失败的概率更高。遵循输出的 `restart_strategy` ，而非仅凭直觉判断重启的重要性。
"推荐的引擎不符合我的预期，所以我直接用 `--preferred` 指定。"	`--preferred` 会在任何启发式规则运行前覆盖所有结果，因此可能掩盖输入中的真实不匹配。首先确认 `runs` / `code` /溯源/HPC的设置是否正确；仅当有特定理由时才进行覆盖，并记录原因。

Security

安全性

Input Validation

输入验证

The script accepts only scalar CLI inputs and boolean flags (
```
--needs-provenance
```
,
```
--needs-restart
```
,
```
--hpc
```
,
```
--json
```
via
```
store_true
```
).
```
runs
```
must be a positive integer (rejects booleans, non-integers, and values
```
<= 0
```
) and is capped at
```
MAX_RUNS = 1,000,000
```
;
```
main()
```
also rejects non-finite values via
```
math.isfinite
```
.
Free-text fields are length-bounded:
```
task
```
<= 2000 characters (
```
MAX_TASK_LEN
```
) and must be non-empty after stripping;
```
code
```
and
```
preferred
```
<= 100 characters each (
```
MAX_FIELD_LEN
```
).

preferred

must be one of the allowed engine names:

auto

one-off

jobflow

atomate2

aiida

pyiron

The
```
task
```
and
```
code
```
strings are not otherwise restricted by an allowlist; only their length and (for
```
task
```
) emptiness are validated.
All invalid input raises
```
ValueError
```
, which is printed to stderr and exits with code 2 before any recommendation is computed.

脚本仅接受标量CLI输入和布尔标志（
```
--needs-provenance
```
、
```
--needs-restart
```
、
```
--hpc
```
、
```
--json
```
通过
```
store_true
```
实现）。
```
runs
```
必须为正整数（拒绝布尔值、非整数和
```
<=0
```
的值），且上限为
```
MAX_RUNS = 1,000,000
```
；
```
main()
```
还会通过
```
math.isfinite
```
拒绝非有限值。
自由文本字段有长度限制：
```
task
```
长度不超过2000字符（
```
MAX_TASK_LEN
```
），且去除空白后不能为空；
```
code
```
和
```
preferred
```
长度不超过100字符（
```
MAX_FIELD_LEN
```
）。
```
preferred
```
必须为允许的引擎名称之一：
```
auto
```
、
```
one-off
```
、
```
jobflow
```
、
```
atomate2
```
、
```
aiida
```
、
```
pyiron
```
。
```
task
```
和
```
code
```
字符串不通过白名单限制；仅验证其长度（以及
```
task
```
是否为空）。
所有无效输入都会触发
```
ValueError
```
，错误信息会打印到stderr，并以退出码2终止，不会生成任何推荐结果。

File Access

文件访问

The script reads and writes no files; all I/O is CLI args in -> stdout (JSON or two summary lines) out.
It accepts no path arguments, so there is no path-sandboxing concern; there are no on-disk size limits because nothing is read from disk.

脚本不读取或写入任何文件；所有I/O均为CLI输入→stdout输出（JSON或两行摘要）。
不接受路径参数，因此不存在路径沙箱问题；由于不读取磁盘内容，也没有磁盘大小限制。

Tool Restrictions

工具限制

The frontmatter declares

allowed-tools: Read, Bash, Write, Grep, Glob

```
Bash
```
is used only to run the bundled
```
scripts/workflow_engine_mapper.py
```
.
```
Read
```
/
```
Grep
```
/
```
Glob
```
are used to inspect the skill's own files and references (e.g.
```
references/workflow_engines.md
```
) and the user's task context;
```
Write
```
is available to scaffold workflow files from the recommended structure.

前置声明

allowed-tools: Read, Bash, Write, Grep, Glob

。

```
Bash
```
仅用于运行内置的
```
scripts/workflow_engine_mapper.py
```
。
```
Read
```
/
```
Grep
```
/
```
Glob
```
用于检查工具自身的文件和参考资料（如
```
references/workflow_engines.md
```
）以及用户的任务上下文；
```
Write
```
用于根据推荐的结构搭建工作流文件。

Safety Measures

安全措施

No
```
eval
```
/
```
exec
```
, no
```
subprocess
```
, and no shell invocation inside the script.
No network access: it does not connect to remote services, submit jobs, or deserialize untrusted data.
Output is emitted as structured JSON via
```
json.dumps
```
(with
```
--json
```
) or plain text summary lines.
DoS caps bound resource use: the
```
runs
```
ceiling (1,000,000) and the
```
task
```
/
```
code
```
/
```
preferred
```
length caps prevent unbounded input.

不使用
```
eval
```
/
```
exec
```
、
```
subprocess
```
，脚本内部也不调用shell。
无网络访问：不连接远程服务、不提交任务、不反序列化不可信数据。
输出通过
```
json.dumps
```
以结构化JSON格式（使用
```
--json
```
时）或纯文本摘要行形式输出。
DoS限制约束资源使用：
```
runs
```
上限（1,000,000）和
```
task
```
/
```
code
```
/
```
preferred
```
的长度限制可防止无限制输入。

References

参考资料

See
```
references/workflow_engines.md
```
for engine selection heuristics.

引擎选择启发式规则详见
```
references/workflow_engines.md
```
。

Version History

版本历史

1.2.2: Add a Verification checklist (evidence tied to the mapper's JSON outputs, precedence, DAG composition, and exit code) and a Common pitfalls & rationalizations table covering one-off permanence, AiiDA over-selection, unread input flags, and forced checkpointing.
1.2.0: Strengthen evals with deterministic
```
script_checks
```
that pin the mapper's exact output (recommended_engine, dag_pattern, provenance_requirements, restart_strategy, migration_triggers) so each case discriminates the skill from a from-memory baseline.
1.1.0: Compose branch+sweep DAG patterns instead of overwriting; emit a forward-looking migration trigger for one-off runs; document AiiDA gating/precedence; add input-validation safeguards (bounds, length caps) matching the Security section.
1.0.0: Initial workflow engine mapping skill.

1.2.2：添加验证清单（与映射工具的JSON输出、优先级、DAG组合和退出码相关的验证项）和常见误区与合理化建议表，涵盖一次性脚本的永久性、过度选择AiiDA、未阅读输入标志和强制checkpointing等问题。
1.2.0：通过确定性的
```
script_checks
```
增强评估，固定映射工具的精确输出（recommended_engine、dag_pattern、provenance_requirements、restart_strategy、migration_triggers），使每个案例都能区分本工具与基于记忆的基线方案。
1.1.0：组合分支+扫描DAG模式而非覆盖；为一次性运行输出前瞻性的迁移触发条件；记录AiiDA的准入规则/优先级；添加与安全章节匹配的输入验证防护（边界、长度限制）。
1.0.0：初始版本的工作流引擎映射工具。