MaaFW Pipeline Development
The full development workflow is divided into two tracks; choose your track before starting:
- New Creation Track: Build a pipeline (task or sub-flow) from scratch — Requirements → Breakdown & State Inventory (including exploration and evidence collection) → File Structure & Naming → Asset Preparation → Node Authoring → Flow Composition → Self-Check & Handover. Tasks can be large or small: large tasks are broken down hierarchically into sub-flows, which are nested and composed, with each layer following the same steps.
- Modification Track: Modify, refactor, or review existing pipelines in a project — Assessment → Reference Search → Modification → Self-Check (see below). Do not rigidly follow the new creation order.
The precise semantics of fields (field names, parameters, default values, version support)
are not duplicated in this document; all are routed to Section 3.1 Task Pipeline Protocol of
MaaLLMWiki: version locking, reference traceability, and marking as "unverified" when unreachable (the full routing process is handled by the
skill if it is installed). The execution model is the foundation of all reasoning below; a minimal version is provided here:
Execution Model (Minimal Mental Model)
Nodes follow a three-stage structure: Recognition → Action → next. The engine operates between nodes as follows:
- A task is started by an entry node name (): the engine initializes with "current node = entry node, polling list = [entry node]" — the entry node is recognized as the only candidate (usually a DirectHit that always matches); if recognition times out, it uses its own , and if it misses, it follows its own on_error.
- After the current node's action completes, its next list is polled sequentially for recognition: each candidate is tried in order per round; if all candidates miss in a round, the engine waits for before taking a new screenshot and retrying (screenshot is skipped if all candidates in the round are DirectHit, with zero screen overhead).
- The first matching candidate wins immediately (subsequent candidates are no longer checked), the engine enters this node and executes its action — if the action succeeds, it polls the node's next list (returning to step 2); if the action fails, it follows the node's on_error.
- If polling exceeds the current node's (default 20s) with no matches → follow the current node's on_error — to adjust the waiting time or timeout destination for a node being recognized, modify its previous node's / , not its own.
- Exception State:
- Entry (three types): Action failure; polling timeout; no existing and enabled nodes in the polled list (immediate miss without waiting for — non-existent references fail loading validation, and most cases that reach this step are nodes with or unset anchors).
- After entry: The on_error list replaces the next list and is polled in the same way — if on_error is empty, next is immediately empty, the loop terminates directly, and the task ends in failure.
- Resolution: Only resolved by a recognition match; if the action fails after matching, the exception state is re-entered immediately — so effectively, exiting an exception = matching and successful action execution.
- Tightening during exception: A second miss (on_error polling timeout, or no existing and enabled nodes in the on_error list — e.g., a misspelled recovery node name, which fails immediately without waiting for timeout) does not enter on_error, and the task fails directly; if the execution chain runs empty while still in an exception state, registered jumps do not take effect, and the task terminates in failure.
- Termination and Success/Failure:
- The loop terminates when next is empty or an external stop / StopTask is issued; task success/failure = whether it is in an exception state at termination (running empty in normal state means success).
- Only exception: (normal state only): A sub-chain running empty does not terminate; instead, it re-polls the parent node's next list (recognizes sequentially from the beginning; the parent node is restored as the current node without re-recognition, its timeout and on_error ownership return to it, and its action is not re-executed). Example:
A.next=["B","[JumpBack]C"]
, , — after C matches and executes, it proceeds to D; when D runs empty, the next poll is for .
on_error Ownership: Determined by which node is currently active — a candidate being recognized does not count as entering the node; entry only occurs after a match. After entry, action execution and next list polling both happen within the current node, and failures follow the current node's on_error: action failures follow the on_error of the node where the action is located, and next list polling timeouts follow the on_error of the node that owns the next list.
Hard Guardrails (Effective Throughout)
- Exploration First: Every state to be written must have real observational evidence (screenshots, OCR text, color values, bounding boxes, etc.); states inferred from documentation, node names, or memory are all considered guessed, and guessed states must not be written down. Detailed rules:
- Progress by transition: Observe the recognition markers and click targets of whichever interface the exploration has reached (based on the current device screen); after clicking, when the marker of the next interface is actually observed (i.e., the "exit" candidate in next), this transition can be written. Popup handling nodes are also only written after the popup has been seen. Write as you observe, no need to explore the entire flow before writing; full end-to-end connectivity and reentrancy are not thresholds for writing, they are checked during self-check and end-to-end acceptance.
- Minimize click side effects: Do not click unnecessary submit/claim buttons; do not actively trigger states that can be confirmed via keyframes.
- Unreproducible states (one-time rewards, first-run guides, interfaces after irreversible operations) cannot be reached actively; use saved keyframes instead — important change frames temporarily stored by tools also count as observational evidence, and should be checked frame by frame before writing; states without even keyframes are still treated as gaps.
- When reusing existing nodes, their entry prerequisites are also states that must be observed.
- If exploration is impossible (no device, no authorization, would trigger prohibited side effects), stop and report the gap; do not force-write plans.
- Not applicable to real-time scenarios: Millisecond-level reactions such as flash sales or QTEs cannot rely on the "observe → decide" exploration mode, and exceed the reaction speed of the polling model — directly state that it cannot be done, do not force-write.
- Coordinate Hygiene: Click targets are derived from recognition results. defaults to (inherits the current node's recognition box), and most nodes do not need to specify it; use for offsets and note the basis; if a different target is truly needed, prioritize string references (previous node name / , empty reference result means action failure); fixed coordinate arrays are treated as hard coding, only used in special scenarios (e.g., negative coordinates calculated from edges) with reasons noted — + hardcoded target coordinates are strictly prohibited.
- Do not fabricate fields: If unsure about field names, parameters, or version support, route to the documentation; prefer writing less and using default values. Root fields starting with are not parsed, and files starting with are not read. Pipeline files are parsed as jsonc — inline and full-line comments are directly usable; use / for structured comments, / for placeholder hints (e.g., indicates this parameter is uniformly configured in interface.json/Custom code), and similarly / (avoid using if possible) — all are non-protocol fields (not documented at all in the 3.1 protocol). The engine silently ignores unknown keys, and using them as structured comments is a conventional usage; for pure explanations, prefer jsonc comments.
- Base Resolution Neutral: The running base resolution is dynamically obtained from the project's controller configuration (default 720px short edge, compatible with portrait and landscape); screenshots for measuring ROIs must go through the framework's screenshot pipeline, not external images.
- Loops must have termination justification: For any pattern where execution returns to the candidate list of a previously executed node ( jumps, next references forming a loop), choose one of three termination justifications and explain it in comments. Detailed rules:
- Limit count: / + reporting;
- Recognition gating: Candidates have real recognition, and the action aims to eliminate that recognition; if the action may fail, still configure a fallback limit count.
- AFK intent: Explicitly declared infinite production loops, exited via external stop or condition nodes, with real progress per round in the loop, and count resets must be explicit;
- only handles "target interface never appears": It cannot handle "keeps matching, can't stop" — as long as a candidate matches every round, timeout will never trigger;
- DirectHit prohibited in loops: As always-matching with no recognition gating, DirectHit is prohibited as a candidate for self-loops or loop bodies — a candidate list of all DirectHit also skips screenshots, so even screen changes cannot be observed.
New Creation Track
Steps 1–5 are rolled out segment by segment: each broken-out sub-flow follows steps 1→5 independently; explore one segment, write one segment, no need to explore the entire task before writing. After all segments are completed and self-checked, proceed to Step 6.
Rolling progress must be accompanied by a progress file (cross-session working memory, stored as an external file in a location outside the resource pack so it is not loaded by the framework): record requirements and breakdown conclusions (sub-flow list), implementation status of each segment (Planned / Written / Tested), evidence frame number for each observed state, and current breakpoint (which interface the device is on). Keyframes are archived bound to states — they serve as observed evidence, material for re-testing recognition (node test mode: screenshot + hits/box expectations), and source for template cropping.
1. Requirements → Breakdown & State Inventory (Including Exploration & Evidence Collection)
- Break down first, then translate (rolling breakdown): Requirements can be large or small; large tasks are first broken into sub-tasks/sub-flows — each sub-flow has clear entry/exit boundaries (conventionally suffixed with /). Sub-flows reusable across tasks are placed in top-level shared files under in the resource pack (). Reuse boundaries are expressed by file ownership and cross-module references, not by node name prefixes (see references/project-structure.md). Sub-flows are further broken down the same way, nested to any depth. Breakdown boundary: a segment of interface/business that can be independently observed and verified. Breakdown does not need to be completed all at once — first set the overall framework of sub-flow boundaries, then add on-screen states and popups to the inventory as you progress.
- On-screen workflow (same-screen first, then exit): After entering an interface, first write and immediately test the exception handling nodes for that interface — same-screen content stays on screen, so it can be tested right after writing (on-screen breadth). Write exits last — after writing and testing, you are already on the next interface, and can continue seamlessly (inter-screen depth). Reversing the order will leave the current screen first, and unadapted popups will require running the entire flow again to come back and test.
- Translate each (sub-)flow into states rather than action lists: initial state, each intermediate state (including possible popups), success state, failure state. Each state answers two questions:
- What to use for recognition: Fixed image → TemplateMatch; Text → OCR (use complete stable text for ); Color block → ColorMatch; Flow nodes with no visual change → DirectHit; Target size/zoom changes (template match mismatch) → FeatureMatch; Complex targets where template/text/color are not feasible → NeuralNetworkClassify / NeuralNetworkDetect (self-trained ONNX model); Multiple conditions on the same screen → And / Or; None of the above can express → Custom (hand over to ). Check the full list and precise parameters against the 3.1 Task Pipeline Protocol via routing.
- Is the evidence observed or guessed: For whichever segment is being written, all states in that segment must be observed (Hard Guardrail 1); segments not yet explored are left to be explored when writing them.
2. File Structure & Naming
Determine placement first, then write content:
- New features are split into files/directories by module; choose one of two file organization styles and stick with it: Module Prefix Style (
AutoCollect/AutoCollectRoute1.json
— file names have module prefixes, paired with directories when there are many modules, clustered by sorting) or Flat Feature Style ( laid flat in the root directory, lighter for moderate numbers of features). Both are proven in large projects. New defaults: Module Prefix Style + PascalCase file names (consistent case with nodes/template images, directories from day one, no growth ceiling). Existing flat projects continue as before, and incremental new modules can use directories; see references/naming.md for details.
- Pipeline JSON has two equivalent writing styles: v1 Flat and v2 Nested ( / are consolidated into , v4.4.0+), with universal field semantics. New defaults: v2 (separate namespaces for recognition/action/flow parameters, misaligned fields cannot occur structurally). Existing projects follow the prevailing style, no need for full-repo conversion, just consistency within the project (see example notes in references/flow-patterns.md for basis).
- Node names are resolved globally across files and are case-sensitive; duplicate names within the same resource pack are errors (duplicate names across packs are a channel override mechanism, see references/naming.md). Do not start with ; square brackets ( ) are attribute syntax for next / on_error reference positions (entries starting with are parsed as attribute prefixes), and node names conventionally do not use them. Entry/exit nodes use / suffixes (convention). Node name semantics are organized by a word-position formula (
Task_Interface_Object_Role_Variant
, slots separated by underscores, see references/naming.md). Reuse boundaries are not expressed via node name prefixes.
- Images are divided into directories by module; asset paths are the reference keys. Renaming must synchronously update pipeline references and image-loading calls in custom code (custom recognition/actions can load images on their own).
- Repository layout and multi-resource packs see references/project-structure.md; folder names under the resource directory must not start with (Android packaging accident).
3. Asset Preparation
Derive asset gaps from the state inventory, fill them before writing nodes:
- Templates are cropped from framework screenshots at base resolution (after lossless scaling of the original image), with the cropping source and recognition input maintaining the same image space. External images / in-game screenshots are not universally prohibited; the criterion is pixel space consistency: Emulator built-in screenshots are usable if their resolution matches the controller's base (or aspect ratio matches and can be losslessly scaled to the base) and there is no occlusion or pollution (notification bar / tool watermark / lossy compression), as they share the same source and space as framework screenshots. Camera photos, mismatched proportions, or polluted images are not usable. Regardless of source, all templates must undergo post-cropping self-match verification as a fallback.
- Choose tools for asset acquisition based on workflow; hard requirements do not change with tools (pixel space consistency + post-cropping self-match verification, see previous item). Agent autonomous workflows prioritize real-time observation bases (e.g., maafw-live) — frame stream is resident, cropping and color picking are fully programmatic, no need for human screen monitoring: templates are cropped from saved original-resolution frames (, same recognition space), color sampling (), point selection via candidate region numbering when no template exists (), and unreproducible states are cropped from saved frames in the keyframe library (including post-hoc offline , no device needed; outputs carry L2 provenance). The workflow and criteria for this tool are in its built-in skill (released with the package, not repeated in this collection):
npx skills add https://github.com/Windsland52/maafw-live --skill maafw-live --global
. Without an observation base, you can also write your own script to take screenshots via the framework's screenshot pipeline and then crop programmatically; for on-site manual debugging, use official toolchains (VSCode maa-support plugin, MFA Toolbox, MPE, etc.) for interactive capture.
- Post-cropping self-match verification: A new template should match at its source position on its source frame (correct position takes priority over score; solid-color template scores are noise) — verify immediately after cropping, don't wait for flow integration testing to discover asset issues.
- Choose stable features for cropping areas, avoid dynamic numbers/animations — dynamic content causes different pixels per capture of the same state, making template matching unstable.
4. Node Authoring
One node per state, recognition first, then action:
- Recognition fields are set based on the type selected in the state inventory; is tightened to the area where the target actually appears (speeds up and prevents false matches). The starting point can be the recognition box expanded by about 20px — too tight easily causes mismatch, too loose easily causes false matches; scan several levels if unstable.
- Action types are selected based on intent; check the full list and precise parameters against the 3.1 Task Pipeline Protocol via routing: Click / , Swipe / / , Key / / / , Input , App start/stop / , Flow / , System capabilities / / , Extension . Click positions follow coordinate hygiene (Hard Guardrail 2).
- To display node messages to users (runtime logs / toast notifications / system notifications / popups), configure the node's : provide templates by message type ( and other placeholders are replaced, supports i18n), select display channel with ; telemetry reporting only takes effect if is configured in the project's interface.json — field definitions are routed to 3.1, template and channel mechanisms are routed to Section "Node Notification Processing" of the 3.3 protocol.
- Parameter starting points: is based on actual measurement (score margin rules only apply to TemplateMatch; OCR / ColorMatch have different semantics; FeatureMatch has no threshold, and strictness is controlled by minimum feature points and KNN distance ratio) — the starting point is the measured score at the target position minus about 0.1 (it will drop when changing scenes), and verify it is still higher than the highest false match score on that screen. If the two sides are too close, it's a template/roi problem that adjusting the threshold can't fix; use to wait for stability instead of adding delay if possible; write fewer uncertain parameters and use default values (Hard Guardrail 3).
- Writing discipline: Insert incrementally instead of rewriting the entire file — JSONC comments carry the "why it's written this way", and full rewrites will destroy them; indentation and field order follow the project's existing conventions (often with formatting plugins); all coordinates use measured values in the framework's scaled coordinate system.
5. Flow Composition
- Main Trunk: Arrange the shortest successful path as a linear next chain. Detailed rules:
- Wait for target to appear: Use "wait for target to appear" nodes for interface transitions, don't make do with long delays;
- Keep parameters default: / / node-level (next polling timeout) should be kept default as much as possible;
- Exception for of wait_freezes: Only use / when you must wait for the screen to stabilize; their own is not in the "keep default" category — default is 20s, and if the waiting area has continuous animation, it will wait the full time before continuing, often paired with a small 1~2s fallback value.
- Branches: The order of the next list is the priority, and the first match executes. Two sorting principles:
- Occlusion constraint first: When one candidate appears, another candidate will definitely match, the former must be ranked first; if reversed, the latter will never be reached (functional disorder). If the occluded one is exactly the exit, and the occluder's action cannot eliminate its own trigger condition, it forms an "always matching" infinite loop that can't fix (this is the prevention side of Hard Guardrail 5);
- Match frequency descending: The remaining candidates are sorted in descending order of frequency; each round recognizes sequentially, with high-frequency ones first, reducing average recognition per round and optimizing global resource consumption;
- Expression methods: Mutually exclusive branches are expressed by order; use for sorting by position/score, and / for combined conditions.
- Loops and Interrupts. Detailed rules:
- Loops: Loop body + exit node are placed in the same list, limits the count to prevent dead loops (see Hard Guardrail 5 for three termination justification options);
- Popup / Loading Handlers: Interfaces that may pop up at any time (announcements, rewards, disconnections) are made into independent handling nodes, attached to key waiting points with (jumps only take effect in normal state; if the chain runs empty in exception state, the task fails);
- Use for cross-branch convergence — references to unset anchors will be silently skipped.
- Waiting and Retry Layering. Detailed rules:
- Waiting is expressed with parameters, not structure: is already waiting by itself, the rhythm is adjusted by / / , self-referencing next has zero benefit for "waiting for the next screen to appear";
- Retries require evidence: Action retries are only added after telemetry shows a real failure rate;
- Self-reference retry is preferred: For linear flows, a single node self-reference + its own is sufficient (budget includes first entry, accumulated across all reference points), no need to create clone nodes;
- Cloning thresholds and discipline: Only clone when independent retry budget or strategy upgrade is truly needed (wait before retry, long press instead of single click); use / suffixes with the same prefix as the source to remain searchable; desc notes which node it is cloned from and which fields are intentionally different from the source (diff surface) — fields outside the diff surface that are inconsistent with the source are drift bugs, and when modifying the source node, fields outside the diff surface must be synchronized;
- Shared handler exception: Nodes referenced by multiple places cannot use small values under the cumulative semantics; the defense is strict recognition gating. Candidates whose is exhausted exit silently (skipped during recognition, no error), and the list is polled as usual — when there are no more matching candidates, it idles until timeout and follows the host node's on_error; exhaustion itself is not a failure signal;
- Review baseline: Unjustified self-references / self-loops are treated as items to be deleted during review.
- Failure Paths: Ask "what if all miss?" for every next list. Detailed rules:
- Fallback structure: (use infinite with caution, v5.5+) + fallback (rollback/retry/report), retries paired with limit;
- The on_error chain itself must be able to match stably — there is no second fallback if it fails again in exception state, and the task terminates directly;
- on_error is only for exception recovery: Do not use it as control flow for normal processes (e.g., to implement if-else effects);
- Retries and are seatbelts, not fixes: For repeated failures, fix the root cause first (unstable recognition / logical defects);
- Blind retries without recognition gating are prohibited: "Click again in the same spot if there's no response" (the second click may land on other elements of the next interface); limited retries where the button is re-recognized as still present before clicking again are legitimate recovery methods;
- Action failures go to convergence: Action failures (framework errors) are not retried via self-reference or self-loops — persistent action failures are defects, not jitters; let on_error converge (report / rollback / re-enter sub-flow entry and re-run recognition), and transient recovery must be explicitly bounded.
Pattern example JSON and stability rules (recognize → operate → re-recognize, use less hard delays) see references/flow-patterns.md.
6. Self-Check & Handover
- Reachability: No isolated nodes, no broken anchor chains, no unresolved references; Four completeness questions — Can the main trunk run? Can popups be handled? Can loading be waited through? Is the behavior when not in the target scenario explicitly defined (auto-skip or failure report, this is a product decision for the project, cannot be omitted by default)?
- For projects with node testing infrastructure (e.g., with directory and screenshot expectations), add test cases for new recognition nodes.
- Handover: Single-node verification and end-to-end acceptance → ; task entry and option wiring → ; logic that cannot be expressed in JSON → .
Modification Track (Legacy Project Entry)
Use this track when modifying, refactoring, or reviewing existing pipelines:
- Assessment (scale scales with changes, use indexes instead of reading everything). Detailed rules:
- Minor fixes (bugs, field adjustments, ROI fine-tuning): Just locate the target node, its definition, and its direct upstream and downstream;
- Structural changes (new features, refactoring, changes to reuse surfaces) require assessing the reuse surface — first look at the map (directory tree + interface.json task list) → extract node name indexes with scripts (list root keys per file, don't read through each file) → search by capability keywords, read full text only for matched files (semantic names from word-position naming are search indexes, check first for shared items);
- Assets: Search as needed according to the directory structure;
- Regardless of scale: First clarify the change boundaries (which nodes are modified, which are only referenced).
- Search References: Search the entire repository for reference points before renaming nodes or asset paths — the consequences of missed changes are graded by reference location, not all are silent: Missed changes in next / on_error fail at load time (existence checks for non-anchor references can catch them; dangling references injected by interface's pipeline_override are also blocked); Missed changes in roi / target references, custom's run_task, template asset paths fail at runtime (empty target resolution means action failure / task does not exist / template lazy loading missing image); Only entries (anchor names skip validation) and option's pipeline_override (override misses) are silent disconnections. Reference point list:
- Node references: Three writing styles of next / on_error (bare string, prefix, object), string references in roi / target, , option's pipeline_override, custom's run_task / override_next;
- Asset paths: template in pipeline and image-loading calls in custom code (custom recognition/actions can load images under on their own);
- Same-prefix clones: Before modifying recognition / action parameters, also search for / suffixes — clones and sources drift silently, also without errors;
- Cross-language references: Node names written in custom / agent code (any language — C++ / Python / Node.js / Go, etc.) are divided into two categories by entry: entries for / / look up nodes by raw string; passing will only look for a node literally named that and fail; for anchor jumps, first use to get the real name then pass it in. The next list of is parsed the same as pipeline reference positions, and / attribute syntax can be used. Anchors themselves can be read and written via the context API ( / , exposed in C API / agent RPC / various bindings); writing real names + searching the entire repo for references before renaming is still the baseline defense.
- Modification: Write new content according to the same standards as steps 2–5 of the New Creation Track as needed.
- Self-Check: Reachability check + item-by-item verification against the anti-pattern list (references/anti-patterns.md — coordinate hygiene, timeout/-1 abuse, loop limit counts, version compatibility).
Further Reading (As Needed)
- references/naming.md — Full naming specifications for files/directories/nodes/images (protocol layer + two schools of practice)
- references/project-structure.md — Repository layout, multi-resource packs, task entry storage, agent organization
- references/flow-patterns.md — Flow design expansion, pattern example JSON, stability rules
- references/anti-patterns.md — Coordinate hygiene details, anti-pattern list, review checklist