Back to Details

ad-graph-dump

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

AutoDeploy: Graph dumps (
AD_DUMP_GRAPHS_DIR
)

AutoDeploy：图形转储（
AD_DUMP_GRAPHS_DIR
）

Where this skill applies

本技能适用场景

This file is part of trtllm-agent-toolkit. Commands and paths such as 
examples/auto_deploy/
and 
tensorrt_llm/
are relative to a TensorRT-LLM source checkout, not the plugin repository.
本文件是trtllm-agent-toolkit的一部分。
examples/auto_deploy/
和
tensorrt_llm/
等命令与路径是相对于TensorRT-LLM源代码检出目录的，而非插件仓库。

When to use this skill

何时使用本技能

  • You need to see how the FX graph changes after each registered transform runs.
  • You are verifying that a subgraph exists, that a fusion matched, or that metadata / wrappers (
    getitem
    , 
    view
    , 
    reshape
    ) appeared or disappeared between dumps.
  • You are pairing log output with on-disk graph files while debugging AutoDeploy.
  • 你需要查看每个已注册转换操作执行后，FX图形发生的变化。
  • 你需要验证子图形是否存在、融合操作是否匹配，或元数据/包装器（
    getitem
    、
    view
    、
    reshape
    ）在转储前后是否出现或消失。
  • 调试AutoDeploy时，你需要将日志输出与磁盘上的图形文件配对分析。

Related skills in this plugin

本插件中的相关技能

SkillUse it for
ad-layer-visualizerExtracting and visualizing a single decoder layer from a dump as a DOT/PNG diagram.
ad-add-fusion-transformationImplementing or reviewing fusion passes once you know what the graphs show.
trtllm-codebase-explorationSearching the TRT-LLM tree for transforms, custom ops, and patterns.
trtllm-code-contributionTests and contribution hygiene after you change TRT-LLM.
技能适用场景
ad-layer-visualizer从转储中提取单个解码器层并将其可视化为DOT/PNG图。
ad-add-fusion-transformation了解图形显示内容后，实现或审核融合处理流程。
trtllm-codebase-exploration在TRT-LLM代码树中搜索转换操作、自定义算子和模式。
trtllm-code-contribution修改TRT-LLM后的测试与代码贡献规范检查。

Environment variable

环境变量

Set:
bash
export AD_DUMP_GRAPHS_DIR=/path/to/output/dir
Implementation: 
GraphWriter.DUMP_GRAPHS_ENV == "AD_DUMP_GRAPHS_DIR"
in 
tensorrt_llm/_torch/auto_deploy/utils/graph_writer.py
.
If unset, no graph files are written.
设置：
bash
export AD_DUMP_GRAPHS_DIR=/path/to/output/dir
实现位置：
tensorrt_llm/_torch/auto_deploy/utils/graph_writer.py
中的
GraphWriter.DUMP_GRAPHS_ENV == "AD_DUMP_GRAPHS_DIR"
。
若未设置该变量，则不会生成任何图形文件。

When dumps are produced

转储生成时机

After each transform application, 
BaseTransform
calls 
graph_writer.dump_graph(mod, t_name, self.config.stage.value)
from 
tensorrt_llm/_torch/auto_deploy/transform/interface.py
(immediately after 
_visualize_graph
). So the dump reflects the module after that transform has run.
每次执行转换操作后，
BaseTransform
会调用
tensorrt_llm/_torch/auto_deploy/transform/interface.py
中的
graph_writer.dump_graph(mod, t_name, self.config.stage.value)
（紧跟在
_visualize_graph
之后）。因此转储内容反映的是该转换操作执行后的模块状态。

Rank / process behavior

Rank/进程行为

From 
GraphWriter.dump_graph
:
  • Dumps run only when 
    AD_DUMP_GRAPHS_DIR
    is set.
  • If 
    ADLogger.rank
    is set and is not 
    0
    , dumping is skipped (non–rank-0 processes do not write files).
根据
GraphWriter.dump_graph
的逻辑：
  • 仅当设置了
    AD_DUMP_GRAPHS_DIR
    时才会生成转储。
  • 若
    ADLogger.rank
    已设置且不为0，则会跳过转储（非rank-0进程不会写入文件）。

Directory lifecycle

目录生命周期

On the first dump on rank 0, 
GraphWriter
removes the target directory if it already exists, then recreates it. Do not point 
AD_DUMP_GRAPHS_DIR
at a directory that must be preserved without copying it first.
当rank-0进程首次生成转储时，
GraphWriter
会先删除目标目录（若已存在），然后重新创建。请勿将
AD_DUMP_GRAPHS_DIR
指向需要保留的目录，除非先进行备份。

File naming and ordering

文件命名与排序

Files are named:
text
{NNN}_{<stage.value>}_{<transform_key>}.txt
  • NNN
    is a monotonically increasing three-digit counter (001, 002, …) in run order across all dumps in that process.
  • The middle segment is each transform’s 
    config.stage
    value (same enum/string used in 
    default.yaml
    under each transform’s 
    stage:
    field).
  • The last segment is the transform’s registry key (
    transform_name
    passed into 
    dump_graph
    ).
So lexicographic sort by filename matches pipeline order for that run.
文件命名格式如下：
text
{NNN}_{<stage.value>}_{<transform_key>}.txt
  • NNN
    是一个单调递增的三位计数器（001、002……），代表该进程中所有转储的执行顺序。
  • 中间段是每个转换操作的
    config.stage
    值（与
    default.yaml
    中每个转换操作下
    stage:
    字段使用的枚举/字符串一致）。
  • 最后一段是转换操作的注册表键（传入
    dump_graph
    的
    transform_name
    ）。
因此，按文件名的字典序排序即可匹配本次运行的流水线顺序。

File contents

文件内容

Each file is text and starts with headers similar to:
text
undefined
每个文件均为文本格式，开头包含类似以下的头部信息：
text
undefined

Transform: <transform_key>

Transform: <transform_key>

Stage: <stage.value>

Stage: <stage.value>

GraphModules found: <count>

GraphModules found: <count>


Then, for every `torch.fx.GraphModule` found under `mod.named_modules()` (including the root), the writer emits a section title and an SSA-style listing with shape/dtype metadata via `dump_ssa_with_meta()` in the same module.

Use this to compare operator chains, consumers, and `node.meta` shape/dtype hints across consecutive files.

随后，对于`mod.named_modules()`下找到的所有`torch.fx.GraphModule`（包括根模块），写入器会生成章节标题，并通过同一模块中的`dump_ssa_with_meta()`输出带有形状/数据类型元数据的SSA格式列表。

你可以利用此内容比较连续文件中的算子链、消费者以及`node.meta`形状/数据类型提示。

Example: capture dumps from a registry build

示例：从注册表构建中捕获转储

From the root of the TensorRT-LLM clone (adjust the script and flags to your workflow):
bash
AD_DUMP_GRAPHS_DIR=/tmp/ad-graphs \
  python examples/auto_deploy/build_and_run_ad.py --model <hf-model-id> --use-registry
Pick any AutoDeploy entrypoint you already use; the requirement is only that the code path runs the transform pipeline with 
AD_DUMP_GRAPHS_DIR
set in the environment.
在TensorRT-LLM克隆目录的根目录下执行（根据你的工作流程调整脚本和参数）：
bash
AD_DUMP_GRAPHS_DIR=/tmp/ad-graphs \
  python examples/auto_deploy/build_and_run_ad.py --model <hf-model-id> --use-registry
你可以使用任何已在使用的AutoDeploy入口点；唯一要求是代码路径在设置了
AD_DUMP_GRAPHS_DIR
环境变量的情况下运行转换流水线。

Logs vs dump files

日志与转储文件

While a transform runs, logging is patched so messages can be prefixed with 
[stage=<stage.value>, transform=<transform_key>]
(see 
with_transform_logging
in 
transform/interface.py
). Transform summaries log 
[SUMMARY]
with 
matches=<n>
or 
skipped
/ 
disabled
(
_log_transform_summary
). Use those lines together with the numbered dump files to tie match counts to graph shape before and after a specific transform.
转换操作运行时，日志会被修补，以便消息前缀可以添加
[stage=<stage.value>, transform=<transform_key>]
（参见
transform/interface.py
中的
with_transform_logging
）。转换摘要日志会带有
[SUMMARY]
标记，包含
matches=<n>
或
skipped
/
disabled
信息（来自
_log_transform_summary
）。你可以将这些日志行与带编号的转储文件结合，将匹配计数与特定转换操作前后的图形形状关联起来。

Pitfalls

注意事项

  • Stale directory: Because the dump dir is deleted on first use, a second run in the same shell without changing 
    AD_DUMP_GRAPHS_DIR
    overwrites prior output.
  • No GraphModules: If the module has no 
    GraphModule
    children, 
    dump_graph
    returns without creating a new file for that step (see early return in 
    graph_writer.py
    ).
  • Distributed: Only rank 0 writes; other ranks skip silently.
  • 目录覆盖：由于首次使用时会删除转储目录，若在同一shell中不修改
    AD_DUMP_GRAPHS_DIR
    就再次运行，会覆盖之前的输出。
  • 无GraphModule：如果模块没有
    GraphModule
    子模块，
    dump_graph
    会提前返回，不会为该步骤创建新文件（参见
    graph_writer.py
    中的提前返回逻辑）。
  • 分布式场景：仅rank-0进程会写入文件，其他进程会静默跳过。

Source references

源码参考

  • tensorrt_llm/_torch/auto_deploy/utils/graph_writer.py
    — env var, filenames, SSA dump.
  • tensorrt_llm/_torch/auto_deploy/transform/interface.py
    — call site after each transform; log prefix decorator.
  • tensorrt_llm/_torch/auto_deploy/utils/graph_writer.py
    —— 环境变量、文件名、SSA转储逻辑。
  • tensorrt_llm/_torch/auto_deploy/transform/interface.py
    —— 每次转换操作后的调用点；日志前缀装饰器。