Back to Details

dagster-expert

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Dagster Expert Skill

Dagster专业技能指南

Expert guidance for building production-quality Dagster projects. Routes you to detailed reference documentation for assets, automation, project structure, and CLI commands.
IMPORTANT: For new assets, schedules, or sensors, ALWAYS use 
dg scaffold
before manual file creation. See cli/scaffold.md.
Invoke EARLY: If the task involves automation conditions, scheduling logic, or integration components (dbt, Fivetran, etc.), invoke this skill or the appropriate sub-skill BEFORE exploring the codebase. The references contain all needed patterns.
为构建生产级Dagster项目提供专业指导，引导你查阅关于asset、自动化、项目结构及CLI命令的详细参考文档。
重要提示：如需创建新的asset、schedule或sensor，手动创建文件前务必使用
dg scaffold
命令。请查看cli/scaffold.md。
尽早调用：如果任务涉及自动化条件、调度逻辑或集成组件（dbt、Fivetran等），在查看代码库前请先调用本技能或对应的子技能。参考文档包含所有所需的实现模式。

Task Router

任务路由

What do you need?Reference
Create a projectQuick Reference below (use 
uvx create-dagster project
)
Add an asset/schedule/sensorcli/scaffold.md (ALWAYS scaffold first) + pattern docs
Add integration component (dbt, Fivetran, Airbyte, Snowflake, etc.)Invoke dagster-integrations skill (contains scaffolding, YAML schema, adapter requirements)
Complex scheduling / different triggers per dependencyautomation/declarative-automation/README.md
Run/materialize assetscli/launch.md
Select specific assetscli/asset-selection.md
List definitionscli/list.md
Validate projectcli/check.md
Understand asset patternsassets.md
Understand project layoutproject-structure.md
Set up automationautomation/README.md
Debug/get logscli/api.md
Configure env varsenv-vars.md
Implementation workflowimplementation-workflow.md
你需要什么帮助？参考文档
创建项目下方快速参考（使用
uvx create-dagster project
命令）
添加asset/schedule/sensorcli/scaffold.md（务必先使用脚手架） + 模式文档
添加集成组件（dbt、Fivetran、Airbyte、Snowflake等）直接调用dagster-integrations技能（包含脚手架、YAML schema、适配器要求）
复杂调度 / 基于依赖的不同触发器automation/declarative-automation/README.md
运行/物化assetcli/launch.md
选择特定assetcli/asset-selection.md
列出定义cli/list.md
验证项目cli/check.md
理解asset模式assets.md
理解项目布局project-structure.md
设置自动化automation/README.md
调试/获取日志cli/api.md
配置环境变量env-vars.md
实施工作流implementation-workflow.md

Keyword Router

关键词路由

KeywordsReference
create project, new project, init, setupQuick Reference below
workspace, multi-project, multiple projectsQuick Reference below
scaffold, generate, create asset/schedule/sensorcli/scaffold.md
dbt, fivetran, airbyte, snowflake, bigquery, external tool, integration component, dagster_dbt, dagster_fivetranInvoke dagster-integrations skill directly
complex triggers, different triggers, hot/cold dependencies, conditional automationautomation/declarative-automation/README.md
automation_condition, AutomationCondition, eager(), any_downstream_conditionsautomation/declarative-automation/README.md
list, show, find, discover, what assetscli/list.md
validate, check, verify, test configcli/check.md
launch, run, materialize, execute, backfillcli/launch.md
select, filter, tag, group, kind, upstream, downstreamcli/asset-selection.md
logs, debug, troubleshoot runcli/api.md
deploy, plus, cloudcli/api.md
asset, dependency, metadata, partitionassets.md
schedule, cron, time-basedautomation/schedules.md
sensor, event-driven, triggerautomation/sensors/
declarative automation, conditionsautomation/declarative-automation/
project structure, code location, definitionsproject-structure.md
environment variables, env, configenv-vars.md
关键词参考文档
创建项目, 新项目, 初始化, 搭建下方快速参考
工作区, 多项目, 多个项目下方快速参考
脚手架, 生成, 创建asset/schedule/sensorcli/scaffold.md
dbt, fivetran, airbyte, snowflake, bigquery, 外部工具, 集成组件, dagster_dbt, dagster_fivetran直接调用dagster-integrations技能
复杂触发器, 不同触发器, 冷热依赖, 条件自动化automation/declarative-automation/README.md
automation_condition, AutomationCondition, eager(), any_downstream_conditionsautomation/declarative-automation/README.md
列出, 展示, 查找, 发现, 资产列表cli/list.md
验证, 检查, 校验, 测试配置cli/check.md
启动, 运行, 物化, 执行, 回填cli/launch.md
选择, 过滤, 标签, 分组, 类型, 上游, 下游cli/asset-selection.md
日志, 调试, 运行故障排查cli/api.md
部署, plus, 云cli/api.md
asset, 依赖, 元数据, 分区assets.md
schedule, cron, 基于时间automation/schedules.md
sensor, 事件驱动, 触发器automation/sensors/
声明式自动化, 条件automation/declarative-automation/
项目结构, 代码位置, 定义project-structure.md
环境变量, env, 配置env-vars.md

Quick Reference

快速参考

bash
undefined
bash
undefined

Project Setup

项目搭建

uvx create-dagster project <name> --uv-sync # --uv-sync creates venv and installs deps uvx create-dagster workspace <name> # For multiple related projects
uvx create-dagster project <name> --uv-sync # --uv-sync 参数用于创建虚拟环境并安装依赖 uvx create-dagster workspace <name> # 用于创建多个关联项目的工作区

Output confirms success—no verification needed

输出信息确认创建成功后无需额外验证

Scaffold (ALWAYS use for new definitions)

脚手架（创建新定义时务必使用）

NOTE: Paths are RELATIVE TO defs/ directory, not project root

注意：路径是相对于defs/目录，而非项目根目录

dg scaffold defs dagster.asset assets/my_asset.py dg scaffold defs dagster.schedule schedules/daily.py dg scaffold defs dagster.sensor sensors/file_watcher.py
dg scaffold defs dagster.asset assets/my_asset.py dg scaffold defs dagster.schedule schedules/daily.py dg scaffold defs dagster.sensor sensors/file_watcher.py

Discover

查看资源

dg list defs dg list defs --assets "group:analytics" dg list components
dg list defs dg list defs --assets "group:analytics" dg list components

Validate

验证项目

dg check defs
dg check defs

Launch

启动任务

dg launch --assets "tag:priority=high" dg launch --assets "+my_asset" # with upstream dg launch --assets my_asset --partition 2024-01-15
undefined
dg launch --assets "tag:priority=high" dg launch --assets "+my_asset" # 包含上游依赖 dg launch --assets my_asset --partition 2024-01-15
undefined

Core Dagster Concepts

核心Dagster概念

Brief definitions only (see reference files for detailed examples):
  • Asset: Persistent object (table, file, model) produced by your pipeline
  • Job: Selection of assets to execute together
  • Schedule: Time-based automation using cron
  • Sensor: Event-driven automation that watches for changes
  • Declarative Automation: Modern automation where you set conditions on assets
  • Partition: Logical division of data (by date, category)
  • Component: Reusable, declarative building block (YAML-based)
仅提供简要定义（详细示例请查看参考文档）：
  • Asset：由数据管道生成的持久化对象（如表格、文件、模型）
  • Job：选择要一起执行的asset集合
  • Schedule：基于时间的自动化，使用cron表达式
  • Sensor：事件驱动的自动化，用于监控变化
  • Declarative Automation：现代化自动化方式，可设置asset的触发条件
  • Partition：数据的逻辑划分（按日期、类别等）
  • Component：可复用的声明式构建模块（基于YAML）

UV Compatibility

UV兼容性

Projects use 
uv
for dependency management:
bash
uv run dg list defs
uv run dg launch --assets my_asset
项目使用
uv
进行依赖管理：
bash
uv run dg list defs
uv run dg launch --assets my_asset

Related Skills

相关技能

  • dagster-integrations - Find and configure integration components (dbt, Fivetran, Sling, etc.)
  • dagster-integrations - 查找并配置集成组件（dbt、Fivetran、Sling等）",