Loading...
Loading...
Create and configure Databricks Asset Bundles (DABs) with best practices for multi-environment deployments. Use when working with: (1) Creating new DAB projects, (2) Adding resources (dashboards, pipelines, jobs, alerts), (3) Configuring multi-environment deployments, (4) Setting up permissions, (5) Deploying or running bundle resources
npx skill4agent add databricks-solutions/ai-dev-kit asset-bundlesproject/
├── databricks.yml # Main config + targets
├── resources/*.yml # Resource definitions
└── src/ # Code/dashboard filesbundle:
name: project-name
include:
- resources/*.yml
variables:
catalog:
default: "default_catalog"
schema:
default: "default_schema"
warehouse_id:
lookup:
warehouse: "Shared SQL Warehouse"
targets:
dev:
default: true
mode: development
workspace:
profile: dev-profile
variables:
catalog: "dev_catalog"
schema: "dev_schema"
prod:
mode: production
workspace:
profile: prod-profile
variables:
catalog: "prod_catalog"
schema: "prod_schema"resources:
dashboards:
dashboard_name:
display_name: "[${bundle.target}] Dashboard Title"
file_path: ../src/dashboards/dashboard.lvdash.json # Relative to resources/
warehouse_id: ${var.warehouse_id}
dataset_catalog: ${var.catalog} # Default catalog used by all datasets in the dashboard if not otherwise specified in the query
dataset_schema: ${var.schema} # Default schema used by all datasets in the dashboard if not otherwise specified in the query
permissions:
- level: CAN_RUN
group_name: "users"CAN_READCAN_RUNCAN_EDITCAN_MANAGEresources:
jobs:
job_name:
name: "[${bundle.target}] Job Name"
tasks:
- task_key: "main_task"
notebook_task:
notebook_path: ../src/notebooks/main.py # Relative to resources/
new_cluster:
spark_version: "13.3.x-scala2.12"
node_type_id: "i3.xlarge"
num_workers: 2
schedule:
quartz_cron_expression: "0 0 9 * * ?"
timezone_id: "America/Los_Angeles"
permissions:
- level: CAN_VIEW
group_name: "users"CAN_VIEWCAN_MANAGE_RUNCAN_MANAGE| File Location | Path Format | Example |
|---|---|---|
| | |
| | |
resources/../databricks.yml./resources:
volumes:
my_volume:
catalog_name: ${var.catalog}
schema_name: ${var.schema}
name: "volume_name"
volume_type: "MANAGED"grantspermissionsapp.yaml# Generate bundle config from existing CLI-deployed app
databricks bundle generate app --existing-app-name my-app --key my_app --profile DEFAULT
# This creates:
# - resources/my_app.app.yml (minimal resource definition)
# - src/app/ (downloaded source files including app.yaml)resources:
apps:
my_app:
name: my-app-${bundle.target} # Environment-specific naming
description: "My application"
source_code_path: ../src/app # Relative to resources/ dircommand:
- "python"
- "dash_app.py"
env:
- name: USE_MOCK_BACKEND
value: "false"
- name: DATABRICKS_WAREHOUSE_ID
value: "your-warehouse-id"
- name: DATABRICKS_CATALOG
value: "main"
- name: DATABRICKS_SCHEMA
value: "my_schema"bundle:
name: my-bundle
include:
- resources/*.yml
variables:
warehouse_id:
default: "default-warehouse-id"
targets:
dev:
default: true
mode: development
workspace:
profile: dev-profile
variables:
warehouse_id: "dev-warehouse-id"| Aspect | Apps | Other Resources |
|---|---|---|
| Environment vars | In | In databricks.yml or resource file |
| Configuration | Minimal (name, description, path) | Extensive (tasks, clusters, etc.) |
| Source path | Points to app directory | Points to specific files |
source_code_path: ..databricks bundle schemadatabricks bundle validate # Validate default target
databricks bundle validate -t prod # Validate specific targetdatabricks bundle deploy # Deploy to default target
databricks bundle deploy -t prod # Deploy to specific target
databricks bundle deploy --auto-approve # Skip confirmation prompts
databricks bundle deploy --force # Force overwrite remote changesdatabricks bundle run resource_name # Run a pipeline or job
databricks bundle run pipeline_name -t prod # Run in specific environment
# Apps require bundle run to start after deployment
databricks bundle run app_resource_key -t dev # Start/deploy the app# View logs for deployed apps
databricks apps logs <app-name> --profile <profile-name>
# Examples:
databricks apps logs my-dash-app-dev -p DEFAULT
databricks apps logs my-streamlit-app-prod -p DEFAULT[SYSTEM][APP]Deployment successfulApp started successfullyInitialized real backendError:Requirements installeddatabricks bundle destroy -t dev
databricks bundle destroy -t prod --auto-approve| Issue | Solution |
|---|---|
| App deployment fails | Check logs: |
| App not connecting to Unity Catalog | Check logs for backend connection errors; verify warehouse ID and permissions |
| Wrong permission level | Dashboards: CAN_READ/RUN/EDIT/MANAGE; Jobs: CAN_VIEW/MANAGE_RUN/MANAGE |
| Path resolution fails | Use |
| Catalog doesn't exist | Create catalog first or update variable |
| "admins" group error on jobs | Cannot modify admins permissions on jobs |
| Volume permissions | Use |
| Hardcoded catalog in dashboard | Use dataset_catalog parameter (CLI v0.281.0+), create environment-specific files, or parameterize JSON |
| App not starting after deploy | Apps require |
| App env vars not working | Environment variables go in |
| Wrong app source path | Use |
| Debugging any app issue | First step: |
../src/./src/developmentproduction"users"