Declarative Automation Bundles & the Databricks CLI
Databricks · Notebooks & Development
Theory
Declarative Automation Bundles (formerly Databricks Asset Bundles, or DABs) describe Databricks resources as code. A bundle is a folder with a databricks.yml file that lists resources such as jobs, pipelines, dashboards and apps, plus targets like dev and prod that point to different workspaces or settings. The source code (notebooks, Python files, SQL) lives next to it in the same Git repository.
The Databricks CLI works with bundles: databricks bundle init creates a project from a template (for example default-python), databricks bundle validate checks the YAML, databricks bundle deploy --target dev uploads the files and creates or updates the resources, and databricks bundle run runs a job. In development mode, deployed resources get a [dev yourname] prefix so they do not clash with teammates. For example, a CI pipeline runs databricks bundle deploy --target prod after every merge to main.
Why we use it
Clicking jobs together in the UI is not repeatable and cannot be reviewed. Bundles make jobs and pipelines version-controlled, testable and deployable to many environments with one command, which is the basis of CI/CD on Databricks.
When to use it
Use bundles for any job or pipeline that goes to production or is shared by a team. Use the UI for quick experiments. Use the plain CLI or REST API for one-off admin tasks that are not part of a project.
How it works
- databricks bundle init from a template
- Edit databricks.yml and the source code
- databricks bundle validate
- Is validation clean? Yes: databricks bundle deploy --target dev. No: Fix the YAML and validate again.
- databricks bundle run --target dev orders_job
- CI deploys with --target prod after merge
Syntax
databricks auth login --host <workspace-url>
databricks bundle init [default-python]
databricks bundle validate
databricks bundle deploy --target dev|prod
databricks bundle run --target dev <job_key>
databricks bundle destroy --target devExample
# databricks.yml
bundle:
name: orders_project
resources:
jobs:
orders_job:
name: orders_job
tasks:
- task_key: load_orders
notebook_task:
notebook_path: ./src/orders_etl.ipynb
targets:
dev:
mode: development
default: true
workspace:
host: https://techtown-dev.cloud.databricks.com
prod:
mode: production
workspace:
host: https://techtown-prod.cloud.databricks.com
# Terminal
# databricks bundle validate
# databricks bundle deploy --target dev
# databricks bundle run --target dev orders_jobKey points
- Declarative Automation Bundles = the new name for Databricks Asset Bundles (DABs)
- databricks.yml holds resources and targets; code sits beside it in the same repo
- mode: development prefixes resources with [dev yourname] and pauses schedules
- validate → deploy → run is the everyday loop; destroy removes deployed resources
- Bundles can also be deployed from the workspace UI inside a Git folder