Declarative Automation Bundles & the Databricks CLI

Databricks · Notebooks & Development

Theory

Declarative Automation Bundles (formerly Databricks Asset Bundles, or DABs) describe Databricks resources as code. A bundle is a folder with a databricks.yml file that lists resources such as jobs, pipelines, dashboards and apps, plus targets like dev and prod that point to different workspaces or settings. The source code (notebooks, Python files, SQL) lives next to it in the same Git repository.

The Databricks CLI works with bundles: databricks bundle init creates a project from a template (for example default-python), databricks bundle validate checks the YAML, databricks bundle deploy --target dev uploads the files and creates or updates the resources, and databricks bundle run runs a job. In development mode, deployed resources get a [dev yourname] prefix so they do not clash with teammates. For example, a CI pipeline runs databricks bundle deploy --target prod after every merge to main.

Why we use it

Clicking jobs together in the UI is not repeatable and cannot be reviewed. Bundles make jobs and pipelines version-controlled, testable and deployable to many environments with one command, which is the basis of CI/CD on Databricks.

When to use it

Use bundles for any job or pipeline that goes to production or is shared by a team. Use the UI for quick experiments. Use the plain CLI or REST API for one-off admin tasks that are not part of a project.

How it works

  1. databricks bundle init from a template
  2. Edit databricks.yml and the source code
  3. databricks bundle validate
  4. Is validation clean? Yes: databricks bundle deploy --target dev. No: Fix the YAML and validate again.
  5. databricks bundle run --target dev orders_job
  6. CI deploys with --target prod after merge

Syntax

databricks auth login --host <workspace-url>
databricks bundle init [default-python]
databricks bundle validate
databricks bundle deploy --target dev|prod
databricks bundle run --target dev <job_key>
databricks bundle destroy --target dev

Example

# databricks.yml
bundle:
  name: orders_project

resources:
  jobs:
    orders_job:
      name: orders_job
      tasks:
        - task_key: load_orders
          notebook_task:
            notebook_path: ./src/orders_etl.ipynb

targets:
  dev:
    mode: development
    default: true
    workspace:
      host: https://techtown-dev.cloud.databricks.com
  prod:
    mode: production
    workspace:
      host: https://techtown-prod.cloud.databricks.com

# Terminal
# databricks bundle validate
# databricks bundle deploy --target dev
# databricks bundle run --target dev orders_job

Key points

Databricks topics

Databricks Fundamentals

Compute

Notebooks & Development

Apache Spark on Databricks

Delta Lake

Unity Catalog & Governance

Data Ingestion

Declarative Pipelines (Lakeflow)

Lakeflow Jobs

Databricks SQL & BI

Architecture, Performance & Cost

Machine Learning & AI

Certification & Interview

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Master Databricks

The lakehouse platform from the ground up: Apache Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL and MLflow. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice SQL / Spark SQL Sharpen the SQL you use in Databricks SQL and Spark SQL: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Databricks  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Databricks needs a real workspace — there's no in-browser engine for it. Click Copy above, then paste it into a notebook or the SQL editor in Databricks Free Edition (free, no credit card) and run it on serverless compute.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic