Skip to main content
Databricks CI/CD guide

Databricks Asset Bundles are now Declarative Automation Bundles

Treat a Databricks project as one deployable release unit. Keep code, jobs, Lakeflow pipelines, permissions and environment targets in version control, then validate, plan and promote the same project through dev and production.

Read the bundles guide
Source controlled Plan before deploy Dev to prod targets
factory-analytics bundle
Release unit

Repository

databricks.yml
resources/
src/
tests/

Declarative Automation Bundle

One project definition

Jobs
Pipelines
Permissions
Artifacts
same source, different targets

dev

development

personal names · schedules paused · rapid iteration

prod

production

stable identity · controls · production resources

The project stays consistent. Targets change the deployment context without cloning the codebase.

Rename without a rewrite

The product name changed. The bundle workflow remains compatible.

Before 16 March 2026

Databricks Asset Bundles

Current name

Declarative Automation Bundles

Existing configuration remains valid

The rename is non-breaking. Existing databricks.yml files and the databricks bundle CLI command continue to work.

Source control

unchanged

databricks.yml

unchanged

bundle CLI

unchanged

What changed

Use the new name, keep the same project model

Databricks renamed Asset Bundles because the word asset had become ambiguous across the platform. The new name better describes their purpose: declarative automation for complete data and AI projects.

A bundle is a project release unit

It combines source files, resource definitions, artifacts and deployment targets.

The workflow is software engineering

Version control, code review, testing and CI/CD are the core use case.

The scope is broader than notebooks

Jobs, Lakeflow pipelines, dashboards, schemas, volumes and many other Databricks resources can be part of the same project definition.

Bundle anatomy

Keep project intent next to the code

A focused bundle should contain the files and Databricks resources that one team releases on the same lifecycle. Databricks recommends smaller bundles instead of one platform-sized bundle.

Manufacturing project

factory-production-analytics

databricks.yml
resources/
ingestion.job.ymlLakeflow Job
transform.pipeline.ymlPipeline
src/
production/Python + SQL
tests/

One ownership boundary, one promotion path and one release cadence.

What databricks.yml coordinates

The root manifest connects code, resources and deployment context.

Bundle

identity + CLI constraint

name · databricks_cli_version

Include

resource configuration

resources/*.yml

Variables

environment inputs

catalog · schema · warehouse

Targets

deployment context

dev · staging · prod

Permissions

resource access

groups · service principals

Artifacts & sync

deployable code

wheels · files · shared folders

databricks.yml · focused project
bundle:
  name: factory-production-analytics
  databricks_cli_version: '>= 1.3.0'
  engine: direct

include:
  - resources/*.yml

targets:
  dev:
    mode: development
    default: true
  prod:
    mode: production

Same bundle, different operating mode

Targets should change environment context, not create separate copies of the project.

Development

Fast, isolated iteration

developer-specific resource names
schedules and triggers paused by default
Lakeflow pipelines use development mode
development-only overrides allowed
Production

Stable identity and stricter checks

production pipeline mode required
optional Git branch validation
service principal recommended
non-personal deployment paths
Separate identities: the identity that deploys the bundle and the run_as identity used by jobs or pipelines do not have to be the same.

Environment strategy

Use one bundle across dev, staging and production

Databricks recommends that one bundle covers the environments for a project. Use target modes, variables and target overrides for the values that genuinely differ.

Do not clone the repository per environment

Promote the same Git-controlled project through different targets.

Override only environment configuration

Catalogs, schemas, warehouse references and run identities can vary without duplicating business logic.

Keep secrets outside the bundle

Use Databricks authentication, connections and secret management. Bundle variables are configuration, not a secret store.

Deployment workflow

Turn a Git change into an explicit deployment plan

Validation catches configuration issues. Planning shows what will change. With the direct deployment engine, a JSON plan can be reviewed and replayed so the approved actions are the actions applied to production.

Databricks CLI
databricks bundle validate -t prod

databricks bundle plan -t prod -o json \
  > plan.json

databricks bundle deploy -t prod \
  --plan plan.json
production deployment plan
createjobs.production_refresh
updatepipelines.factory_gold
sameschemas.analytics

Validate

Plan

Deploy approved plan

Selective --select deployments are useful for development, but Databricks does not recommend them for production promotion.

CI/CD

Use CI/CD for production promotion, not a developer laptop

Declarative Automation Bundles are Databricks' recommended CI/CD approach. For automated deployments, use a service principal and workload identity federation where possible so the pipeline does not depend on a long-lived Databricks secret.

Review the source change

Pull request checks should validate code, tests and bundle configuration before promotion.

Federate the pipeline identity

Azure DevOps, GitHub Actions and other CI systems can exchange OIDC identity tokens for Databricks OAuth tokens.

Separate deploy and run permissions

The CI identity needs permission to deploy. The workload itself can run under a dedicated run_as service principal.

Production promotion lane

Git to Databricks without stored deployment secrets

Developer

Pull request

source + bundle configuration

CI pipeline

test + validate
OIDC federation
plan.json

Control

Production approval

review exact planned actions

Databricks

Deploy approved plan

stable bundle identity + production run_as

Adopt existing resources

Bring existing jobs and pipelines under bundle management without recreating them

Existing workspace resources can be converted into bundle configuration. The important step is binding the generated resource definition to the existing remote object before normal deployments take over.

01

Generate

Create bundle YAML from an existing job, pipeline, dashboard or app.

02

Review

Remove workspace-specific noise and align the definition with your project standards.

03

Bind

Associate the bundle resource key with the existing remote resource ID.

04

Deploy

Apply future changes through the bundle without creating a duplicate resource.

Important: generated configuration by itself does not adopt the existing resource. Deploying it without binding can create a new resource instead.
Existing job adoption
databricks bundle generate job --existing-job-id 123456789

databricks bundle deployment bind factory_job 123456789 -t prod

databricks bundle plan -t prod

Deployment boundary

Use Bundles for Databricks projects and Terraform for external infrastructure

Current Databricks developer guidance is clear: use Declarative Automation Bundles for Databricks resources in the project lifecycle. Keep Terraform for cloud-level resources and privileged platform administration.

Declarative Automation Bundles

Project-owned Databricks resources

Lakeflow Jobs and pipelines

source code and artifacts

dashboards and project schemas

resource permissions and run identities

Terraform

Cloud and platform foundation

Azure networking and private endpoints

storage accounts and Key Vault

workspace and account provisioning

admin actions outside the project team

Ownership test: if the resource is released with the application or data product, put it in the bundle. If it establishes the cloud or workspace platform itself, keep it in the platform IaC layer.

Direct deployment engine

The Terraform-backed bundle engine is on the way out

The direct deployment engine is now the current path. It uses the Databricks Go SDK, produces richer plans and no longer depends on the Terraform provider for bundle deployment state.

Original model

Terraform engine

Bundle deployments originally used the Databricks Terraform provider and Terraform state.

terraform.tfstate

GA · CLI 1.3.0

Direct engine

New bundles use direct deployment by default. Plans can show field-level diffs and be replayed during deployment.

resources.json

CLI 1.14.0+

Automatic migration path

Bundles still on the Terraform engine can be auto-migrated after a clean dry-run conversion. Databricks recommends moving to direct deployment.

Terraform engine slated for retirement

Faster deploys

Databricks reports deployment improvements of up to 40% with the direct engine.

Replayable plans

Create a JSON plan, approve it and apply that same plan during deployment.

Broader resource support

Direct deployment supports additional Databricks resource types and optional immutable folders.

Shared template

ffa-data-product-template

Production

bundle A

Inventory

bundle B

Delivery

bundle C

same structure
same CI contract
independent release

Scale the practice

Standardize with templates, not one giant bundle

Custom bundle templates let a platform team encode folder structure, common CI files, tests and resource defaults. Project teams still keep separate bundle identities and release independently.

Create a repeatable starting point

Use databricks bundle init with a default or custom template.

Keep bundles small and focused

Split by ownership, permission boundary, release cadence or independent rollback needs.

Share common code deliberately

Multiple bundles can live in one repository and use shared folders where common libraries genuinely belong together.

Common mistakes

Avoid deployment patterns that reintroduce environment drift

Bundles help when the project definition is the source of truth. Manual workspace edits and unclear ownership undermine that model quickly.

Avoid

Separate bundle copies for dev and prod

Better default

One bundle with environment targets

Avoid

One giant bundle for the whole data platform

Better default

Split by ownership and release lifecycle

Avoid

Deploying production from a developer identity

Better default

Use CI/CD with a production service principal

Avoid

Long-lived CI secrets

Better default

Use workload identity federation where possible

Avoid

Generating YAML and deploying immediately

Better default

Bind existing remote resources before adoption

Avoid

Using --select for production promotion

Better default

Promote the complete tested release unit

Avoid

Keeping new bundles on the Terraform engine

Better default

Use the direct deployment engine

How Food For Analytics implements it

Titan uses bundles as the release unit for Databricks data products

Platform infrastructure and project delivery have different lifecycles. Titan keeps Azure and workspace foundations in platform IaC, while Databricks jobs, pipelines and project configuration move through Declarative Automation Bundles.

Start consistently

Titan project template

Common repo structure, validation, permissions and target conventions are defined once.

jobspipelinestests

Titan on Azure Databricks

Project bundle

Each owned data product can move as one source-controlled release unit instead of relying on manual workspace configuration.

databricks.yml
resources
source code
permissions

Promote deliberately

Dev to production

CI validates and plans the deployment, then production receives the approved bundle definition.

devtestprod

FAQ

Frequently asked questions

Practical answers about Databricks Asset Bundles and Declarative Automation Bundles.

When were Databricks Asset Bundles renamed?

Databricks renamed Databricks Asset Bundles to Declarative Automation Bundles on 16 March 2026. The rename is non-breaking and existing bundle configuration does not need to be rewritten.

Should new projects use the direct deployment engine?

Yes. The direct deployment engine is generally available and is the default for new bundles created with Databricks CLI 1.3.0 and above. Databricks recommends moving existing Terraform-engine bundles to direct deployment because the older engine is being retired.

What is the recommended production deployment pattern?

Use a production target, deploy through CI/CD, use a stable non-personal bundle identity and use a service principal for production workloads. For CI/CD authentication, Databricks recommends workload identity federation where possible instead of long-lived secrets.

Are Declarative Automation Bundles a replacement for Terraform?

Not for the whole platform. Current Databricks developer guidance recommends Bundles for Databricks project resources and Terraform for cloud-level resources, workspace provisioning, networking and other privileged platform administration.

Can existing Databricks jobs and pipelines be adopted into a bundle?

Yes. Bundle configuration can be generated from existing resources. Bind the generated bundle resource to the existing remote resource before normal deployment so the existing object is updated rather than duplicated.

Should dev and production use separate bundles?

Usually no. Databricks recommends one bundle per owned project across its environments. Use development and production targets to vary deployment settings, identities and environment-specific configuration.

Practical next step

Make Databricks releases repeatable

Define the project boundary, environment targets and production identity before turning manual workspace configuration into automated deployment.

One release unit

Reviewed plan

Controlled prod