Over the last two weeks I’ve worked extensively with the relatively new Databricks Asset Bundle (or DAB) feature. In a nutshell, DABs allow you to deploy Databricks resources, i.e. notebooks, workflows, pipelines, automatically by declaring them in a YAML file. An analogy would be Terraform for cloud resource deployment.
I especially appreciated the definition of different targets: dev, test, and prod. This allows you to automatically deploy changes to your Databricks resources by making commits/PRs on a branch. So new commits to my main branch will deploy changes to the production Databricks workspace. Commits to develop to my development workspace, and test for testing.
It’s really neat how variables (like Data Catalog names) can be scoped to a particular target in one databricks.yml file, to have different values for each target.
Are you already using Databricks Asset Bundles?