Databricks Query Tags Improve dbt Pipeline Usage Attribution

Databricks Query Tags allow for granular usage attribution of dbt pipelines by automatically injecting tags like dbt_model_name and enabling custom tags for cost centers, projects, and environments. These tags are recorded in system.query.history, simplifying cost attribution, performance debugging, and workload monitoring.

The dbt-databricks adapter automatically injects metadata tags such as @@dbt_model_name, @@dbt_materialized, @@dbt_core_version, and @@dbt_databricks_version. These tags provide per-model visibility without requiring manual configuration, enriching every dbt model execution.

Users can configure Query Tags at the profile level in dbt profile configuration or at the model level in dbt_project.yml or model SQL definitions. Model-level tags merge with profile-level tags, with model-level values taking precedence in case of conflicts.

Populated Query Tags appear in the `query_tags` column within system.query.history as a MAP data type. This allows for direct SQL querying and analysis, enabling users to extract custom and auto-injected tags into individual columns for aggregation and reporting.

Query Tags enable direct cost attribution and performance analysis via SQL queries or Genie's natural language interface. A reference project includes an AI/BI dashboard that visualizes compute time per dbt model, materialization type, and environment, aiding in optimization efforts.

Databricks metric views (available with dbt-databricks 1.12+) can also carry Query Tags via the `query_tags` config parameter. This allows for tracking the SQL queries associated with metric view creation or refresh, distinct from Unity Catalog object tags.