databricks Databricks Blog ·

Databricks Unifies Genie Agent Governance Across Structured and Unstructured Data

bloggovernancedatabricksgaarchitect
feature announcement

Databricks details how its Genie Agents leverage existing Unity Catalog governance to securely operate across structured and unstructured data. This allows agents to reason over tables, metrics, and documents while respecting user permissions, simplifying architecture and eliminating the need for separate systems for data types. The solution prevents data loss and unauthorized access by establishing Unity Catalog as the security perimeter, rather than relying on the LLM. It utilizes features like Automatic Identity Management (AIM), object privileges, row filters, column masks, and the newly generally available Attribute-Based Access Control (ABAC) policies.

  • Attribute-Based Access Control (ABAC) policies now GA
  • Genie Agents operate under end-user credentials
  • Automatic Identity Management ensures up-to-date access
  • Comprehensive access control for structured data
  • Apply consistent governance to unstructured documents
Features (1)
  • Attribute-Based Access Control (ABAC) policies now GA

    Attribute-Based Access Control (ABAC) policies are now generally available in Unity Catalog, along with governed tags and automated data classification. ABAC allows defining policies once and applying them across hundreds of tables based on tags, simplifying the management of row and column security compared to per-table configurations.

Notes (4)
  • Genie Agents operate under end-user credentials

    Databricks Genie Agents enforce governance by running with the end user's identity, ensuring all queries and answers are filtered by existing permissions. This design positions Unity Catalog as the security perimeter, preventing manipulation or bypass at the model layer, which is crucial for defensible governance controls.

  • Automatic Identity Management ensures up-to-date access

    Automatic Identity Management (AIM) for Microsoft Entra ID and Okta automatically syncs users, groups, and memberships to Databricks, eliminating the need for SCIM applications. This ensures access controls for Genie Agents are always current, with just-in-time provisioning enabling immediate access and revocation based on identity provider changes.

  • Comprehensive access control for structured data

    Genie Agents leverage Unity Catalog's four layers of access control for structured data: object privileges, row filters, column masks, and Attribute-Based Access Control (ABAC). These mechanisms ensure granular control over data assets like tables, views, and metric views, from table-level access to masking sensitive columns and filtering rows based on user permissions.

  • Apply consistent governance to unstructured documents

    The same robust governance principles and Unity Catalog mechanisms used for structured data are extended to unstructured files stored in Unity Catalog Volumes. This allows a single Genie Agent to reason across both data types while maintaining consistent, permissions-aware insights based on the end user's access rights.

Read the original announcement →

https://www.databricks.com/blog/how-ground-genie-agents-both-structured-data-and-documents-without-losing-governance

Related releases