ReleaseBytes
databricks Databricks Blog ·

Databricks Lakehouse for R&D Data and AI Agents

blogdataaidatabricksengineer
announcement

Cellcentric's Data Hub, built on Databricks Unity Catalog and Lakehouse Federation, integrates scattered R&D data into a unified, AI-ready product. By prioritizing context coverage as a quality metric, the platform accelerates R&D investigations and provides agents with the same governed context as human users. This approach ensures secure and traceable data access for both employees and AI clients.

  • →Context as a Data Quality Metric Accelerates Investigations
  • →Fuel Cell Passport Data Product Example
  • →Data Products Combine Governance and Rich Context
  • →Governed Agent Access via Identity Flow
  • →Cellcentric's Data Hub integrates R&D data on Databricks
Features (4) ›
  • Context as a Data Quality Metric Accelerates Investigations

    By making documentation and context coverage a first-class quality metric, the platform has reduced R&D investigation times from weeks to days. This approach ensures data products have sufficient context for AI consumption and human understanding.

  • Fuel Cell Passport Data Product Example

    The Fuel Cell Passport data product exemplifies the approach, integrating data from five enterprise systems like SAP and MES. It models seven hierarchy levels and uses a state-based temporal model for comprehensive historical and point-in-time analysis.

  • Data Products Combine Governance and Rich Context

    The Data Hub layers a product structure over Unity Catalog's governed assets. This includes ownership, lifecycle state, domain, and detailed markdown catalog entries explaining the product's purpose, usage, and caveats for both humans and AI agents.

  • Governed Agent Access via Identity Flow

    Identity management flows from Azure AD through the Data Hub into Databricks using OAuth 2.0 token exchange, enabling governed access for AI agents. This prevents agents from bypassing platform security or gaining unauthorized production write access.

Enhancements (1) ›
  • Unified Operating Model for Human and AI Access

    The architecture supports a single operating model where identity and data access are governed through Unity Catalog. Agents access data using the authenticated user's identity, ensuring they operate within the same boundaries and permissions as human users.

Notes (1) ›
  • Cellcentric's Data Hub integrates R&D data on Databricks

    Cellcentric has built a governed context layer for data and AI called the Data Hub on Databricks, utilizing Unity Catalog and Lakehouse Federation. It provides a unified interface for employees and an MCP server for AI agents, integrating diverse R&D data sources.

Read the original announcement →

https://www.databricks.com/blog/why-rd-data-belongs-lakehouse-and-why-agents-need-it-there

Related releases