ReleaseBytes
databricks Databricks Blog ·

Databricks to preview Lakebase, streaming, and Lakehouse innovations at VLDB 2026

blogaidatabrickspreviewarchitectmedia
announcement

Databricks will present several database and Lakehouse innovations at VLDB 2026, aimed at supporting AI agent workflows and real-time analytics. Key among these is Lakebase, a serverless PostgreSQL architecture over open lake storage that decouples compute from storage for transactional processing. The company will also detail advancements in Spark Structured Streaming, along with new Lakehouse optimizations like AutoLiquid for automated data layout and Ultron for history-based query optimization. These developments contribute to a vision of Lake Transactional Analytical Processing (LTAP), unifying transactional and analytical demands.

  • →Lakebase: Serverless Postgres Over Open Lake Storage for AI Agent Workloads
  • →AutoLiquid for Autonomic Lakehouse Data Layout Optimization
  • →Ultron: History-Based Query Optimization at Databricks
  • →Keynote: Database Engineering in the AI Era
  • →Evolution of Apache Spark Structured Streaming Architecture
Features (3) ›
  • Lakebase: Serverless Postgres Over Open Lake Storage for AI Agent Workloads

    Lakebase is a third-generation cloud database architecture designed for AI agent workloads, decoupling serverless PostgreSQL compute from storage. It provides sub-second cold starts, efficient Git-like database workflows, and low-latency analytics on live transactional data by persisting data in cloud object storage using open formats.

  • AutoLiquid for Autonomic Lakehouse Data Layout Optimization

    AutoLiquid automates the selection of optimal clustering keys for Lakehouse tables using a simple CLUSTER BY AUTO primitive. It employs heuristics and efficient shadow verification to outperform manual key selection on over 95% of evaluated workloads, significantly improving query performance.

  • Ultron: History-Based Query Optimization at Databricks

    Ultron is a history-based query optimization framework that leverages the repetitive nature of analytical workloads and executed query history to improve optimizer choices. This has significantly improved performance of production workloads, including a 25% reduction in median join latency.

Enhancements (1) ›
  • Evolution of Apache Spark Structured Streaming Architecture

    Databricks will detail how Structured Streaming's architecture has evolved to meet real-world needs, including microbatch pipelining for up to 3x throughput improvements. New stateful APIs simplify complex business logic, and the system now supports fine-grained access control.

Notes (2) ›
  • Keynote: Database Engineering in the AI Era

    Databricks Co-founder Reynold Xin will discuss the 'third golden age' of database engineering, driven by AI agents. He will introduce Lakebase, applying storage-compute separation to OLTP, and Lake Transactional Analytical Processing (LTAP), which unifies transactional and analytical processing.

  • Demo of Enzyme Incremental View Maintenance Engine

    A demo paper will showcase Enzyme, Databricks' incremental view maintenance engine for data engineering workloads. It demonstrates how materialized views are incrementally maintained within the Databricks platform.

Read the original announcement →

https://www.databricks.com/blog/building-ai-era-lakebase-streaming-and-lakehouse-innovations-vldb-2026

Related releases