Google details BigQuery's cross-cloud caching for borderless Lakehouse
This post explains the enhancements to BigQuery's borderless Lakehouse, which enables data engineers, data scientists, and AI agents to query governed data directly where it lives across clouds. It details cross-cloud caching, now in preview, which transparently accelerates queries and significantly reduces remote data transfer costs by caching frequently accessed data locally in Google Cloud. The article also mentions BigQuery cross-cloud connections, also in preview, for querying non-Iceberg data across different cloud environments. These features aim to lower the total cost of ownership for multi-cloud analytics and AI workloads, addressing challenges of distributed data estates.
- →Enhancements to BigQuery's Borderless Lakehouse
- →Technical mechanisms of cross-cloud caching
- →Practical example of cross-cloud caching efficiency
Notes (3) ›
- Enhancements to BigQuery's Borderless Lakehouse
The post details two enhancements for BigQuery's borderless Lakehouse, designed to optimize multi-cloud efficiency for data querying. This includes the preview of cross-cloud caching, which accelerates queries and cuts remote transfer costs, and BigQuery cross-cloud connections, also in preview, for querying non-Iceberg data in other clouds.
- Technical mechanisms of cross-cloud caching
Cross-cloud caching operates at sub-file block granularity, caching only specific column chunks and dictionary pages. It includes default encryption at rest using Google-managed keys, tenant and regional isolation for data residency, and freshness checks that fetch remote object metadata to prevent stale reads and ensure accuracy.
- Practical example of cross-cloud caching efficiency
The article illustrates cross-cloud caching with an e-commerce scenario, demonstrating how an initial query populates the cache. Subsequent queries, even with modifications, achieve high cache hit rates and drastically reduce data transfer from remote clouds, highlighting significant cost and performance benefits at enterprise scale.
https://cloud.google.com/blog/products/data-analytics/borderless-lakehouse-cross-cloud-caching-and-connections/
Related releases
- Google introduces Workbench workshops for building multi-agent systems at DevFest Google Cloud Blog ·
- Google Public Sector Showcases AI Solutions for Government Service Delivery Google Cloud Blog ·
- Looker 26.18 Addresses Bugs, Enhances Theming & Conversational Analytics Google Cloud release notes ·
- BigQuery introduces GA unit testing for SQL transformation pipelines Google Cloud release notes ·
- Pine59's Migration to Airflow 3 on Google Cloud Improves Data and MLOps Pipelines Google Cloud Blog ·
- BigQuery Adds Migration Lineage Service for Data Warehouse Planning Google Cloud release notes ·