Google Cloud details Hive Metastore modernization with Lakehouse runtime catalog

The article outlines three core pain points with standalone Apache Hive Metastores in modern cloud environments: architectural and scaling bottlenecks due to reliance on relational database backends, siloed identity and security governance requiring fragmented policies, and significant operational overhead and TCO from managing high-availability instances.

Google Cloud offers the Lakehouse runtime catalog, a fully serverless, highly available, and unified metadata registry. Introduced last year, it supports both legacy Hive/Parquet tables and modern open table formats like Apache Iceberg, natively implementing the Iceberg REST Catalog specification to decouple metadata discovery from compute engines.

The Lakehouse runtime catalog provides multi-engine interoperability across Google Cloud Managed Spark, BigQuery, and open-source engines via standard REST interfaces. It supports open APIs for Iceberg and Hive catalogs, enables zero-data-copy by pointing to existing data in Cloud Storage, and offers AI-powered governance and security through integration with Knowledge Catalog and Cloud IAM.

The platform provides a capability for zero-data-copy migration from self-managed Hive Metastores to the Google Cloud Lakehouse. This process connects directly to the legacy metastore, extracts external table definitions and partition maps, and registers them into the serverless Lakehouse catalog, enabling immediate use with various analytics engines.