ReleaseBytes
gcp Google Cloud Blog ·

BigQuery DTS adds new connectors and Iceberg ingestion

blogdataawsazuregcpsnowflakegapreviewretailgcp-bigquerygcp-cloud-storage
feature patch

Google Cloud's BigQuery Data Transfer Service (DTS) has expanded its capabilities with new connectors for enterprise databases, e-commerce platforms, and a new feature for direct ingestion into Apache Iceberg tables. These updates aim to reduce ETL pipeline complexity and cost for data ingestion, benefiting data engineers and analysts. The new features include previews for Iceberg ingestion and remote agent support, alongside generally available connectors for Shopify, Klaviyo, HubSpot, Mailchimp, Microsoft SQL Server, PostgreSQL, MySQL, and Snowflake.

  • →Direct ingestion to Apache Iceberg managed tables (Preview)
  • →Remote Model Context Protocol (MCP) Server (Preview)
  • →New connectors for enterprise databases
  • →New connectors for e-commerce and marketing platforms
  • →Snowflake migration connector (GA)
Features (5) ›
  • Direct ingestion to Apache Iceberg managed tables (Preview)

    Users can now ingest data directly from cloud storage sources like GCS, S3, and Azure Blob Storage into Apache Iceberg managed tables. This feature enables cross-cloud compatibility for query engines while leveraging BigQuery's performance. It is currently in preview.

  • Remote Model Context Protocol (MCP) Server (Preview)

    A fully managed remote MCP Server is now available in preview. This allows developers to connect AI applications and agents to DTS, enabling programmatic discovery and configuration of data sources and transfers.

  • New connectors for enterprise databases

    BigQuery DTS now supports Microsoft SQL Server (Preview) for full or incremental transfers of tables, schemas, and operational data. PostgreSQL and MySQL are now generally available for replicating high-volume workloads from on-premise, CloudSQL, or other clouds.

  • New connectors for e-commerce and marketing platforms

    New preview connectors are available for Shopify (order histories, inventory), Klaviyo (engagement logs), HubSpot (pipeline and marketing metrics), and Mailchimp (campaign performance, audience lists). These aim to centralize customer data for analysis.

  • Snowflake migration connector (GA)

    A generally available connector for Snowflake allows data migration with features like incremental transfer, auto schema detection, and private connectivity. It supports data residing on Google Cloud, AWS, and Azure.

Enhancements (1) ›
  • Enhancements to major connector updates

    Connectors for ServiceNow, Salesforce, and Oracle have been enhanced with native incremental update support. This improvement speeds up large-scale pipeline refreshes for CRM, ITSM, and financial workflows.

Read the original announcement →

https://cloud.google.com/blog/products/data-analytics/new-bigquery-data-transfer-service-capabilities/

Related releases