BigQuery Enhances Unstructured Data Analysis with New Search Capabilities
BigQuery has launched new features to simplify the process of extracting insights from unstructured data like PDFs, audio, and images. This aims to reduce the complexity of building AI-powered search and RAG pipelines by managing embedding generation and offering enhanced search functions. These updates are particularly beneficial for enterprises managing large volumes of diverse data, such as in the healthcare sector, and are now generally available or in public preview.
- →Autonomous Embedding Generation is Now Generally Available
- →AI.SEARCH Function Achieves General Availability with Performance Gains
- →Hybrid Search for Unstructured Data Enters Public Preview
- →Integrated Unstructured Data Analytics Platform Vision
Features (3) ›
- Autonomous Embedding Generation is Now Generally Available
BigQuery now automatically and continuously generates embeddings for new data as it's ingested, simplifying RAG and search application development. Users can choose external models or native Gemma embedding models, and it now supports generating embeddings over images for multimodal search.
- AI.SEARCH Function Achieves General Availability with Performance Gains
The AI.SEARCH function is now generally available, providing a streamlined, natural-language search experience without requiring embeddings in the search path. Significant optimizations have been made for single-query execution, yielding up to a 133x gain in slot efficiency for online applications and agentic searches.
- Hybrid Search for Unstructured Data Enters Public Preview
BigQuery's new hybrid search capability combines semantic vector search with lexical keyword matching, enabling more precise data retrieval. This approach improves search precision and can reduce LLM hallucination by reranking results based on both semantic relevance and keyword frequency.
Enhancements (1) ›
- Integrated Unstructured Data Analytics Platform Vision
These search and embedding features are part of a broader initiative to build an end-to-end unstructured data analytics platform within BigQuery. The platform aims to cover access, processing, grounding, and activation of insights from documents and other unstructured data sources.
https://cloud.google.com/blog/products/data-analytics/bigquery-search-innovations-unify-structured-unstructured-data/
Related releases
- BigQuery Enhances Conversational Analytics with New ML Functions Google Cloud release notes ·
- BigQuery Generative AI Functions Now Support New Gemini Models Google Cloud release notes ·
- BigQuery Conversational Analytics Adds AI.PREDICT for Predictive Modeling Google Cloud release notes ·
- Google Cloud Introduces Data Agent Kit for Agentic Data Analysis in IDEs Google Cloud Blog ·
- Optimizing RAG Apps: KDDI's Automated Evaluation & Performance Strategy Google Cloud Blog ·
- Google Threat Intelligence tracks evolving adversarial AI tactics and supply chain risks Google Cloud Blog ·