ReleaseBytes
gcp Google Cloud Blog ·

AlloyDB AI Functions enhance PostgreSQL multilingual full-text search

blogdatagcpengineermedia
feature

AlloyDB now integrates Gemini models directly into its database via AI Functions, enabling intelligent word segmentation for logographical languages like Chinese, Japanese, and Korean. This overcomes limitations of traditional tokenization and external preprocessing pipelines, offering more accurate full-text search without complex ETL. The solution is available now for AlloyDB customers and benefits database developers and architects working with multilingual data.

  • →Enhanced full-text search for logographical languages with native AI
  • →AlloyDB AI Functions integrate Gemini for native in-database text processing
  • →Generated columns for automatic search vector and embedding updates
  • →High-throughput batching procedure for document segmentation
Features (1) ›
  • Enhanced full-text search for logographical languages with native AI

    AlloyDB's new capabilities address limitations in traditional PostgreSQL full-text search for languages like Chinese, Japanese, and Korean, which lack whitespace delimiters. By using Gemini's multilingual capabilities, AlloyDB can now perform intelligent word segmentation and stop-word removal directly within the database.

Enhancements (2) ›
  • Generated columns for automatic search vector and embedding updates

    The solution utilizes generated columns for `search_vector` and `embedding`, ensuring that both the full-text search index and vector embeddings are automatically updated when segmented content changes. This simplifies application logic to a single update statement.

  • High-throughput batching procedure for document segmentation

    A new stored procedure, `segment_all_documents`, is introduced to handle large datasets efficiently. It employs array-based batching to process rows in parallel, aggregate content, and call AI models, mitigating risks associated with raw cursor loops like excessive locking and rollback failures.

Notes (1) ›
  • AlloyDB AI Functions integrate Gemini for native in-database text processing

    AlloyDB AI Functions allow direct calls to Gemini models from SQL, enabling text preprocessing and segmentation within the database. This avoids data movement, external microservices, and offers stored procedure-based batching for efficient handling of large datasets.

Read the original announcement →

https://cloud.google.com/blog/products/databases/how-alloydb-overcomes-indexing-limitations-with-ai-functions/

Related releases