ReleaseBytes
databricks Databricks Blog ·

Databricks AI Search adds high-QPS scaling

blogaidatabricksgaengineerretail
feature announcement

Databricks AI Search now supports scaling endpoints to thousands of queries per second (QPS) with a single configuration parameter, eliminating the need for manual replica management or load balancing. This feature is crucial for real-time applications like search bars, recommendation systems, and entity resolution that experience high traffic volumes. It's generally available today for all users, simplifying the transition from prototype to production.

  • →High QPS scaling for AI Search endpoints
  • →Production observability for AI Search endpoints
  • →Performance considerations for high-QPS workloads
  • →Future enhancements for AI Search scaling
Features (1) ›
  • High QPS scaling for AI Search endpoints

    Databricks AI Search now supports scaling endpoints to thousands of QPS with a single configuration parameter, eliminating manual replica and load balancer management. This feature is generally available today.

Enhancements (1) ›
  • Production observability for AI Search endpoints

    The AI Search UI now displays endpoint QPS, latency, and health for every endpoint, providing crucial visibility for production workloads. This helps users monitor scaling status and traffic patterns.

Notes (2) ›
  • Performance considerations for high-QPS workloads

    For maximum performance in high-QPS production workloads, use service principal authentication, as personal access tokens are capped at lower QPS limits suitable only for prototyping.

  • Future enhancements for AI Search scaling

    Databricks plans to introduce automatic scaling for traffic spikes and support for Storage-Optimized endpoints later this year, further simplifying large-scale search operations.

Read the original announcement →

https://www.databricks.com/blog/prototype-production-high-qps-databricks-ai-search

Related releases