ReleaseBytes
gcp Google Cloud Blog ·

How KDDI Optimized Its Buffmee RAG App With Google's AI Evaluation and ADK

blogaigcpengineermediaenergygcp-bigquery
announcement

KDDI significantly optimized its consumer-facing Retrieval-Augmented Generation (RAG) app, Buffmee, achieving a 38% reduction in latency and an 18% improvement in Time To First Token (TTFT). They accomplished this by implementing automated AI evaluation frameworks, such as LLM-as-a-Judge, and using BigQuery Agent Analytics with the Agent Development Kit (ADK) to identify performance bottlenecks. The techniques enabled Buffmee to provide highly accurate, reliable, and personalized learning experiences grounded in over 100 sources. This case study details valuable strategies for other developers building scalable and reliable generative AI applications using Google Cloud's AI platform.

  • →Automated AI Evaluation for Content Reliability
  • →Performance Optimization Using BigQuery Agent Analytics
  • →Core Technical Principles for Robust AI Evaluation
Notes (3) ›
  • Automated AI Evaluation for Content Reliability

    KDDI's development team designed a systematic AI evaluation process using Gemini Enterprise Agent Platform Evaluation Service, incorporating LLM-as-a-Judge and the Rule of Hundreds. This automated framework replaced manual testing, improving Buffmee's groundedness scores by 25% for its extensive content corpus.

  • Performance Optimization Using BigQuery Agent Analytics

    The team utilized BigQuery Agent Analytics and the Agent Development Kit (ADK) log analysis agent to identify and resolve deep-stack bottlenecks in real time. Optimizations included refining system prompts and sub-agent routing, which led to a 38% reduction in application response latency and an 18% improvement in Time To First Token (TTFT).

  • Core Technical Principles for Robust AI Evaluation

    KDDI implemented four key practices: transitioning to binary evaluation for critical metrics, strategic content sampling across diverse formats, calibrating thresholds based on product judgment, and modularly splitting large prompts into ADK Skills to prevent LLM attention drift and latency degradation.

Read the original announcement →

https://cloud.google.com/blog/topics/customers/how-kddi-optimized-rag-performance-with-agent-development-kit/

Related releases