FDA Builds Widely Adopted Generative AI Platform on Databricks

Databricks Unity Catalog provided the necessary governance layer to address the FDA's strict security requirements for trade secrets and sensitive regulatory data. It ensured data containment, controlled asset sharing, and enforced granular table-level access across the platform, earning trust from various agency centers.

Following the deployment of ELSA to all 16,000 FDA staff, adoption rapidly grew to 85% within two months. Users can access multiple AI models through a single interface and are now creating custom AI agents to automate workflows using their own data and standard operating procedures.

The FDA is leveraging Databricks ML and NLP capabilities through MLflow within ELSA to extract critical data from millions of regulatory submission pages. For instance, reviewers can now find starting materials for drug applications in approximately three minutes, a task that previously took days.

Medical doctors, scientists, and administrative staff are independently building hundreds of new AI agents per week within ELSA. This is enabled by the combination of structured, governed data and accessible tooling, making agent creation available beyond data scientists.

The FDA's Office of Digital Transformation built ELSA, a generative AI platform, and Halo, a governed data foundation running on Databricks, to unify data across its eight centers. This consolidation effort brought 50-60 data sources into a single Databricks platform, drastically speeding up data sharing and replacing batch processing with real-time streaming.