AWS blog details building a scalable recommendation system

The system establishes a centralized data lake using Amazon S3 and AWS Lake Formation to create a single source of truth for recommendation pipelines. Lake Formation provides fine-grained cross-account access and schema governance through AWS Glue Data Catalog, ensuring data consistency and enabling faster pipeline development by querying curated, validated 'golden datasets'.

Amazon MWAA is used as the orchestration layer, managing complex multi-step workflows and removing the operational burden of Airflow infrastructure. The team developed reusable custom Airflow operators, encapsulating core compute engines to streamline new pipeline development and ensure a consistent pattern across recommendation pipelines.

Amazon Athena is employed for data access, running SQL queries against the Glue Data Catalog and unloading results to S3. AWS Glue handles compute-intensive data transformations using PySpark, with Auto Scaling worker pools for scalability and ephemeral jobs that read from and write to S3, requiring no long-running clusters.

Amazon SageMaker powers ML workloads including model training (using SageMaker Training Jobs on GPU instances), batch inference (via SageMaker Batch Transform), and vector search for approximate nearest neighbor (ANN) search using FAISS within SageMaker Processing Jobs. This integration within MWAA ensures end-to-end lineage, automatic retry mechanisms, and proper resource sequencing for ML tasks.