Epic Games Optimizes Fortnite Analytics on Amazon OpenSearch Service

Fortnite utilizes Amazon OpenSearch Service to store and analyze game event data, enabling complex queries and multi-layered aggregations to inform decision-making and enhance player experience. These insights help identify gameplay trends and understand player engagement with new content.

Epic Games faced challenges in scaling their OpenSearch Service cluster to support more frequent aggregation jobs without incurring prohibitive costs. The existing configuration struggled to maintain optimal performance and resource utilization for high-volume writes and CPU-intensive batch jobs.

Analysis showed that despite using memory-optimized instances, system memory was not fully utilized, and node-level metrics revealed CPU hotspots. This was attributed to an uneven distribution of query execution, where primary shard counts did not match the data node count, leading to specific nodes being overutilized.

AWS recommended migrating from memory-optimized r7g instances to compute-optimized c7g instances, offering a better ratio of vCPU to RAM for CPU-intensive aggregation workloads. This change increased aggregate CPU capacity by approximately 33% and reduced costs by about 10%.

The sharding strategy was updated to match the primary shard count to the data node count with one replica. This ensures even distribution of write loads and aggregation queries across all data nodes, eliminating previous node hotspots and improving overall cluster balance.

Redundant text field types were removed from index mappings for fields only used in aggregation or filter contexts, reducing storage overhead. For high-cardinality string fields, the murmur3 field type was implemented to compute hashes at index time, improving query performance by avoiding expensive string hashing at query time.