AWS Neuron 2.31.0 enhances performance and adds EKS operator
AWS Neuron 2.31.0 introduces significant updates, including enhanced MX FP8 support and tensor layout transformations in NKI 0.5.0. A public beta of the Neuron UltraServer Operator for Amazon EKS aims to automate UltraServer workloads. These updates benefit engineers and architects working with AWS Trainium and Inferentia chips for AI/ML workloads, particularly those on EKS, by improving performance and simplifying configuration.
- →Neuron UltraServer Operator for Amazon EKS now in public beta
- →NKI Library adds experimental kernels for various AI architectures
- →NKI 0.5.0 adds MX FP8 scale dtype support and tensor indirection
- →Neuron Compiler backend redesign improves Trn2 and Trn3 performance
- →Neuron Runtime simplifies configuration with contiguous shared scratchpad
Features (2) ›
- Neuron UltraServer Operator for Amazon EKS now in public beta
This new operator automates the discovery, workload allocation, and resource claim generation for Trainium UltraServer workloads on Amazon EKS. It simplifies the management of these high-performance computing workloads.
- NKI Library adds experimental kernels for various AI architectures
The NKI Library introduces 14 new experimental kernels, covering MoE training collectives, deformable attention, DeepSeek MLA projection, and ring attention. PyTorch reference implementations are also provided.
Enhancements (4) ›
- NKI 0.5.0 adds MX FP8 scale dtype support and tensor indirection
Version 0.5.0 of the Neuron Kernel Interface (NKI) includes support for MX FP8 scale dtype, enabling more efficient computation. It also introduces tensor indirection for compute operations and new NkiTensor view APIs for zero-cost tensor layout transformations.
- Neuron Compiler backend redesign improves Trn2 and Trn3 performance
The Neuron Compiler features a redesigned code generation backend, which is now enabled by default on Trn2 and Trn3 instances. This redesign is expected to deliver notable performance improvements for supported hardware.
- Neuron Runtime simplifies configuration with contiguous shared scratchpad
The Neuron Runtime now supports contiguous shared scratchpad, eliminating the need for manual scratchpad page size configuration. This change simplifies device setup and management.
- Neuron Explorer enhances workload debugging with improved tracing
Updates to Neuron Explorer include linking System Trace Viewer source code and improved default grouping. These enhancements aim to provide better visibility and aid in debugging complex workloads.
https://aws.amazon.com/about-aws/whats-new/2026/07/aws-announce-neuron-2-31-0