AWS Neuron 2.32 Enhances NKI, MXFP8 Training, and Variable-Size Collectives
AWS Neuron 2.32.0 is now generally available, introducing Neuron Kernel Interface (NKI) 0.6.0 with new programming constructs and an on-device top-K instruction. The release also adds 13 new NKI Library kernels for Mixture of Experts (MoE) training and sparse attention, alongside MXFP8 support for end-to-end blockwise MoE training. Neuron Runtime gains variable-size collectives for Trn2 and Trn3, improving flexibility for distributed AI/ML workloads. These updates primarily benefit data scientists and engineers leveraging AWS Neuron for advanced machine learning model development and deployment.
- →Expanded Neuron Kernel Interface (NKI) Programming and Library Kernels
- →Neuron Agentic Development Skill for Transformer Models
- →Variable-Size Collectives for Neuron Runtime on Trn2 and Trn3
- →Updated vLLM Neuron Plugin and Development Tools
Features (2) ›
- Expanded Neuron Kernel Interface (NKI) Programming and Library Kernels
NKI 0.6.0 is introduced with an on-device top-K instruction, variable-length `all-gather` for differently sized tensors, and new loop constructs. The NKI Library adds 13 new kernels for Mixture of Experts (MoE) training and sparse attention, including context encoding for DeepSeek-V3.2 and MXFP8 attention/forward/backward passes for blockwise MoE layers.
- Neuron Agentic Development Skill for Transformer Models
A new Neuron Agentic Development skill is now available, designed to port transformer models to the vLLM Neuron plugin. This facilitates easier integration and optimization of transformer-based models within the Neuron ecosystem.
Enhancements (2) ›
- Variable-Size Collectives for Neuron Runtime on Trn2 and Trn3
The Neuron Runtime now supports variable-size `all-gather`, `reduce-scatter`, and `all-to-all` collectives on Trn2 and Trn3 instances. This allows each rank in a distributed computation to contribute or receive a different number of elements, enhancing flexibility for complex AI/ML workloads.
- Updated vLLM Neuron Plugin and Development Tools
The vLLM Neuron plugin has been updated to vLLM 0.24.0 and is included in all Neuron Deep Learning AMIs and Containers. Additionally, the Neuron Compiler now offers explicit control over 64-bit integer compilation, and Neuron Explorer adds per-core host CPU utilization to its System Trace Viewer.
https://aws.amazon.com/about-aws/whats-new/2026/08/aws-announce-neuron-2-32-0
Related releases
- Amazon ECS adds automated agent connectivity detection and repair AWS What's New ·
- Amazon EC2 C8gd, M8gd, and R8gd Instances Now Available in Additional Regions AWS What's New ·
- Amazon EC2 P6-B300 Instances Now Available in Asia Pacific (Seoul) Region AWS What's New ·
- AWS Local Zone in Las Vegas, Nevada, Now Generally Available AWS What's New ·
- Amazon EC2 Enables AMI Creation with Local Snapshots on Outposts AWS What's New ·
- AWS Marketplace Launches Support for Amazon Lightsail AWS What's New ·