aws AWS Big Data Blog ·

Amazon MSK Now Supports In-Place ZooKeeper-to-KRaft Cluster Upgrades

blogdataawsgaengineer
feature

Amazon MSK now supports in-place upgrades for Provisioned clusters transitioning from ZooKeeper to KRaft metadata mode. This allows users to modernize their existing clusters to the Raft-based protocol embedded in Kafka 4.0+, which officially removes ZooKeeper. The upgrade process is managed, keeps the cluster available for produce/consume traffic, and retains data with no expected downtime if best practices are followed. This prepares clusters for Kafka 4.x and future releases, with Kafka 3.9.x serving as the bridge version.

  • In-place ZooKeeper-to-KRaft upgrades for Amazon MSK
  • Key prerequisites for MSK ZooKeeper-to-KRaft upgrades
  • Managed, multi-phase upgrade process with built-in recovery
Features (1)
  • In-place ZooKeeper-to-KRaft upgrades for Amazon MSK

    Amazon MSK now supports in-place upgrades for Provisioned clusters, allowing migration from ZooKeeper to KRaft metadata mode. This enables modernization to Apache Kafka's embedded Raft-based consensus protocol, which becomes standard in Kafka 4.0+, providing a streamlined foundation for future evolution.

Notes (2)
  • Key prerequisites for MSK ZooKeeper-to-KRaft upgrades

    Clusters must run Kafka 3.9.x in ZooKeeper mode for the upgrade; earlier versions require an initial upgrade to 3.9.x. Client applications need Apache Kafka client v3.0+ and must remove any ZooKeeper connection strings. Several pre-upgrade checks are required, including disabling ZooKeeper access and ensuring no under-replicated partitions.

  • Managed, multi-phase upgrade process with built-in recovery

    The upgrade involves a managed, multi-phase migration including controller quorum bootstrap, metadata migration, broker transition, and validation, with data plane operations remaining available. Amazon MSK performs rigorous safety checks and includes built-in recovery, automatically reverting the cluster if issues prevent completion.

Read the original announcement →

https://aws.amazon.com/blogs/big-data/announcing-in-place-zookeeper-to-kraft-cluster-upgrades-for-amazon-msk/

Related releases