What Is ZooKeeper’s Role in Apache Kafka®?

Apache Kafka® Without ZooKeeper: Introduction to KRaft

As of Kafka 3.3+, ZooKeeper has been deprecated in favor of a new metadata management system called KRaft, which stands for Kafka Raft metadata mode. With KRaft, Kafka no longer requires ZooKeeper to manage the cluster. Instead, Kafka’s control plane now has its own built-in way to handle metadata, leader election, and broker coordination.

KRaft improves Kafka in several ways:

• Simplifying the architecture by removing the need for a separate ZooKeeper service

• Improving scalability because metadata updates are faster and more efficient

• Reducing operational complexity, making installation, upgrades, and monitoring easier

• Increasing reliability through faster failover and a more integrated control plane
  
  

With KRaft, Kafka becomes more self-contained and easier to operate, especially in large-scale or cloud environments.

Why Is Kafka Replacing ZooKeeper?

Kafka is moving away from ZooKeeper, because it became difficult to scale, operate, and manage in large, modern streaming environments. As Kafka deployments grew, ZooKeeper’s limitations made cluster management more complex than necessary.

Let’s review the main reasons the community began looking for an alternative way to manage Kafka metadata in 2019.

Scaling Challenges With ZooKeeper

ZooKeeper was never designed for the high-rate metadata updates that large clusters require. As Kafka clusters scaled vertically and added more topics, more partitions, and more brokers, ZooKeeper became a Kafka bottleneck:

• Metadata write throughput was limited

• Controller failover took longer

• Large clusters became harder to expand

Operational Complexity

Running ZooKeeper meant maintaining two distributed systems: Kafka itself and a separate ZooKeeper ensemble. This increased operational overhead, requiring teams self-managing Kafka to take on:

• Extra configuration and maintenance

• Additional monitoring and alerting

• More complicated upgrades

• Strict dependency between Kafka and ZooKeeper versions

This added complexity often contributed to production issues, especially in fast-growing clusters.

Latency and Failover Delays

ZooKeeper stored metadata outside Kafka. So when a broker failed or leadership changed, the controller had to update metadata through ZooKeeper, causing delays that create reliability issues at scale, including:

• Slower failover between client applications

• Longer recovery during outages

• Reduced cluster responsiveness under load

ZooKeeper vs KRaft: A Complete Side-by-Side Comparison

The Kafka community wanted a simpler, unified architecture where Kafka manages its own metadata. This led to the creation of KRaft (Kafka Raft metadata mode), a consensus protocol which allows Kafka to manage the control plane using its own internal metadata quorum instead of ZooKeeper. 

KRaft was introduced in KIP-500 to remove Kafka’s dependency on ZooKeeper and move metadata management into the Kafka cluster. Running Kafka in KRaft mode allows you to unify the control plane, rather than splitting it between two different systems.

The table below compares ZooKeeper vs. KRaft-based across architecture, operations, and security.

Side-by-Side Comparison of ZooKeeper vs. KRaft

Do You Still Need ZooKeeper for Kafka Today?

Whether you still need ZooKeeper today depends on the version of Kafka you or your Kafka service is running, but newer versions (3.3+) no longer require it.

While ZooKeeper has been the backbone of Kafka for over a decade, the ecosystem is aggressively moving toward KRaft (Kafka Raft Metadata mode), which eliminates ZooKeeper entirely. Here is the breakdown by situation:

• For New Clusters: If you are deploying a new Kafka cluster today (using Kafka 3.3+), you should default to KRaft mode. It simplifies operations, deployment, and monitoring.

• For Existing Clusters: Yes, for now. If you are running an older version of Kafka or have not yet executed a migration, you still need ZooKeeper—for now. However, you must start planning your exit strategy.

Key Timeline and Versions as of December 2025

For a deep dive into the metadata architecture, refer to Confluent’s KRaft documentation.

A Checklist for ZooKeeper-to-KRaft Migrations for Your Kafka Clusters

Migrating a running cluster from ZooKeeper to KRaft without downtime is a significant operation that requires a "bridge release." This allows the cluster to run in a dual-mode where it temporarily maintains metadata in both ZooKeeper and the KRaft controller quorum.

To ensure a seamless transition before you upgrade to Kafka 4.0, Kafka 4.1, or subsequent releases, follow this structured approach:

• Audit Your Ecosystem: Identify all client applications and tools relying on direct ZooKeeper access. These must be refactored to communicate with Kafka brokers directly, as the ZK port will eventually close.

• Upgrade to a Bridge Release: Update your cluster to a Kafka version that supports migration. Kafka 3.5 or 3.6 is recommended for the most stable migration tools.

• Provision Controller Nodes: Deploy the new KRaft controller nodes that will eventually take over metadata management.

• Enable Dual-Write Mode: Configure the cluster to write metadata to both ZooKeeper and the new KRaft controllers. This allows the systems to synchronize.

• Rollover Brokers: Restart brokers in KRaft-migration mode. They will begin reading metadata from the KRaft quorum instead of ZooKeeper.

• Finalize Cutover: Once all metrics are stable, disable the ZooKeeper connection. The cluster is now running purely on KRaft.

• Decommission ZooKeeper: Safely shut down and remove the ZooKeeper servers from your infrastructure.

Warning: Do NOT attempt to upgrade directly to Kafka 4.0 running Kafka versions older than 3.3 or running versions 3.3-3.9 in ZooKeeper mode. You must pass through a "bridge release" to perform the migration first.

What Is Confluent’s Approach to the ZooKeeper to KRaft Transition?

As organizations move from ZooKeeper-based Kafka to KRaft, many teams look for a safer and more predictable migration path. Confluent provides managed Kafka on KRaft on Confluent Cloud, along with tools, guides, and automation to make the transition easier and lower risk—especially for production environments.

Learn more about how Confluent is navigating the ZooKeeper-to-KRaft migration and supporting its users and customers through this transition:

• Docs: 

  • Confluent Platform Migration Guide 

  • KRaft Migration Guide for Confluent Platform Using Confluent for Kubernetes

  • KRaft Migration Check Utility 

• Blog: “Confluent Cloud Is Now 100% KRaft and You Should Be Too”

• Confluent Developer: Kafka Architecture Course – The Control Plane 

Try Confluent Cloud for Free