Elevating Kafka: Driving operational excellence with Albertsons + Forrester | Watch Webinar


A Kafkaesque Raft Protocol

ยซ Kafka Summit Europe 2021

KIP-500 set the vision for Zookeeper-free Kafka. However, even without Zookeeper, the need for consensus never went away. In this talk, we will discuss one of the core communityโ€™s initiatives, a native Raft-like protocol used to ensure different brokers can agree on critical pieces of metadata such as which replicas are available for writing (i.e. partition leaders).

Specifically, we will cover the following topics:

  • Why did we abandon the external consensus and what benefits internal consensus provides.
  • How this protocol is different from standard Raft, and the critical design trade-offs we made in its implementation.
  • How the new Quorum Controller serves as the โ€œKafka control planeโ€ and how it gets integrated with the Raft protocol
  • What next steps we envisage for Kafkaโ€™s replication protocol for metadata and beyond.

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how