Register for Apache Kafka®, Confluent, and the Data Mesh
Tags: ZooKeeper
Why ZooKeeper Was Replaced with KRaft – The Log of All Logs
Why replace ZooKeeper with an internal log for Apache Kafka® metadata management? This post explores the rationale behind the replacement, examines why a quorum-based consensus protocol like Raft was utilized […]
Apache Kafka Made Simple: A First Glimpse of a Kafka Without ZooKeeper
At the heart of Apache Kafka® sits the log—a simple data structure that uses sequential operations that work symbiotically with the underlying hardware. Efficient disk buffering and CPU cache usage, […]
42 Things You Can Stop Doing Once ZooKeeper Is Gone from Apache Kafka
Soon, Apache Kafka® will no longer need ZooKeeper! With KIP-500, Kafka will include its own built-in consensus layer, removing the ZooKeeper dependency altogether. The next big milestone in this effort […]
Apache Kafka Lag Monitoring at AppsFlyer
This article covers one crucial piece of every distributed system: visibility. At AppsFlyer, we call ourselves metrics obsessed and truly believe that you cannot know what you cannot see. We […]
Preparing Your Clients and Tools for KIP-500: ZooKeeper Removal from Apache Kafka
As described in the blog post Apache Kafka® Needs No Keeper: Removing the Apache ZooKeeper Dependency, when KIP-500 lands next year, Apache Kafka will replace its usage of Apache ZooKeeper […]
Apache Kafka Needs No Keeper: Removing the Apache ZooKeeper Dependency
Currently, Apache Kafka® uses Apache ZooKeeper™ to store its metadata. Data such as the location of partitions and the configuration of topics are stored outside of Kafka itself, in a […]
What’s New in Apache Kafka 2.4
On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 2.4.0. This release includes a number of key new features and improvements […]
Apache Kafka Supports 200K Partitions Per Cluster
In Kafka, a topic can have multiple partitions to which records are distributed. Partitions are the unit of parallelism. In general, more partitions leads to higher throughput. However, there are […]
Distributed Consensus Reloaded: Apache ZooKeeper and Replication in Apache Kafka
This post was jointly written by Neha Narkhede, co-creator of Apache Kafka, and Flavio Junqueira, co-creator of Apache ZooKeeper. Many distributed systems that we build and use currently rely on dependencies like […]
Apache Kafka 0.8.2.1 release
The Apache Kafka community just announced the 0.8.2.1 release. This is a a bug fix release and fixes 4 critical issues reported in the 0.8.2.0 release (the full list of […]
What’s coming in Apache Kafka 0.8.2
I am very excited to tell you about the forthcoming 0.8.2 release of Apache Kafka. Kafka is a fault-tolerant, low-latency, high-throughput distributed messaging system used in data pipelines at several […]