Apache Kafka Supports 200K Partitions Per Cluster
In Kafka, a topic can have multiple partitions to which records are distributed. Partitions are the unit of parallelism. In general, more partitions leads to higher throughput. However, there are […]
Elastic Scaling in the Streams API in Kafka
This blog post is the first in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10. Current […]
Deploying Apache Kafka on AWS Elastic Block Store (EBS)
Apache Kafka is designed to be highly performant, reliable, scalable, and fault tolerant. At the same time, the performance and reliability of a Kafka cluster is highly dependent on the […]
Apache Kafka Security 101
TLS, Kerberos, SASL, and Authorizer in Apache Kafka 0.9 – Enabling New Encryption, Authorization, and Authentication Features Apache Kafka is frequently used to store critical data making it one of […]
Apache Kafka 0.9 is Released
I am pleased to announce the availability of the 0.9 release of Apache Kafka. This release has been in the works for several months with contributions from the community and […]
Apache Kafka, Purgatory, and Hierarchical Timing Wheels
Apache Kafka has a data structure called the “request purgatory”. The purgatory holds any request that hasn’t yet met its criteria to succeed but also hasn’t yet resulted in an […]
Getting started with Kafka in node.js with the Confluent REST Proxy
Previously, I posted about the Kafka REST Proxy from Confluent, which provides easy access to a Kafka cluster from any language. That post focused on the motivation, low-level examples, and […]
Making Apache Kafka Elastic With Apache Mesos
This post has been written in collaboration with Derrick Harris from Mesosphere and Joe Stein, a Kafka committer. For an updated version of this article, please see Apache Mesos, Apache Kafka and […]
Hands-free Kafka Replication: A lesson in operational simplicity
Building operational simplicity into distributed systems, especially for nuanced behaviors, is somewhat of an art and often best achieved after gathering production experience. Apache Kafka‘s popularity can be attributed in […]
A Comprehensive REST Proxy for Kafka
As part of Confluent Platform 1.0 released about a month ago, we included a new Kafka REST Proxy to allow more flexibility for developers and to significantly broaden the number […]