Kafka Cluster
Creating a Serverless Environment for Testing Your Apache Kafka Applications
If you are taking your first steps with Apache Kafka®, looking at a test environment for your client application, or building a Kafka demo, there are two “easy button” paths […]
Project Metamorphosis Month 5: Global Event Streaming in Confluent Cloud
This is the fifth month of Project Metamorphosis: an initiative that addresses the manual toil of running Apache Kafka® by bringing the best characteristics of modern cloud-native data systems to […]
How Tencent PCG Uses Apache Kafka to Handle 10 Trillion+ Messages Per Day
As one of the world’s biggest internet-based platform companies, Tencent uses technology to enrich the lives of users and assist the digital upgrade of enterprises. An example product is the […]
Scaling Apache Kafka to 10+ GB Per Second in Confluent Cloud
Apache Kafka® is the de facto standard for event streaming today. The semantics of the partitioned consumer model that Kafka pioneered have enabled scale at a level and at a […]
Elastically Scaling Confluent Platform on Kubernetes
This month, we kicked off Project Metamorphosis by introducing several Confluent features that make Apache Kafka® clusters more elastic—the first of eight foundational traits characterizing cloud-native data systems that map […]
Building Confluent Cloud – Here’s What We’ve Learned
In July 2017, Confluent launched a private preview of what would later be known as Confluent Cloud. This platform as a service product has grown rapidly; less than three years […]
15 Things Every Apache Kafka Engineer Should Know About Confluent Replicator
Single-cluster deployments of Apache Kafka® are rare. Most medium to large deployments employ more than one Kafka cluster, and even the smallest use cases include development, testing, and production clusters. […]
Dawn of Kafka DevOps: Managing Kafka Clusters at Scale with Confluent Control Center
When managing Apache Kafka® clusters at scale, tasks that are simple on small clusters turn into significant burdens. To be fair, a lot of things turn into significant burdens at […]
Apache Kafka Supports 200K Partitions Per Cluster
In Kafka, a topic can have multiple partitions to which records are distributed. Partitions are the unit of parallelism. In general, more partitions leads to higher throughput. However, there are […]
Elastic Scaling in the Streams API in Kafka
This blog post is the first in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10. Current […]
Deploying Apache Kafka on AWS Elastic Block Store (EBS)
Apache Kafka is designed to be highly performant, reliable, scalable, and fault tolerant. At the same time, the performance and reliability of a Kafka cluster is highly dependent on the […]
Apache Kafka Security 101
TLS, Kerberos, SASL, and Authorizer in Apache Kafka 0.9 – Enabling New Encryption, Authorization, and Authentication Features Apache Kafka is frequently used to store critical data making it one of […]
Apache Kafka 0.9 is Released
I am pleased to announce the availability of the 0.9 release of Apache Kafka. This release has been in the works for several months with contributions from the community and […]
Apache Kafka, Purgatory, and Hierarchical Timing Wheels
Apache Kafka has a data structure called the “request purgatory”. The purgatory holds any request that hasn’t yet met its criteria to succeed but also hasn’t yet resulted in an […]