Kafka 0.10.0 is the newest major release of Apache Kafka, which includes more than 400 JIRAs and 13 KIPs, including Kafka Streams — a simple solution for stream processing, Message Timestamps — an additional field inside the message that enables log retention and rolling as well as stream processing based on event-time, Rack Awareness — functionality to assign replicas to brokers so they can span multiple racks or availability zones, and many other new features.
Confluent Platform 3.0 introduces the first commercial product through the Confluent Platform offering — Control Center, for comprehensive management and monitoring of Apache Kafka. Read this page to learn more.
Kafka Connect contributions have been growing rapidly over the past month! Checkout this new hub for available certified connectors.
Thanks to Yelp for hosting the Kafka Meetup this month! Enrico Canzonieri and Usman Masood gave great presentations on operating Kafka at scale and executing continuous SQL queries on top of Kafka, respectively.
If you want to learn more about applying Kafka for real-time data analysis, read this article for a production use case in online anomaly detection, and also this talk about using Kafka and Samza for spam detection, and also this blog post about using Kafka and Flink to migrate your batch jobs to streaming.
Why Apache Beam for stream processing? Checkout Google and data Artisan’s perspectives.
Guozhang Wang is a PMC member of Apache Kafka, and also a tech lead at Confluent leading the Kafka Streams team. He received his Ph.D. from Cornell University where he worked on scaling data-driven applications. Prior to Confluent, Guozhang was a senior software engineer at LinkedIn, developing and maintaining its backbone streaming infrastructure on Apache Kafka and Apache Samza.