Simplify & secure Kafka at scale—see what’s new in Confluent Platform 8.0 | Register Now

Anmeldung Kontakt

Demo ansehenKostenlos loslegen
Demo ansehenKostenlos loslegen

Log Compaction | Highlights in the Kafka and Stream Processing Community | November 2015

Verfasst von

kafka-logo-wide (1)

The Apache Kafka community just concluded its busiest month ever. As we are preparing for the upcoming release of Kafka 0.9.0.0, the community worked together to close a record number of bugs and added quite a few last-minute improvements to new features. Here’s a summary of some of the community highlights:

  • SASL/Kerberos-based authentication was added to Kafka and will be included in the 0.9.0.0 release
  • The new Kafka Consumer has a client-side partition-assignment support, allowing consumer groups to implement their own partition assignment strategy, in addition to those provided in Apache Kafka. You can read the very detailed design document or take a look at the patch.
  • Huge improvements on the Copycat export/import feature – Copycat now works in distributed mode and there is a nice REST API too.
  • The community also worked hard on making Kafka more testable. In the last few month the old spagetti-like system tests slowly got replaced by the more modular Ducktape-based integration tests. Last week we removed the old tests completely and moving forward we will only maintain the new test suite.
  • This is the month of the Kafka Meetup! New Kafka meetups are popping up in cities all over the world including Seattle, Austin and London. I encourage you to attend if you happen to be in one of those cities.
  • If you are a Kafka internals geek, you’ll enjoy reading the deep details behind Kafka’s delayed response mechanism.
  • For those interested in the latest in stream processing, Data Artisans shared interesting highlights from the first Apache Flink conference.
  • Remember that the Kafka Summit “Call for Proposals” is open so make sure you get your entry in before the January 11, 2016 deadline. If you’re planning to attend and have ideas of sessions you’d like to see on the program tweet us @ConfluentInc #kafkasummit to let us know.

If you are interested in contributing to Apache Kafka, check out our contributor guide to help you get started.

Apache Kafka

Download the Confluent Platform 

  • Gwen Shapira is a Software Enginner at Confluent. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. She currently specialises in building real-time reliable data processing pipelines using Apache Kafka. Gwen is an Oracle Ace Director, an author of books including “Kafka, the Definitive Guide”, and a frequent presenter at data related conferences. Gwen is also a committer on the Apache Kafka and Apache Sqoop projects.

Ist dieser Blog-Beitrag interessant? Jetzt teilen

Solving ETL Challenges with Apache Kafka®, Confluent Tableflow and Zero ETL

Solving ETL Challenges with Apache Kafka®, Confluent Tableflow and Zero ETL

Confluent Tableflow unifies operational and analytical data by integrating Kafka with zero ETL, leveraging open table formats such as Iceberg and Delta Lake. It offers advantages over traditional zero ETL by enabling data reuse, schema decoupling, and better scalability, streamlining data sharing.

Introducing KIP-848: The Next Generation of the Consumer Rebalance Protocol

Introducing KIP-848: The Next Generation of the Consumer Rebalance Protocol

Big news! KIP-848, the next-gen Consumer Rebalance Protocol, is now available in Confluent Cloud! This is a major upgrade for your Kafka clusters, offering faster rebalances and improved stability. Our new blog post dives deep into how KIP-848 functions, making it easy to understand the benefits.