Server Side Assignors for Consumer Groups : KIP-848

In this talk, we delve into a specific detail of the latest evolution of Kafka's consumer group protocol as introduced in KIP-848 - Server-Side Assignors. This transformation not only streamlines client operations but also introduces a new dimension of efficiency and flexibility in message processing.

Before exploring the features of server-side assignors, it's essential to understand the fundamental concept of Kafka consumer groups. In Kafka, a consumer group is a collection of consumers working together to process data from one or more topics. The assignors are responsible for dividing the partitions of topics subscribed to by the group - among the consumers.

The server side assignor is pluggable and the client can choose the one that it wants to use by providing its name in the heartbeat request. There are two types of assignors configured in the coordinator: Range and Uniform.

The Range Assignor, known for its ability to distribute partitions across consumers evenly, ensures that each consumer handles at least one partition per topic, making it an ideal choice for scenarios with fluctuating topic demands. Conversely, the Uniform Assignor, which comes in two flavors – Optimized and General – adopts a more intelligent approach, selecting the most suitable assignment strategy based on the group's subscription pattern.

By the end of this session, attendees will gain a deeper understanding of Kafka's new server-side assignors, equipped with the knowledge to make informed decisions for optimizing their consumer group configurations.