Introducing Confluent labs Parallel Consumer client

Consuming messages in parallel is what Apache Kafka® is all about, so you may well wonder, why would we want anything else? It turns out that, in practice, there are a number of situations where Kafka’s partition-level parallelism gets in the way of optimal design.

This session will go over some of these types of situations that can benefit from parallel message processing within a single application instance (aka slow consumers or competing consumers), and then introduce the new Parallel Consumer labs project from Confluent, which can improve functionality and massively improve performance in such situations.

It will cover the

• Different ordering modes of the client
• Relative performance improvements
• Usage with other components like Kafka Streams
• An introduction to the internal architecture of the project
• How it can achieve all this in a reassignment friendly manner