Go Interactive -- A Deep Dive into Interactive Queries in Kafka Streams

Interactive Queries (IQ) is one of the most differentiating features in Kafka Streams compared to other stream processing engines. IQ allows to directly query the states created by a Kafka Streams application from outside the application. By leveraging IQ, users can build in a few lines of code -- for example -- distributed lookup tables that Kafka Streams updates automatically from a Kafka topic without the need to create custom solutions to sharding, load distribution, and resiliency. IQ makes Kafka Streams an invaluable building block for event-driven microservices.

In this talk, I will introduce IQ including a brief history of IQ and a couple of use cases. Then, I will compare IQ version 1 with the improved IQ version 2 and talk about the motivations behind developing the newer version. For both versions, I will dive into the internals of IQ by following an IQ request into Kafka Streams. I will show how Kafka Streams accepts an IQ request, produces the corresponding response, and returns it to the user. Finally, I will give an overview about recent KIPs related to IQ, review the current limitations of IQ, and look into the future of IQ.