Mastering Kafka Streams and ksqlDB

“Stream processing has never been a more essential programming paradigm. Mastering Kafka Streams and ksqlDB illuminates the path to succeeding at it.” -Jay Kreps, Co-creator of Apache Kafka and Co-founder and CEO of Confluent

We live in a world of exponential data growth, and businesses are increasingly built around events - the real-time data in a company. But what is the right infrastructure for harnessing this real-time data? What are the best technologies to integrate databases and process, enrich, and transform data in real-time?

Enter Kafka Streams and ksqlDB.

About Kafka Streams:

Kafka Streams is a stream processing library built for Apache Kafka. It allows companies to process data in real-time, enabling resilient stream processing operations like filters, joins, maps, aggregations, and other transformations. It also gives companies the option to perform stateful stream processing by defining the underlying topology.

About ksqlDB:

ksqlDB is a new kind of database purpose-built for stream processing apps. It gives you one mental model, in SQL, for doing everything you need with data in motion: data acquisition, processing, and querying. ksqlDB has just one dependency: Apache Kafka®.

While working with unbounded and fast-moving data streams has historically been difficult, Kafka Streams and ksqlDB make building stream processing applications easy and fun.

In this practical guide, Mitch Seymoure shows software engineers how to use these tools to build highly scalable stream processing applications for moving, enriching, and transforming large amounts of data in real-time.

Learn important stream processing concepts, use cases, and several interesting business problems. You’ll learn the strengths of both Kafka Streams and ksqlDB to help you choose the best tool for each unique stream processing project. Non-Java developers will find the ksqlDB path to be an especially gentle introduction to stream processing.

• Learn the basics of Kafka and the pub/sub communication pattern
• Get a full overview of Kafka Streams, streaming architecture, its operational features, and how to get started
• Learn how to build stateless and stateful stream processing applications using Kafka Streams and ksqlDB
• Perform advanced stateful operations, including windowed joins and aggregations
• Understand how stateful stream processing works under the hood
• Learn about ksqlDB’s data integration features, powered by Kafka Connect
• Work with different types of collections in ksqlDB and perform push and pull queries
• Learn advanced ksqlDB concepts like data modeling, streams and tables, applying stateless transformations, joins, aggregations, and more
• Deploy your Kafka Streams and ksqlDB applications to production