How to Build a Data Mesh with Stream Governance | Join Webinar


Integrating Sparkplug IoT Edge of Network Nodes with Kafka

« Kafka Summit London 2023

In this session, we have a look at a real-world IoT project in which hundreds of residential building complexes are equipped with thousands of sensors and actuators that communicate via Kafka to an optimization system to reduce energy consumption and eventually help to protect our planet.

MQTT is a commonly used protocol in the IIoT world, especially in conjunction with the widely used Sparkplug standard. With the available MQTT source and sink connectors for Kafka Connect, one would think integrating such IoT systems with Kafka should be straightforward. But is it?

We will learn the core concepts of MQTT and the Sparkplug protocol, Kafka Connect, and Confluent’s MQTT Source and Sink connectors. Having the basics covered, we will follow a message from an IoT device to Kafka and back. We dive deeper to identify issues and see for example how to guarantee message ordering in multi-partitioned topics with a custom partitioner or single message transform (SMT).

If you want to learn about the MQTT and Sparkplug protocol, how to connect Sparkplug-based IoT devices to Kafka, or are just curious about our IoT use case then this session is perfect for you!

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how