[Ebook] The Builder's Guide to Streaming Data Mesh | Read Now

Presentation

CERN IoT Kafka Pipelines

« Kafka Summit London 2024

This presentation outlines CERN's approach for decoding data from IoT devices connected to

a LoRa network using Apache Kafka, Kafka Connect, and Kafka Streams, along with Kaitai Struct,

a declarative binary format parsing language used for payload decoding.

We introduce CERN's IoT LoRa network deployment, our LoRa to Kafka bridge, and the processing platform based on Kaitai Struct specifications.

This platform is designed to automatically generate payload decoders and, if necessary, database schemas.

These are then used within Kafka Streams applications to process, decode, and inject the database schema into the payload data as required.

Our focus is on offering a simple, quick, and accessible method for end-users to easily develop and integrate new IoT streaming applications able to handle any binary data format from heterogeneous sensors (multi type, multi vendor) such as proximity, temperature, structural integrity, etc.

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how