Register now: The Top Five Use Cases & Architectures for Data In Motion in 2022

Securing the Message Bus with Kafka Streams

Organizations have a need to protect Personally Identifiable Information (PII). As Event Streaming Architecture (ESA) becomes ubiquitous in the enterprise, the prevalence of PII within data streams will only increase. Data architects must be cognizant of how their data pipelines can allow for potential leaks. In highly distributed systems, zero-trust networking has become an industry best practice. We can do the same with Kafka by introducing message-level security.

A DevSecOps Engineer with some Kafka experience can leverage Kafka Streams to protect PII by enforcing role-based access control using Open Policy Agent. Rather than implementing a REST API to handle message-level security, Kafka Streams can filter, or even transform outgoing messages in order to redact PII data while leveraging the native capabilities of Kafka.

In our proposed presentation, we will provide a live demonstration that consists of two consumers subscribing to the same Kafka topic, but receiving different messages based on the rules specified in Open Policy Agent. At the conclusion of the presentation, we will provide attendees with a GitHub repository, so that they can enjoy a sandbox environment for hands-on experimentation with message-level security.

Presenters

Paul Otto

Paul is a passionate technology leader and systems engineer. He has more than 20 years of professional experience, including more than 10 involving clustered systems and data engineering. He leads projects at Raft to help modernize the US Air Force by leveraging his experience at companies such as Ticketmaster.

Paul is a lifelong technologist. Some of his earliest memories involve him helping his father set up light measurement tests and troubleshooting problems with the software and firmware his father wrote. Paul spent seven years in the US Army, learning everything from electronics repair, to turbine engines. Since leaving the military, Paul completed undergraduate and graduate education, and gained a wide breadth of experience in software and systems engineering. Paul brings a creative approach to problem-solving which he attributes to his unique life experiences. When he’s not geeking out, Paul enjoys time with his wife and children.

Ryan Salcido

Ryan is a DevSecOps Engineer at Raft with a bachelor's degree in Computer Science from California State University, Sacramento. Ryan's previous experience consisted of working in the government sector as a software engineer building web applications. As a DevSecOps Engineer, Ryan focuses on developing microservices that communicate with Kafka.