Towards Client-Side Field-Level Cryptography

Apache Kafka offers several security features ranging from authentication and authorization mechanisms to over-the-wire encryption. This notwithstanding, end-to-end encryption between Kafka-based client applications, which fully protects payloads from fraudulent access at the broker's side can still be considered a blind spot.

After highlighting the main benefits of explicit data-at-rest protection, this session discusses in-depth how to selectively encrypt and decrypt sensitive payload fields in the context of streaming data pipelines built upon Apache Kafka Connect and ksqlDB apps. In particular, an ecosystem community project named Kryptonite for Kafka - written and open-sourced by the speaker - is introduced.

During this demo-driven talk, you will experience how to benefit from

• a configurable single message transformation (SMT) that lets you perform encryption and decryption operations in Kafka Connect worker nodes without any additional code and

• a custom user-defined function (UDF) for ksqlDB to conveniently encrypt and decrypt specific columns in your SQL-based stream processing apps

Client-side field-level cryptography makes streaming data pipelines more secure by safeguarding your most sensitive and precious data against any form of uncontrolled or illegal access once it hits the Apache Kafka brokers.