The benefit of real-time data can be measured by how frequently the data in question changes, nowhere is this more apparent than threat detection. Responding to an ever changing landscape of attacks and exploits requires a system that can not only handle the scale and dynamic nature of the data but also a dynamically changing set of detection rules. We developed Confluent SIGMA, an open source project built on Kafka Streams for the open SIGMA DSL, to handle real-time rule additions and modifications. In this talk we will cover:
- The architecture of our Kafka Streams layer that makes it possible to use external data feeds as rule input
- How we handle dynamic criteria for joins and filters
- Best practices for writing dynamic rule engines in Kafka Streams
- Upcoming improvements to Kafka Streams to support versioned rules
Although Confluent SIGMA focuses on cyber threat detection this same pattern can also be applied to any DSL (domain specific language) that would benefit from real-time stream processing. After attending you will have the framework to drive dynamic rules through Kafka Streams for any use case that might require it.
Presenter
Will LaForest
ConfluentIn his current position, Mr. LaForest works with customers across a broad spectrum of industries and government enabling them to realize the benefits of a data in motion and event driven architecture. He is passionate about innovation in data technology and has spent 26 years helping customers wrangle data at massive scale. His technical career spans diverse areas from software engineering, NoSQL, data science, cloud computing, machine learning, and building statistical visualization software but began with code slinging at DARPA as a teenager. Mr. LaForest holds degrees in mathematics and physics from the University of Virginia.
Presenter
Michael Peacock
ConfluentMr. Peacock is an accomplished professional with 20+ years of experience leading new business activities for technical support and developing and designing high-available, real-time, mission-critical software across various Government programs. Throughout his career, he has been responsible for supporting leadership teams to define capabilities, technical landscape, and growth strategies for the organization and has identified and addressed strategic goals to support diversification and growth by providing innovative solutions for current and emerging customers.
As a Federal Solutions Engineer at Confluent, Mr. Peacock works directly with multiple organizations to understand their architecture, and recommend solutions to modernize and implement advanced technologies in the areas of real-time event streaming, AI/ML, edge computing, and cloud migration strategies.