A simple solution to stream processing, powered by Apache Kafka™
Kafka Streams is an open-source solution that you can integrate into your application to build and execute powerful stream processing functions. If you use Kafka for stream data transport, Kafka Streams can immediately add stream processing capabilities to your application without the burden of adding an entirely separate distributed processing cluster for another stream processing framework. Whether you have an IoT application, a monitoring function, a complex continuous query, or you are tracking inventory changes, Kafka Streams enables you to build your application with ease.
Kafka Streams is included with the Apache Kafka release v 0.10 as well as the Confluent Platform v3.0.Download Confluent Platform
Stream processing frameworks are very powerful tools. With this immense capability however, comes some complexity; they need their own dedicated cluster of machines, and often rely on a distributed database to perform look-ups and aggregations. Also, your application likely will be required to handle re-processing of old data separately.
The outcome? A lot of moving parts that need to be kept in synchronization with each other to make the application work.
Meet Kafka Streams.Read the blog about Kafka Streams
Microservices are often built upon a Kafka-based data stream. Instead of a large monolithic application, microservices deliver small, decoupled processes that execute a constrained feature set against a stream. These are ideal for Kafka Streams because they work on real-time streams, depend on reliable message delivery, and might not warrant the cost of deploying a separate stream processing framework.
One of the best ways to take advantage of stream data is to compare or analyze data in different streams against each other, or to join streams together to create more meaningful information. Continuous queries are used to automate real-time intelligence at scale across an organization. Kafka Streams allows functions of this scale to be implemented with a low infrastructure footprint and entirely within the Kafka environment.
Continuous transformations modify or aggregate the data in a stream. These humble but critical processes are low level services that prepare data for real-time analysis. Their small scale is well-suited to a lightweight solution like Kafka Streams, which supports both stateful and stateless processing.
The endless search for potentially meaningful anomalies in organizational data is moving into a real-time environment. If data pipelines exist, implementing event-triggered processes against them using Kafka Streams is a simple extension of the streaming infrastructure Kafka already provides.
Have someone from Confluent contact you.Contact Us