[Webinar] Master Apache Kafka Fundamentals with Confluent | Register Now
Stream Processing
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Unlock Cost Savings with Freight Clusters–Now in General Availability
Confluent Cloud Freight clusters are now Generally Available on AWS. In this blog, learn how Freight clusters can save you up to 90% at GBps+ scale.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Watermarks, Tables, Event Time, and the Dataflow Model
The Google Dataflow team has done a fantastic job in evangelizing their model of handling time for stream processing. Their key observation is that in most cases you can’t globally […]
Unifying Stream Processing and Interactive Queries in Apache Kafka
This post was co-written with Damian Guy, Engineer at Confluent, Michael Noll, Product Manager at Confluent and Neha Narkhede, CTO and Co-Founder at Confluent. We are excited to announce Interactive […]
Event sourcing, CQRS, stream processing and Apache Kafka: What’s the connection?
Event sourcing as an application architecture pattern is rising in popularity. Event sourcing involves modeling the state changes made by applications as an immutable sequence or “log” of events. Instead […]
Flink vs Kafka: a Complete Comparison
This blog post is written jointly by Stephan Ewen, CTO of data Artisans, and Neha Narkhede, CTO of Confluent. Stephan Ewen is PMC member of Apache Flink and co-founder and CTO […]
Data Reprocessing with the Streams API in Kafka: Resetting a Streams Application
This blog post is the third in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10.
Secure Stream Processing with the Streams API in Kafka
This blog post is the second in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10. Current […]
Elastic Scaling in the Streams API in Kafka
This blog post is the first in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10. Current […]
Distributed, Real-time Joins and Aggregations on User Activity Events using Kafka Streams
In previous blog posts we introduced Kafka Streams and demonstrated an end-to-end Hello World streaming application that analyzes Wikipedia real-time updates through a combination of Kafka Streams and Kafka Connect. […]
Introducing Kafka Streams: Stream Processing Made Simple
I’m really excited to announce a major new feature in Apache Kafka v0.10: Kafka’s Streams API. The Streams API, available as a Java library that is part of the official […]
How I Learned to Stop Worrying and Love the Schema
The rise in schema-free and document-oriented databases has led some to question the value and necessity of schemas. Schemas, in particular those following the relational model, can seem too restrictive, […]
Real-time full-text search with Luwak and Samza
This is an edited transcript of a talk given by Alan Woodward and Martin Kleppmann at FOSDEM 2015. Traditionally, search works like this: you have a large corpus of documents, […]
Putting Apache Kafka To Use: A Practical Guide to Building an Event Streaming Platform (Part 2)
This is the second part of our guide on streaming data and Apache Kafka. In part one I talked about the uses for real-time data streams and explained the concept of […]
Putting Apache Kafka To Use: A Practical Guide to Building an Event Streaming Platform (Part 1)
Data systems have mostly focused on the passive storage of data. Phrases like “data warehouse” or “data lake” or even the ubiquitous “data store” all evoke places data goes to […]
Stream Processing, CEP, Event Sourcing, and Data Streaming Explained
Some people call it stream processing. Others call it event streaming, complex event processing (CEP), or CQRS event sourcing. Sometimes, such buzzwords are just smoke and mirrors, invented by companies […]