[Webinar] How to Implement Data Contracts: A Shift Left to First-Class Data Products | Register Now
Kafka Streams
Publishing with Apache Kafka at The New York Times
At The New York Times we have a number of different systems that are used for producing content. We have several Content Management Systems, and we use third-party data and […]
Leveraging the Power of a Database ‘Unbundled’
When you build microservices using Apache Kafka®, the log can be used as more than just a communication protocol. It can be used to store events: messaging that remembers. This […]
Exactly-Once Semantics Are Possible: Here’s How Kafka Does It
I’m thrilled that we have hit an exciting milestone the Apache Kafka® community has long been waiting for: we have introduced exactly-once semantics in Kafka in the 0.11 release and […]
Building a Real-Time Streaming ETL Pipeline in 20 Minutes
There has been a lot of talk recently that traditional ETL is dead. In the traditional ETL paradigm, data warehouses were king, ETL jobs were batch-driven, everything talked to everything […]
The Future of ETL Isn’t What It Used To Be
In his book Design Patterns Explained, Alan Shalloway compares his car to an umbrella. After all, he uses both to stay dry in the rain. The umbrella has an advantage […]
Getting Started with the Kafka Streams API using Confluent Docker Images
Introduction What’s great about the Kafka Streams API is not just how fast your application can process data with it, but also how fast you can get up and running […]
Watermarks, Tables, Event Time, and the Dataflow Model
The Google Dataflow team has done a fantastic job in evangelizing their model of handling time for stream processing. Their key observation is that in most cases you can’t globally […]
Unifying Stream Processing and Interactive Queries in Apache Kafka
This post was co-written with Damian Guy, Engineer at Confluent, Michael Noll, Product Manager at Confluent and Neha Narkhede, CTO and Co-Founder at Confluent. We are excited to announce Interactive […]
Event sourcing, CQRS, stream processing and Apache Kafka: What’s the connection?
Event sourcing as an application architecture pattern is rising in popularity. Event sourcing involves modeling the state changes made by applications as an immutable sequence or “log” of events. Instead […]
Announcing Confluent Platform 3.0.1 & Apache Kafka 0.10.0.1
A few months ago, we announced the release of open-source Confluent Platform 3.0 and Apache Kafka 0.10, marking the availability of Kafka Streams — the new stream processing engine of […]
Flink vs Kafka: a Complete Comparison
This blog post is written jointly by Stephan Ewen, CTO of data Artisans, and Neha Narkhede, CTO of Confluent. Stephan Ewen is PMC member of Apache Flink and co-founder and CTO […]
Data Reprocessing with the Streams API in Kafka: Resetting a Streams Application
This blog post is the third in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10.
Log Compaction | Highlights in the Apache Kafka and Stream Processing Community | August 2016
It is August already, and this marks exactly one year of monthly “Log Compaction” blog posts – summarizing news from the very active Apache Kafka and stream processing community. Hope […]
Secure Stream Processing with the Streams API in Kafka
This blog post is the second in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10. Current […]