Big Data

The rise of IoT devices means that we have to collect, process, and analyze orders of magnitude more data than ever before. As sensors and devices become ever more ubiquitous, […]

Is event streaming or batch processing more efficient in data processing? Is an IoT system the same as a data analytics system, and a fast data system the same as […]

This post was originally published at the Codecentric blog with a focus on “old” join semantics in Apache Kafka versions 0.10.0 and 0.10.1. Version 0.10.0 of the popular distributed streaming […]

Introducing a Kafka Sink Connector for PostgreSQL from JustOne Database, Inc. JustOne Database is great at providing agile analytics against streaming data and Confluent is an ideal complementary platform for delivering those messages...

I’m really excited to announce a major new feature in Apache Kafka v0.10: Kafka’s Streams API. The Streams API, available as a Java library that is part of the official […]

TLS, Kerberos, SASL, and Authorizer in Apache Kafka 0.9 – Enabling New Encryption, Authorization, and Authentication Features Apache Kafka is frequently used to store critical data making it one of […]

Many of today’s most popular Big Data software projects such as Apache Hadoop and Apache Kafka are managed under the umbrella of the Apache Software Foundation. Hence a formidable way […]

I am very excited that LinkedIn’s deployment of Apache Kafka has surpassed 1.1 trillion (yes, trillion with a “t”, and 4 commas) messages per day. This is the largest deployment of Apache […]

This post was jointly written by Neha Narkhede, original co-creator of Apache Kafka, and Flavio Junqueira, co-creator of Apache ZooKeeper. Many distributed systems that we build and use currently rely on dependencies like […]

There has been much renewed interest in using log-centric architectures to scale distributed systems that provide efficient durability and high availability. In this approach, a collection of distributed servers can […]

One of the things I realised while doing research for my book is that contemporary software engineering still has a lot to learn from the 1970s. As we’re in such […]

Apache Kafka is widely used to enable a number of data intensive operations from collecting log data for analysis to acting as a storage layer for large scale real-time stream […]

Building operational simplicity into distributed systems, especially for nuanced behaviors, is somewhat of an art and often best achieved after gathering production experience. Apache Kafka‘s popularity can be attributed in […]

This is a repost of a recent article that I wrote for ODBMS. In the last few years, there has been significant growth in the adoption of Apache Kafka. Current […]