Confluent
Compatibility Testing For Apache Kafka
Connecting to Apache Kafka

Compatibility Testing For Apache Kafka

Jay Kreps

Testing is one of the hardest parts of building reliable distributed systems. Kafka has long had a set of system tests that cover distributed operation but this is an area that is simply never good enough.

At Confluent and Cloudera we’ve both been working on improving the testing capabilities for Kafka.

An area of particular importance is compatibility. Companies that want to build reliable data real-time data flow and processing around Kafka need to be able to do so without fear of incompatibilities that could arise release to release or between versions of Kafka from different vendors.

We’re announcing today a project with the folks at Cloudera and the rest of the open source community to develop high quality tests to certify API and protocol compatibility between versions and distributions.

We’ll be doing this as part of the normal Apache development process, much as we do any other Kafka development.

We think ensuring this kind of compatibility is one of the key aspects of building a healthy ecosystem of systems, applications, and processing frameworks, that is the core of our stream data platform goal.

Subscribe to the Confluent Blog

Subscribe
Email *

More Articles Like This

Bill Bejeck

Predicting Flight Arrivals with the Apache Kafka Streams API

Bill Bejeck . .

Kafka Streams makes it easy to write scalable, fault-tolerant, and real-time production apps and microservices. This post builds upon a previous post that covered scalable machine learning with Apache Kafka, ...

streaming platform around apache kafka
Gwen Shapira

The Future of ETL Isn’t What It Used To Be

Gwen Shapira . .

In his book Design Patterns Explained, Alan Shalloway compares his car to an umbrella. After all, he uses both to stay dry in the rain. The umbrella has an advantage ...

Eno Thereska

Watermarks, Tables, Event Time, and the Dataflow Model

Eno Thereska . .

This post was co-written with Michael Noll, Product Manager, Confluent, and Matthias J. Sax, Engineer, Confluent. The Google Dataflow team has done a fantastic job in evangelizing their model of handling ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Try Confluent Platform

Download Now