Kafka In the Cloud: Why It’s 10x Better With Confluent | Get free eBook

I Don’t Always Test My Streams, But When I Do, I Do it in Production

Testing stream processing applications (Kafka Streams and ksqlDB) isn’t always straightforward. You could run a simple topology manually and observe the results. But how about repeatable tests that you can run anytime, as part of a build without a Kafka cluster or Zookeeper? Luckily, Kafka Streams includes the TopologyTestDriver module (and ksqlDB includes test-runner) that allows you to do precisely that. After learning this, no doubt, your test coverage is sky-high! However, how will your stream processing application perform once deployed to production? You might depend on external resources such as databases, web services, and connectors. Viktor will start this talk covering the basics of unit testing of Kafka Streams applications using TopologyTestDriver. Viktor will also look at some popular open-source libraries for testing streams applications. Viktor demonstrates TestContainers, a Java library that provides lightweight, disposable instances of shared databases, Kafka clusters, and anything else that can run in a Docker container and how to use it for integration testing of processing applications! And lastly, Viktor will show ksqlDB’s test-runner to unit test your KSQL applications.


Viktor Gamov

Viktor Gamov is a developer advocate at Confluent, the company that makes an event streaming platform based on Apache Kafka. Back in his consultancy days, Viktor developed comprehensive expertise in building enterprise application architectures using open source technologies. He enjoys helping architects and developers design and develop low-latency, scalable, and highly available distributed systems. He is a professional conference speaker on distributed systems, streaming data, JVM, and DevOps, and he regularly speaks at events like JavaOne, Devoxx, OSCON, and QCon. He co-authored O’Reilly’s Enterprise Web Development and writes on the Confluent blog.