Introducing Streamhouse: the open data architecture for AI | Learn More
Have you ever considered how the throughput of your Kafka connectors could affect your real-time data processing capabilities, or what bottlenecks might exist in your current configuration? In this lightning talk, we dive into the world of performance testing of connectors and fine-tuning configurations for optimal performance. We also go over benchmarks for performance of a few connectors.
We'll start by introducing our performance framework, from the initial setup of resources, load generation, right down to metrics collection and the final teardown of resources. Central to our presentation will be various performance test scenarios for S3 Sink Connector and some widely used Change data capture source connectors. The talk will also touch upon chaos testing and the implications it has on the performance of connectors. We will pinpoint various elements external to connectors that play a pivotal role in shaping their performance.
By the end of the session, attendees will walk away with invaluable insights into performance testing of Kafka Connectors, acquiring the know-how to optimize their connector configurations, and recognizing scenarios that could detrimentally impact connector performance.