Increasing Kafka Connect Throughput

While Apache Kafka® is the central nervous system for streaming data, Kafka Connect is the highway that allows traffic to flow in and out of Kafka without the need to write a producer/consumer. The question that arises here is how do we fit more cars (records) on this highway and get better throughput?

In this presentation, we will use JDBC source and sink connectors as examples of how to tune source/sink connectors:

• Elaborate internals of source vs sink connectors(client/producer vs consumer/client)
• List of configurations to tune for producer/consumer
• List of configurations to tune from source/sink JDBC connector
• Throughput results of tuned vs untuned
• Which metrics to monitor
• What can’t be tuned (data conversion between JDBC client and producer/consumer)

Let us Connect all of our systems through the largest possible highway!