Découvrez de nombreuses nouveautés à l'occasion de notre lancement du 2e trimestre 2023 : moteur Kora Engine, règles de qualité des données, et bien plus encore | S'inscrire pour une démo
Kafka makes it possible to ingest and process large amounts of data concurrently by decoupling your data streams. This successful approach introduces some challenges when monitoring Kafka pipelines - to understand what’s happening, we need to monitor every single component and how they interact with each other.
In this talk, we will take a close look at Kafka’s architecture as well as the key infrastructure, JVM, and system metrics you should monitor for each of its components. Then, we will walk through how to diagnose common Kafka performance anomalies through observing patterns in the metrics for the various components. Finally, we will walk through setting up an open source observability pipeline with OpenTelemetry to enable you to collect and process Kafka metrics at scale.