Kora Engine, Data Quality Rules y mucho más en nuestra nueva versión del 2T'23 | Regístrese para la demostración
In our analytics use case we have a real-time ingestion pipeline where we ingest data from Kafka topics, map it to a particular type of object, perform the desired aggregations on the input records and finally write it to our database which is then available for querying. We calculate the ingestion end to end latency from the time when the record was published to Kafka until it is available for querying. The latency that is calculated is then populated to a dashboard for SLO and SLA calculation. This talk will dive into the details of calculating the end to end latency for our real time ingestion pipeline.