[Webinar] Q1 Confluent Cloud Launch Brings You the Latest Features | Register Now

Presentation

Simplifying Asynchronous Service Communication with Kafka, Debezium and the Outbox Pattern

« Kafka Summit Americas 2021

Loggi is one of the Brazilian unicorns which is transforming logistics through technology to connect the entire country. At Loggi, we've widely used the CDC pattern with Debezium + Kafka to capture the changes from our main database as a way of communication between services, as an Event Notification pattern. Later, we concluded this wasn't enough, since we often need to join information from multiple tables to take an action. In this talk, we describe this journey, and how we leverage Debezium’s Outbox pattern to transactionally emit events following the Event-Carried State Transfer pattern, where all the needed information is sent to Kafka at once, therefore avoiding consumer requests to enrich the event data. Additionally, we describe an Event Broker service we’ve implemented to transform Kafka’s consumption from pull-based to push-based, helping other teams to integrate their services into this new architecture faster.

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how