Transactions were added to Apache Kafka with KIP-98. While much of the protocol remains intact, transactions in Kafka have evolved over time to handle edge cases and errors found over the years. KIP-890 hopes to cover most of the remaining gaps in the protocol. This talk will give a refresher on transactions and idempotency and chronicle the various KIPs that improved the protocol over the years. We will also discuss the problem of hanging transactions and how KIP-890 hopes to solve it as well as strengthen the transactional protocol overall.
Presenter
Justine Olshan
ConfluentJustine graduated from Carnegie Mellon University in 2020 with a degree in computer science. During summer 2019, she was a software engineering intern at Confluent where she worked on improving the Apache Kafka® producer. After graduating, she returned full time to Confluent and continues to work on improving Kafka through various KIPs including KIP-516 which introduced topic IDs to Kafka and KIP-890 which strengthened the transactional protocol. She became an Apache Kafka committer in 2022 and PMC member in 2023.