Poison Pills in Kafka

In messaging systems, a poison pill is a special message that, when consumed by a consumer, signals that there is an issue with the message or the processing logic.

We encountered the poison pills in our production system, and to mitigate them, we implemented a design pattern to work around them. This was when we were using Kafka as a compacted topic to store data, and this can be used by any new microservice spinning up to hydrate itself.

In the 10-minute lightning talk, I will walk through the problems faced in our production system and how we handled these poison pills.