Migrating a core technology like Kafka can be daunting. Whether it is moving from queues to stream processing, Apache Kafka to Confluent Platform or even Confluent Platform to Confluent Cloud there are many obstacles that need to be overcome, both expected and unexpected.
In this talk we will go over our experience in migrating all these technologies and the pitfalls and powerups that we encountered along the way.
We will touch on the planning processes we went through for each migration and the time we needed to get each migration from conception to deployment in production. We will also look at:
Producer and consumer code and configuration changes
Ideas on automation
Possible architecture changes
Lastly, we will touch on the business conversations that were needed to get the green light for each of these migrations, and the decisions made for migrating to each technology.
We will analyze the things we did right along our migration journeys and the things we would improve for the next time we need to migrate a system.