Learnings of Running Kafka Tiered Storage at Scale

KIP-405 introduced tiered storage in Apache Kafka. It introduces the separation of compute and storage in brokers that improves the scalability, efficiency, and elasticity of Kafka clusters. We implemented this feature and have been running it in production for several months in different tiers of clusters at Uber.

We will talk about the following:

• The principles followed in building the feature.
• The journey of deploying and running it in our production clusters with different workloads.
• The learnings from running it in production at a large scale, that led to a few interesting features extended from KIP-405.
• The issues encountered and how we have fixed and mitigated them.