Elevating Kafka: Driving operational excellence with Albertsons + Forrester | Watch Webinar


똑똑! 누구세요?

« Current 2022

When managing multi-tenant Kafka clusters, the ability to identify individual clients is crucial but can be challenging to achieve.

Previously at Shopify, a single SSL certificate was used by nearly all clients to connect to our Kafka clusters. As Kafka distinguishes users based on their certificate’s subject, all clients were masked as the same user, and thus we were unable to identify who was connecting and what operations they were enacting. As a result, ensuring proper data usage and ownership was not possible, as the producers and consumers of each Kafka topic were unknown. Without this insight, protective measures such as request quotas and access-control were rendered impractical.

We set out to provide our thousands of Kafka clients with their own unique identity, by automating certificate management within our Kubernetes platform via controllers. In providing our clients with their own identities, we subsequently improved observability around data usage/ownership by implementing a custom Kafka authorizer to gather client request metrics. Lastly, we enhanced our Kafka infrastructure’s resiliency by introducing request quotas and protected our clusters from unintended operations with user access-control.

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how