Project Metamorphosis: Unveiling the next-gen event streaming platformLearn More

Securing Kafka

Watch Video

Kafka Summit 2016 | Operations Track

Kafka was originally developed at LinkedIn in 2010. It was originally an open system to encourage adoption; developers could easily create new data streams, add data to the pipeline, and read data as it was created. It succeeded brilliantly at encouraging developers to build new data applications, improved the reliability of systems and applications, and helped LinkedIn scale it’s logging and data infrastructure.

Unfortunately, as Kafka usage grew at LinkedIn (and at other sites), we discovered problems with a totally open system. Developers might inadvertently cause production problems when creating new Kafka streams, engineers might change the configuration of critical systems, and employees might get access to sensitive data. As Kafka has been adopted by larger enterprises with more complex security requirements, we have had to rethink our architecture.

In this talk, we will explain how we have secured Apache Kafka. We will explain the threats that Kafka Security mitigates, the changes that we made to Kafka to enable security, and the steps required to secure an existing Kafka cluster.

  • Specifically, we will cover:
  • New security features in Kafka 0.9
  • The risks and threats with a distributed data streaming system
  • Common issues with deploying a secure Kafka system
  • The access control model for Kafka
  • Configuring authentication, access control, and encryption
  • Using a secure Kafka cluster with other secure (and insecure) systems
  • Testing, monitoring and tuning a secure Kafka cluster
  • Future work in Kafka security

Speaker:

Jun Rao, Co-founder, Confluent; Kafka Summit Program Committee

Sign Up Now

Start your 3-month trial. Get up to $200 off on each of your first 3 Confluent Cloud monthly bills

New signups only.

By clicking “sign up” above you understand we will process your personal information in accordance with our Privacy Policy.

By clicking "sign up" above you agree to the Terms of Service and to receive occasional marketing emails from Confluent. You also understand that we will process your personal information in accordance with our Privacy Policy.

Free Forever on a Single Kafka Broker
i

The software will allow unlimited-time usage of commercial features on a single Kafka broker. Upon adding a second broker, a 30-day timer will automatically start on commercial features, which cannot be reset by moving back to one broker.

Select Deployment Type
Manual Deployment
  • tar
  • zip
  • deb
  • rpm
  • docker
or
Auto Deployment
  • kubernetes
  • ansible

By clicking "download free" above you understand we will process your personal information in accordance with our Privacy Policy.

By clicking "download free" above, you agree to the Confluent License Agreement and to receive occasional marketing emails from Confluent. You also agree that your personal data will be processed in accordance with our Privacy Policy.

This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising, and analytics partners.