Build Predictive Machine Learning with Flink | Workshop on Dec 18 | Register Now

Confluent and Apache Kafka in 2017

Written By

2017 was an amazing year for Apache Kafka® as an open source project and for Confluent as a company. I wanted to take moment to thank all the contributors and users of the open source project, as well as our customers and partners and the people here at Confluent for contributing to such an outstanding year.

One of the most impressive things for me to see in 2017 was the degree to which companies were starting to view Kafka as a strategic data platform, and the scope and centrality of the applications they were developing. We got to work with companies reimagining retail, healthcare, cars, travel, banking, and virtually every other industry you can imagine, around streams of real-time events. I could not be more excited about this: our vision for Kafka is for it to grow into a central nervous system of the modern company, and we seem to be well on our way to that vision.

We’re already working on a new and exciting set of challenges for 2018, but I thought it would be fun to celebrate a couple of the big steps we took last year. Here are few.

Confluent Milestones

  • We grew our subscription bookings by over 4x year over year with 98 percent customer satisfaction. And yes this is on top of the amazing growth from last year.
  • Confluent more than doubled in size, adding over one hundred new employees to the company. We also added several new members of our leadership team, including Simon Hayes, Vice President of Corporate and Business Development; Sarah Sproehnle, Vice President of Customer Success; and Ivan Ernest, Chief People Officer.
  • We released Confluent Cloud, an Apache Kafka as a Service offering. This offering lets us manage, monitor, upgrade and expand Kafka and Confluent Platform for you as an elastic cloud offering. So, rather than build Kafka-expertise in-house you can get Kafka at the click of a button.
  • We released KSQL, a streaming SQL engine that enables continuous, interactive SQL queries on Apache Kafka.
  • We dramatically expanded the capabilities of Confluent Platform, our enterprise distribution of Kafka that lets you manage, monitor and scale Kafka in your own datacenter.

Apache Kafka Community Milestones

  • The community participated in two Kafka Summits in San Francisco and New York City with over 1,300 attendees and 30 sponsors across the events. There will be two Kafka Summits in 2018: one in San Francisco and the first-ever international event in London.
  • Apache Kafka is growing into a spectacular open source project. We had more than 19,500 meetup members and tens of thousands of production users. Development velocity continues to accelerate with over 75 new core design proposals (KIPs) completed this year.
  • The community announced the highly-anticipated release of exactly-once semantics, a set of features enabling idempotence and transaction to ensure correct reading, writing and processing for continual data streams. This was a huge effort involving design and coding across the larger Kafka development community.
  • Kafka finally went 1.0, a huge milestone for the whole community.

We’ve come a long way since the earliest Kafka prototypes we did at LinkedIn, but in so many ways the most exciting things lay ahead of us. Stay tuned for what’s next.

  • Jay Kreps is the CEO and co-founder of Confluent, the foundational platform for data in motion built on Apache Kafka. As a pioneer in a new category of data infrastructure, Confluent’s significant growth underscores the importance of data in motion across all industries. Prior to Confluent he was the lead architect for data and infrastructure at LinkedIn. He is the initial developer of several open source projects, including Apache Kafka.

Did you like this blog post? Share it now