Kafka Summit NYC: Streaming Pipelines Track – What to Expect

It is just a few weeks out until Kafka Summit NYC! Since we’re on the program committee for this event and are also the track leads for the Streaming Pipelines track, we’ve felt it would be fun to share how we picked the talks for the track and also share a few of the sessions we’re most excited to see in May.

For background, this track was designed for developers and users to discuss how they’ve used Apache Kafka^® to build integrated data architectures and expand their use cases to the cloud.

When discussing the track as the program committee, we sought to cultivate a group of practitioners who could share their experiences, successes and lessons with the greater Kafka and stream processing community. We looked for compelling Kafka use cases that would pique our interest, the topic would appeal to at least 30% of Summit attendees, and the abstracts had strong titles with clear details on why members of the community would care to hear the talk. From a holistic view, all the sessions in the track should cover various aspects and components of the Kafka ecosystem. Sounds easy enough, right?

There were more than 100 talks submitted for Kafka Summit New York. With this number of quality talks, it was a daunting task to narrow them down to 8 for each track. At the end of the process, we’re proud of the session abstracts that were chosen for this track and hope you will agree once you have a chance to attend them.

We’re excited to highlight our ‘can’t-miss’ sessions in this track:

	Every Message Counts: Kafka as a Foundation for Highly Reliable Logging at Airbnb Youssef Francis, Software Engineer & Jun He, Software, Engineer AirbnbAirbnb is so popular that everyone will be familiar with their use-case and are therefore a great way to introduce the audience to a high-reliability Kafka architecture.
	Billions of Messages a Day: Yelp’s Real-time Data Pipeline Justin Cunningham, Technical Lead, YelpLast year, Yelp described their Kafka-based real-time data pipeline in a multi-part blog series. We’ve since seen many companies implementing a very similar data pipeline and it always turned out to be very successful. We want to expose a large audience to their architecture pattern for data pipelines since it worked so well for many different companies.
	California Schemin’! How the Schema Registry has Ancestry Basking in Data Chris Sanders, Director, Data Warehouse and Visualization, AncestryAncestry had unique data integration challenges and taken a very systematic approach to handling them, building a “factory” for reliable data pipelines. We’ve seen many organizations struggling at how to balance agile development practices with maintaining data quality, and we are looking forward to learn from Ancestry’s experience.

 

This event will be the largest gathering of Kafka experts across a wide range of industries so we hope to see you there. Register early, and you can follow the event on twitter at #kafkasummit.