Conquering All Your Stream Processing Needs with Kafka and Spark

Conquering All Your Stream Processing Needs with Kafka and Spark

On-demand recording

Kafka Summit 2016 | Systems Track

Apache Spark, specifically Spark Streaming, is becoming one of the most widely used stream processing system for Kafka. At its heart, Spark is an extremely fast and general-purpose distributed data processing platform. This allows the unification of all kinds of data processing using a single framework – streaming, SQL, and machine learning. For Kafka users, this means that they can use Spark to run batch jobs, streaming pipelines as well as interactive queries on Kafka data. In this talk, I am going to give a brief overview of the Spark framework and elaborate on how different components of Spark can be used to process data from Kafka. Specifically, I am going to cover the following.

  • Real-time processing of Kafka streams with Spark Streaming
  • Batch and interactive querying of Kafka data with Spark and Spark SQL
  • Schema-aware streaming ETL from with Streaming DataFrames

Speakers:

Tathagata Das, Software Engineer, Databricks

We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.

Agenda

GENERAL SESSIONS
08:00 – 09:00 Registration & Partner Showcase
09:00 – 9:10 Welcome
09:10 – 09:45 Keynote: State of Streaming, what’s next
09:45 – 10:15 Keynote: Customer – Euronext ??
10:15 – 10:45 Partner Session
10:45 – 11:15 Coffee Break & Sponsor Showcase
11:15 – 11:45 Keynote: Customer – Criteo ??
11:45 – 12:15 Partner Session
12:15 – 13:00
Lunch
13:00 – 14:00
14:00 – 14:30 CONFLUENT Monitor Kafka Like a Pro with C3
14:30 – 15:00 Customer Use Case – Credit Mutuel ??
15:00 – 15:45 CONFLUENT KSQL in Production
15:45 – 16:15 Coffee Break & Sponsor Showcase
16:15 – 16:35 CONFLUENT : Global Kafka / Kubernetes
16:35 – 16:55 Partner Session
16:55 – 17:15 Customer Case : BNP Secu ?
17:15 – 19:00 Topic Corners & Get Together