Generating Real-time Recommendations with NiFi, Kafka, and Spark

Generating Real-time Recommendations with NiFi, Kafka, and Spark

On-demand recording

Kafka Summit 2016 | Systems Track

We’ll start the talk with a live, interactive demo generating audience-specific recommendations using NiFi, Kafka, Spark Streaming, SQL, ML, and GraphX.

Next, we’ll dive deep into the data flow between each of the key components. We use NiFi to track all data transformations using its “data provenance” capabilities. We use Kafka for its high-throughput and high-availability features. And we use Spark Streaming to dynamically train our ML models in real-time using an incremental Streaming Matrix Factorization library from Databricks.

Lastly, we’ll discuss the latest Netflix Recommendations Pipeline including their highly-scalable Netflix Open Source components for ML model serving.

We’re going to cover a lot of material in a short amount of time. All of the slides, demos, and Docker images will be available at advancedspark.com.

Speakers:

Chris Fregly, Principal Data Solutions Engineer, IBM Spark Technology Center

We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.

Agenda

GENERAL SESSIONS
08:00 – 09:00 Registration & Partner Showcase
09:00 – 9:10 Welcome
09:10 – 09:45 Keynote: State of Streaming, what’s next
09:45 – 10:15 Keynote: Customer – Euronext ??
10:15 – 10:45 Partner Session
10:45 – 11:15 Coffee Break & Sponsor Showcase
11:15 – 11:45 Keynote: Customer – Criteo ??
11:45 – 12:15 Partner Session
12:15 – 13:00
Lunch
13:00 – 14:00
14:00 – 14:30 CONFLUENT Monitor Kafka Like a Pro with C3
14:30 – 15:00 Customer Use Case – Credit Mutuel ??
15:00 – 15:45 CONFLUENT KSQL in Production
15:45 – 16:15 Coffee Break & Sponsor Showcase
16:15 – 16:35 CONFLUENT : Global Kafka / Kubernetes
16:35 – 16:55 Partner Session
16:55 – 17:15 Customer Case : BNP Secu ?
17:15 – 19:00 Topic Corners & Get Together