[Webinar] Mastering Kafka Security Across Hybrid Environments → Register Now


Playing with Xbox Data

« Kafka Summit London 2023

A fun introduction to the world of Kafka Connect and Kafka Streams by using it to process data from Xbox Live. Gaming is a social activity, so Xbox includes a social aspect. Details about what games you play and when you’re playing are shared with your friends on the Xbox Live service. Achievements that you earn and complete in games are shared with your friends. This makes for a fun source of data, that we can play with using Kafka Connect and Kafka Streams.

  1. Getting the data into Kafka — I’ll show how I’m using Kafka Connect to surface data from Xbox REST APIs as streams of events on Kafka topics.
  2. Processing the events — I’ll show how I’m using Kafka Streams to process the Xbox events as they’re emitted - such as cleaning up some of the noisy events with filters, enriching events with additional context using transformations, and creating gamerscore leaderboards using aggregations.

As a project, it is just a bit of fun. But it is a useful exercise for illustrating how to get started with Kafka Connect and Kafka Streams, and a way to demonstrate the principles using a real source of real events, rather than using fake or generated events. Plus it is a good excuse for me to spend more time gaming, and justifying it as work!

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how