Building Adaptive User Experiences Using Contextual Multi-Armed Bandits with Apache Kafka

At Expedia Group, providing a customized experience for travellers is key to unlocking the best possibilities for each individual traveller and each type of trip.Contextual multi-armed bandits provide a natural approach to develop personalization of user experience and improve content relevancy. In this talk,we present the end-to-end scalable system developed to democratize the use of contextual bandits at EG.The architecture comprises of an online inference component as well as a continuous feedback loop that tracks the users’ affinity towards certain content or page layouts. Kafka is the backbone of our system, powering high-performance streaming jobs that provide bandits with real-time feedback signals to learn from over time. We describe our experience using Kafka for the user interactions events and bandit feedback messages at scale. Lastly,we look at how we plan to expand our use of Kafka to build an off policy evaluation framework to evaluate the effectiveness of new algorithms.