Robinhood uses Kafka in every line of its business, from stock and crypto trading to its self-clearing system and online data analytics. Different systems need different reliability guarantees. We will talk about various architectures we use to achieve these desperate goals while maintaining sanity in our client side code.
This talk discusses how we removed SPoF through investments in Kafka infrastructure and our client libraries, letting us support a multitude of requirements of various systems inside robinhood. In addition we will discuss:
- Learnings from building client libraries to support these HA strategies
- K8s sidecars to help our applications work with sharded architecture
- Observability and debuggability tools we built to support use cases
Presenter
Sreeram Ramji
RobinhoodI work in Streaming Platform team in Robinhood which owns Kafka and Flink Infra and abstractions. I have worked in building up the Data Compute and Ingestion teams in Robinhood before this. I have also worked for 4 years in Yahoo! building its Data Highway a platform for transporting petabytes of data every day