[Webinar] Q1 Confluent Cloud Launch Brings You the Latest Features | Register Now

Presentation

Lessons from building a stream-first metadata platform

« Kafka Summit Europe 2021

For data-driven enterprises, the most important objective is unlocking the value of their data. To enable this, data scientists are increasingly turning towards data discovery tools (also known as data catalogs) that can help them locate the right dataset or insight and use it correctly. But are all data catalogs the same? In this talk, I describe how a stream-first architecture was a critical design element that benefited the implementation of our data catalog. We follow the evolution of LinkedIn DataHub’s architecture over the past few years from a simple search tool to a streaming metadata platform that drives productivity and governance workflows across the company. Join this talk to learn:

  • How different data discovery / catalog tools are architected and the tradeoffs in each kind of architecture
  • How streaming architectures can benefit metadata
  • How event-driven metadata architectures can supercharge your data productivity and governance workflows at your company

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how