[Webinar] How to Implement Data Contracts: A Shift Left to First-Class Data Products | Register Now

Online Talk

The Changing Face of ETL: Event-Driven Architectures for Data Engineers

今すぐ登録

Available On Demand

Data integration in architectures built on static, update-in-place datastores inevitably end up with pathologically high degrees of coupling and poor scalability. This has been the standard practice for decades, as we attempt to build data pipelines on top of databases that do a poor job modeling the fundamental objects that drive our businesses and systems: events.

Events carry both notification and state, and form a powerful primitive on which to build systems for developers and data engineers alike. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities. Everyone gains from using the standards-based, scalable and resilient streaming platform.

In this talk, we’ll discuss the concepts of events, their relevance to both software engineers and data engineers and their ability to unify architectures in a powerful way. We’ll see how stream processing makes sense in both a microservices and ETL environment, and why analytics, data integration and ETL fit naturally into a streaming world.

Robin は Confluent の DevRel チームで働いています。彼のデータエンジニアリングのキャリアは、COBOL を使ってメインフレーム上にデータウェアハウスを構築するところから始まりました。その後 Oracle のアナリティクスソリューションの開発を経て、近年では Kafka エコシステムとモダンなデータストリーミングの分野で活躍しています。仕事以外では、ランニングやおいしいビール、揚げ物中心の朝食が大好きです(もっとも、これらを同時に楽しむことはめったにありませんが)。