[Webinar] How to Implement Data Contracts: A Shift Left to First-Class Data Products | Register Now

Online Talk

No More Silos: Integrating Databases into Apache Kafka®

Watch Now

Companies new and old are all recognizing the importance of a low-latency, scalable, fault-tolerant data backbone, in the form of the Apache Kafka® streaming platform. With Apache Kafka, developers can integrate multiple sources and systems, which enables low latency analytics, event-driven architectures and the population of multiple downstream systems.

In this talk, we’ll look at one of the most common integration requirements – connecting databases to Apache Kafka. We’ll consider the concept that all data is a stream of events, including that residing within a database. We’ll look at why we’d want to stream data from a database, including driving applications in Apache Kafka from events upstream. We’ll discuss the different methods for connecting databases to Apache Kafka, and the pros and cons of each. Techniques including Change-Data-Capture (CDC) and Kafka Connect will be covered, as well as an exploration of the power of KSQL, streaming SQL for Apache Kafka, for performing transformations such as joins on the inbound data.

Watch now to learn:

  • Why databases are just a materialized view of a stream of events
  • The best ways to integrate databases with Apache Kafka
  • Anti-patterns to be aware of
  • The power of KSQL for transforming streams of data in Apache Kafka

Robin は Confluent の DevRel チームで働いています。彼のデータエンジニアリングのキャリアは、COBOL を使ってメインフレーム上にデータウェアハウスを構築するところから始まりました。その後 Oracle のアナリティクスソリューションの開発を経て、近年では Kafka エコシステムとモダンなデータストリーミングの分野で活躍しています。仕事以外では、ランニングやおいしいビール、揚げ物中心の朝食が大好きです(もっとも、これらを同時に楽しむことはめったにありませんが)。