Apache Flink is a powerful open-source stream processing framework that enables real-time data processing at scale. One of the key features of Flink is its rich ecosystem of connectors that allow users to easily integrate with a wide range of data sources and sinks. However, working with connectors can be challenging, especially for users who are new to Flink or stream processing.
This talk aims to help users better understand Flink connectors, the Flink connector ecosystem, and their importance in building scalable and robust data processing pipelines. It will cover topics such as:
- An introduction to Flink connectors and their role in stream processing
- A deep dive into the different Flink connector APIs, including the Unified Source and Sink API, SourceReaderBase and the Async Sink API.
- The benefits of using unified batch and streaming APIs in Flink
By the end of this talk, attendees will have a solid understanding of Flink connectors, the connector interface, and be better equipped to build efficient and reliable data processing pipelines with Flink.
Presenter
Martijn Visser
ConfluentMartijn Visser is Group Product Manager at Confluent and PMC member and committer for the Apache Flink project. He works with the open source community on user-facing features such as the Table/SQL and DataStream API, connectors, and formats. Prior to joining Confluent, he worked as product manager at Immerok, which was acquired by Confluent.
He has also worked as product manager at Ververica, where he was responsible for product development on Apache Flink and Ververica Platform, and as product lead at ING, where he was responsible for ING's Streaming Data and Engagement Platform.