Kafka Connect Azure Data Lake Storage Gen2
The Azure Data Lake Gen2 sink connector periodically polls data from Kafka and in turn uploads it to Azure Data Lake Gen2. A partitioner is used to split the data of every Kafka partition into chunks. Each chunk of data is represented as an Azure Data Lake Gen2 file. The key name encodes the topic, Kafka partition, and start offset of this data chunk.