Version 1.5.0
Plugin type:
Sink
Enterprise support:
Confluent supported
Verification:
Confluent built
Author:
Confluent, Inc.
Documentation

Kafka Connect Azure Data Lake Gen1 Sink

The Azure Data Lake Gen1 Sink Connector integrates Azure Data Lake Gen1 with Apache Kafka. The connector can export data from Apache Kafka® topics to Azure Data Lake Gen1 files in either Avro or JSON formats. Depending on your environment, the Azure Data Lake Gen1 connector can export data by guaranteeing exactly-once delivery semantics to consumers of the Azure Data Lake Gen1 files it produces.

The Azure Data Lake Gen1 sink connector periodically polls data from Kafka and in turn uploads it to Azure Data Lake Gen1. A partitioner is used to split the data of every Kafka partition into chunks. Each chunk of data is represented as an Azure Data Lake Gen1 file.

Show more

Installation

Confluent Hub CLI installation

Use the Confluent Hub client to install this connector with:
confluent-hub install confluentinc/kafka-connect-azure-data-lake-gen1-storage:1.5.0
Copy

Download installation

Or download the ZIP file and extract it into one of the directories that is listed on the Connect worker's plugin.path configuration properties. This must be done on each of the installations where Connect will be run.
By downloading you agree to the terms of use and software license agreement.
Configure an instance of your connector
Once installed, you can then create a connector configuration file with the connector's settings, and deploy that to a Connect worker.

Support

This connector is supported by Confluent as part of a Confluent Platform subscription.