Kafka Connect HDFS
The HDFS connector allows you to export data from Kafka topics to HDFS files in a variety of formats and integrates with Hive to make data immediately available for querying with HiveQL. The connector periodically polls data from Kafka and writes them to HDFS.
Confluent Hub CLI installation
Use the Confluent Hub client to install this connector with:
confluent-hub install confluentinc/kafka-connect-hdfs:5.5.1
Or download the ZIP file and extract it into one of the directories that is listed on the Connect worker's plugin.path configuration properties. This must be done on each of the installations where Connect will be run.
Once installed, you can then create a connector configuration file with the connector's settings, and deploy that to a Connect worker.
Confluent supports the HDFS sink connector alongside community members as part of its Confluent Platform offering.