Kafka Connect GCS

By Confluent, Inc.

confluentinc/kafka-connect-gcs:5.0.0-preview
20%
Version

5.0.0-preview

Features

  • Single Message Transform
  • Control Center Integration
  • Kafka Connect API

Tags

Kafka Connect GCS

Confluent, Inc.

confluentinc/kafka-connect-gcs:5.0.0-preview
20%

The GCS connector, currently available as a sink, allows you to export data from Kafka topics to GCS objects in either Avro or JSON formats.

Being a sink, the GCS connector periodically polls data from Kafka and in turn uploads it to GCS A partitioner is used to split the data of every Kafka partition into chunks

Each chunk of data is represented as an GCS object, whose key name encodes the topic,
the Kafka partition and the start offset of this data chunk If no partitioner is specified in the configuration, the default partitioner which preserves Kafka partitioning is used The size of each data chunk is determined by the number of records written to GCS and by schema compatibility.
It is included in Confluent Enterprise Platform, or can be downloaded and installed separately It can be used for free for 30 days, but after that does require an Enterprise license Contact Confluent for more details.

More......Less

Install your connector

Use the Confluent Hub client to install this connector with:

confluent-hub install confluentinc/kafka-connect-gcs:5.0.0-preview

Or download the ZIP file and extract it into one of the directories that is listed on the Connect worker's plugin.path configuration properties. This must be done on each of the installations where Connect will be run. See here for more detailed instructions.

Configure an instance of your connector

Once installed, you can then create a connector configuration file with the connector's settings, and deploy that to a Connect worker. See here for more detailed instructions.

Source code is not available for this connector.

Documentation is not available for this connector.

This connector is fully supported by Confluent as part of a Confluent Enterprise Platform subscription.
Rate this connector

We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.