Kafka Connect
Kafka Connect is a framework included in Apache Kafka that integrates Kafka with other systems. Its purpose is to make it easy to add new systems to your scalable and secure stream data pipelines.
To copy data between Kafka and another system, users instantiate Kafka Connectors for the systems they want to pull data from or push data to. Source Connectors import data from another system (e.g. a relational database into Kafka) and Sink Connectors export data (e.g. the contents of a Kafka topic to an HDFS file).
This page lists many of the notable connectors available.
Confluent Connectors
Confluent Connectors are developed, tested, documented and are fully supported by Confluent.
- ActiveMQ (Source)AMQ, ActiveMQ, JMS, Message BrokerConfluentConfluent
- Amazon S3 (Sink)S3ConfluentConfluent
- Elasticsearch (Sink)search, Elastic, log, analyticsConfluentConfluent
- HDFS (Sink)HDFS, Hadoop, HiveConfluentConfluent
- IBM MQ (Source)IBM MQ, JMS, Message BrokerConfluentConfluent
- JDBC (Source)JDBC, MySQLConfluentConfluent
- JDBC (Sink)JDBC, MySQLConfluentConfluent
- JMS (Source)JMS, Message BrokerConfluentConfluent
- Replicator (source)Kafka, Cluster, ReplicationConfluentConfluent
Certified Connectors
Certified Connectors have been developed by vendors utilizing the Kafka Connect framework. These Connectors have met criteria for code development best practices, schema registry integration, security, and documentation.
- Attunity (Source)CDC, Oracle, MSSQL Server, MySQL, DB2, Postgres, Cloudera, MapR, Flat FilesAttunityAttunity
- Azure IoTHub (Source)IoT, messagingMicrosoftAzure
- Couchbase (Source)Couchbase, NoSQLCouchbaseCouchbase
- Couchbase (Sink)Couchbase, NoSQLCouchbaseCouchbase
- Dbvisit Replicate (Source)CDC, OracleDbvisitDbvisit
- HVRCDC, Oracle, MS SQL, DB2, PostgresHVRHVR
- IBM Data Replication (Source)CDCIBMIBM
- JustOne (Sink)PostgresJustOneJustOne
- Kinetica (Source)GPU, RDBMSKineticaKinetica
- Kinetica (Sink)GPU, RDBMSKineticaKinetica
- Oracle GoldenGate CDC, OracleOracleOracle
- SAP HANA (Sink)HANA, RDBMSSAPSAP
- SAP HANA (Source)HANA, RDBMSSAPSAP
- Striim (Source)CDC, MS SQLServer, Oracle, MySQLStriimStriim
- Syncsort DMX (Source)DB2, IMS, VSAM, CICSSyncsortSyncsort
- Syncsort DMX (Sink)DB2, IMS, VSAM, CICSSyncsortSyncsort
- Vertica (Source)VerticaHP EnterpriseHP Enterprise
- Vertica (Sink)VerticaHP EnterpriseHP Enterprise
- VoltDB (Sink)VoltDB, NewSQLVoltDBVoltDB
- Xenon (Sink)XenonLevyxXenon
Additional Connectors Available
Other notable Connectors that have been developed utilizing the Kafka Connect framework.
- Amazon Kinesis (Sink)messagingCommunityCommunity
- Apache Ignite (Source)File SystemCommunityCommunity
- Apache Ignite (Sink)File SystemCommunityCommunity
- ArangoDB (Sink)NoSQLCommunityCommunity
- AWS Lambda (Sink)Cloud, AWS, LambdaCommunityCommunity
- Azure DocumentDB (Sink)DocumentDB, Azure, NoSQLCommunityCommunity
- Blockchain (Source)Bitcoin, BlockchainCommunityCommunity
- Bloomberg Ticker (Source)Application feedCommunityCommunity
- Cassandra (Sink)CassandraCommunityCommunity
- Cassandra (Sink)Cassandra, NoSQLCommunityCommunity
- CoAP (Source)Constrained Application Protocol, CoAPCommunityCommunity
- CouchDB (Source and Sink)CouchDB, NoSQLCommunityCommunity
- CouchDB (Sink)CouchDB, NoSQLCommunityCommunity
- DynamoDBDynamo, NoSQLCommunityCommunity
- Elasticsearch (Sink)Elastic, search, log, analyticsCommunityCommunity 1 Community 2 Community 3
- Files/Directories (Source)File System, Directories, LogsCommunityCommunity 1Community 2
- FileSystem Connector (Source)File System, S3, HDFSCommunityCommunity
- FTP (Source)File SystemCommunityCommunity 1Community 2
- Github (Source)Application FeedCommunityCommunity
- Google BigQuery (Sink)Analytics, Data WarehouseCommunityCommunity
- Google PubSub (Source)MessagingCommunityCommunity
- Google PubSub (Sink)MessagingCommunityCommunity
- Graphite (Sink)graphite databaseCommunityCommunity
- Hazelcast (Sink)Datastore, In-memoryCommunityCommunity
- Hbase (Sink)Hbase, NoSQLCommunityCommunity 1Community 2
- IBM MQ (Source)IBM MQ, messagingCommunityCommunity
- IBM MQ (Sink)IBM MQ, messagingCommunityCommunity
- Infinispan (Sink)Infinispan, NoSQLCommunityCommunity
- InfluxDB (Sink)Datastore, Time-seriesCommunityCommunity
- Internet Relay Chat (Source)Application feedCommunityCommunity
- Jenkins (Source)Application feedCommunityCommunity
- JMS (Sink)MessagingCommunityCommunity
- JMX (Source)jmx, metricsCommunityCommunity
- Kinesis Connector (Source)MessagingCommunityCommunity
- Kudu (Sink)KuduCommunityCommunity
- MarkLogic (Sink)Document Store, NoSQLCommunityCommunity
- Mixpanel (Source)analyticsCommunityCommunity
- MongoDB (Source)Mongo, MongoDB, NoSQLCommunityCommunity 1Community 2
- MongoDB CDC - Debezium (Source)MongoDB, CDCCommunityCommunity
- MongoDB (Sink)Mongo, MongoDB, NoSQLCommunityCommunity 1Community 2Community 3Community 4
- MQTT (Source)MQTT, messagingCommunityCommunity 1
- MySQL CDC - Debezium (Source)MySQL, CDC, OracleCommunityCommunity
- NATS (Source)MessagingCommunityCommunity
- NATS (Sink)MessagingCommunityCommunity
- OrientDB (Sink)OrientDB, NoSQLCommunityCommunity
- PostgreSQL CDC - Debezium (Source)PostgreSQL, CDCCommunityCommunity
- PubNubPubNub, mobile appsCommunityCommunity
- RabbitMQMessagingCommunityCommunity
- Redis (Source)DatastoreCommunityCommunity
- Redis (Sink)Datastore, MessagingCommunityCommunity
- REST (Source and Sink)RESTCommunityCommunity
- RethinkDB (Source) NoSQL, StreamingCommunityCommunity
- RethinkDB (Sink)NoSQL, StreamingCommunityCommunity
- Salesforce (Source)DatastoreCommunityCommunity
- Solr (Sink)search, solrCommunityCommunity 1 Community 2
- Solr (Source)search, solrCommunityCommunity
- Splunk (Sink)Datastore, logCommunityCommunity
- Splunk (Source)Datastore, logCommunityCommunity
- SQS (Source)AWS, SQS, MessagingCommunity Community
- Syslog (Source)logCommunityCommunity
- S3 (Sink)AWS, S3, logCommunity Community 1 Community 2
- Twitter (Source)Application feedCommunityCommunity 1Community 2
- Twitter (Sink)Application feedCommunityCommunity