Announcing AWS PrivateLink Support in Confluent Cloud

We’re happy to announce that Confluent Cloud, our fully managed event streaming service powered by Apache Kafka^®, now supports AWS PrivateLink for secure network connectivity, in addition to the existing VPC peering, AWS Transit Gateway, and secure internet connectivity options. AWS PrivateLink is supported on Confluent Cloud Dedicated clusters whether you procure Confluent Cloud directly from Confluent or AWS Marketplace.

AWS PrivateLink is an AWS proprietary networking service that allows one-way secure access from your VPC to both AWS and third-party services. Now you can create a new Dedicated cluster in Confluent Cloud with PrivateLink enabled, set up a VPC endpoint in your AWS VPC, and securely connect to Confluent Cloud’s event streaming platform from your VPC.

Multiple financial institutions are already using AWS PrivateLink for their Confluent Cloud deployments, and we’re excited to now offer this capability to all Confluent Cloud customers using Dedicated clusters.

Why do customers choose PrivateLink?

Enterprises use PrivateLink for its unique combination of security and simplicity.

For many companies, a multi-layer data security policy starts with minimizing network attack vectors exposed to the public internet. Security breaches, DDOS attacks, spam, and other concerns can be prevented by blocking internet access to key resources like Kafka clusters. VPC peering—where two parties share network addresses across two networks—has historically been a common solution for private network connectivity, but it has its downsides.

VPC peering requires both parties to coordinate on an IP address block for communication between the networks. Many companies, especially large enterprises, have limited IP space, so finding an available IP address block can be challenging and requires a lot of back and forth between teams and between the peering parties. This can be especially painful in large organizations with hundreds of networks connected in a sophisticated topology. Applications that need access to Kafka are likely spread across many networks, so peering them all to Confluent Cloud is a lot of work.

Once a VPC peering connection is set up, each party has access to the other network—that’s what connectivity means—but this isn’t always desirable. Confluent Cloud users want their clients to initiate connections to Confluent Cloud but restrict Confluent from having access back into their network.

PrivateLink enables network-level security without the downsides of VPC peering. Confluent exposes a PrivateLink service endpoint for each new cluster, for which customers can create corresponding VPC endpoints in their own AWS VPCs. Customers don’t have to juggle with IP address blocks for Confluent Cloud because their clients connect using the VPC endpoint. It’s a one-way connection from the customer to Confluent Cloud, so there’s less surface area for the network security team to keep secure. Making dozens or hundreds of PrivateLink connections to a single Confluent Cloud cluster doesn’t require any extra coordination, either with Confluent or within your organization.

With all these benefits, it’s not surprising AWS recommends PrivateLink as the best method for private connectivity between AWS VPCs.

Behind the scenes

Supporting AWS PrivateLink in Confluent Cloud has been a major effort. Historically, Confluent Cloud has used Elastic Load Balancers. However, PrivateLink service endpoints must be configured on AWS Network Load Balancers. In order to provide the easiest possible setup experience for our customers, we’ve updated our networking infrastructure and automation by building new services to manage the creation of service endpoints for new clusters and registration of customer AWS accounts for each cluster. The outcome: Spin up a Dedicated cluster in Confluent Cloud and get a PrivateLink service endpoint in minutes directly through the Confluent Cloud UI, totally self-serve, so the clusters can be up and running in no time.

Getting started with AWS PrivateLink in Confluent Cloud

Connect to Confluent Cloud securely from your AWS account using AWS PrivateLink and enjoy the unique benefits of this connectivity.

1. Create a new Dedicated cluster in AWS, which requires an annual commitment. You choose the AWS region you want to run in, whether the cluster should run in one or multiple availability zones, and the cluster’s capacity.
   
2. After the cluster is provisioned, you’ll receive an email and see an alert in the Confluent Cloud UI to finish setting up the PrivateLink connection. Confluent Cloud requires you to register the Account IDs you’ll connect from, so we can guarantee that only your organization can make PrivateLink connections to your Confluent Cloud cluster. You can connect from any or all of the VPCs in the AWS accounts you register, so even if you have Kafka clients spread across dozens of VPCs, you can securely connect them all to Confluent Cloud.
3. Set up subnets, VPC endpoints, and DNS in your AWS account. AWS PrivateLink actually consists of two parts: the service endpoint exposed by Confluent Cloud and VPC endpoints that you configure within your AWS Account. Confluent offers a Terraform script to automate the setup of your VPC endpoints, since there are quite a few steps. If Terraform is not for you, we also have detailed docs you can follow.
   
4. Start streaming!

Confluent Cloud with self-serve AWS PrivateLink support provides enhanced user experience with secure and hassle-free connectivity. We would love to hear about your experience.

Join some of the most security-conscious, highly-regulated companies in the world using AWS PrivateLink on Confluent Cloud, and get started for free using the promo code CL60BLOG to get an additional $60 of free Confluent Cloud usage.*