Show Me How: Build Streaming Data Pipelines for Real-Time Data Warehousing | Register Today
We’re happy to announce that Confluent Cloud, our cloud-native service for Apache Kafka®, now supports Azure Private Link for secure network connectivity, in addition to the existing Azure Virtual Network (VNet) peering and secure internet connectivity options. Azure Private Link is supported on Dedicated clusters whether you procure Confluent Cloud directly from Confluent or through the Azure Marketplace.
Azure Private Link is an Azure proprietary networking service that allows one-way secure connection access from your VNet to both Azure and third-party services. Now you can create a new Dedicated cluster in Confluent with Private Link enabled, set up a private endpoint in your Azure VNet, and securely connect to Confluent’s platform for data in motion from your VNet.
Multiple organizations are already using Azure Private Link for their Confluent deployments, and we’re excited to now offer this capability to all Confluent customers using Dedicated clusters. Continue reading to learn more about this networking option or skip to instructions for setup in your account.
Enterprises use Private Link for its unique combination of security and simplicity.
For many companies, a multi-layer data security policy starts with addressing network attack vectors exposed to the public internet. Security breaches, DDOS attacks, spam, and other concerns can be prevented by blocking internet access to key resources like Kafka clusters. VNet peering—where two parties share network addresses across two networks—has historically been a common solution for private network connectivity, but it has its downsides.
VNet peering requires both parties to coordinate on a unique IP address block for communication between the networks. Many companies, especially large enterprises, have limited IP space, so finding an available IP address block can be challenging and requires a lot of back and forth between teams and between the peering parties. This can be especially painful in large organizations with hundreds of networks connected in a sophisticated topology. Applications that need access to Kafka are likely spread across many networks, so peering them all to Confluent is a lot of work.
Once a VNet peering connection is set up, each party has access to the other network—that’s what connectivity means—but this isn’t always desirable. Confluent users want their clients to initiate connections to Confluent Cloud but restrict Confluent from having access back into their network.
Private Link enables network-level security without the downsides of VNet peering. Confluent exposes Private Link service alias(es) for each new cluster, for which customers can create corresponding private endpoints in their own Azure VNets. Users don’t have to juggle with IP address blocks for Confluent because their clients connect using the private endpoint with an associated IP address whose scope is local to the user’s VNet. It’s a one-way connection from the user to Confluent, so there’s less surface area for the network security team to keep secure. Making dozens or hundreds of Private Link connections to a single Confluent cluster doesn’t require any extra coordination with Confluent nor within your organization.
With all these benefits, it’s not surprising that Azure recommends Private Link as the best method for private connectivity between Azure VNets.
Supporting Private Link in Confluent has been a major effort. Last year, we rolled out support for AWS PrivateLink by introducing a new networking stack to help expose Confluent through AWS PrivateLink using AWS Network Load Balancers. We also introduced self-serve customer AWS account registration with Confluent to enable complete automation of PrivateLink interface endpoint creation from the customer VPC.
We have now extended Private Link support to Azure by adapting the networking stack to run on Azure, and we have integrated it with Azure Standard Load Balancers and Azure Private Link Service. The Confluent and Azure Private Link solution also includes self-serve Azure subscription registration with Confluent to enable complete automation of private endpoint creation from the customer VNet. The outcome: Spin up a Dedicated cluster in Confluent and get an Azure Private Link Service alias in minutes directly through the Confluent UI, totally self-serve, so your clusters can be up and running in no time.
Connect to Confluent securely from your Azure subscription using Azure Private Link Service and enjoy the unique benefits of this connectivity.
As you can see, with just four simple steps, Confluent and self-serve Azure Private Link support provides an enhanced user experience with secure and hassle-free connectivity. We would love to hear about your experience and encourage you to join our Community Forum and connect with industry experts!
Join some of the most security-conscious, highly-regulated companies in the world using Azure Private Link on Confluent Cloud, and get started for free. When you sign up, you’ll receive $400 to spend within Confluent Cloud during your first 60 days. You can also use the promo code
CL60BLOG to get an additional $60 of free Confluent Cloud usage.*
Use the Confluent CLI and API to create Stream Designer pipelines from SQL source code.
This post details how to minimize internal messaging within Confluent platform clusters. Service mesh and containerized applications have popularized the idea of control and data planes. This post applies it to the Confluent platform clusters and highlights its use in Confluent Cloud.