Kafka In the Cloud: Why It鈥檚 10x Better With Confluent | Get free eBook
We鈥檙e happy to announce that Confluent Cloud, our cloud-native service for Apache Kafka庐, now supports Azure Private Link for secure network connectivity, in addition to the existing Azure Virtual Network (VNet) peering and secure internet connectivity options. Azure Private Link is supported on Dedicated clusters whether you procure Confluent Cloud directly from Confluent or through the Azure Marketplace.
Azure Private Link is an Azure proprietary networking service that allows one-way secure connection access from your VNet to both Azure and third-party services. Now you can create a new Dedicated cluster in Confluent with Private Link enabled, set up a private endpoint in your Azure VNet, and securely connect to Confluent鈥檚 platform for data in motion from your VNet.
Multiple organizations are already using Azure Private Link for their Confluent deployments, and we鈥檙e excited to now offer this capability to all Confluent customers using Dedicated clusters. Continue reading to learn more about this networking option or skip to instructions for setup in your account.
Enterprises use Private Link for its unique combination of security and simplicity.
For many companies, a multi-layer data security policy starts with addressing network attack vectors exposed to the public internet. Security breaches, DDOS attacks, spam, and other concerns can be prevented by blocking internet access to key resources like Kafka clusters. VNet peering鈥攚here two parties share network addresses across two networks鈥攈as historically been a common solution for private network connectivity, but it has its downsides.
VNet peering requires both parties to coordinate on a unique IP address block for communication between the networks. Many companies, especially large enterprises, have limited IP space, so finding an available IP address block can be challenging and requires a lot of back and forth between teams and between the peering parties. This can be especially painful in large organizations with hundreds of networks connected in a sophisticated topology. Applications that need access to Kafka are likely spread across many networks, so peering them all to Confluent is a lot of work.
Once a VNet peering connection is set up, each party has access to the other network鈥攖hat鈥檚 what connectivity means鈥攂ut this isn鈥檛 always desirable. Confluent users want their clients to initiate connections to Confluent Cloud but restrict Confluent from having access back into their network.
Private Link enables network-level security without the downsides of VNet peering. Confluent exposes Private Link service alias(es) for each new cluster, for which customers can create corresponding private endpoints in their own Azure VNets. Users don鈥檛 have to juggle with IP address blocks for Confluent because their clients connect using the private endpoint with an associated IP address whose scope is local to the user鈥檚 VNet. It鈥檚 a one-way connection from the user to Confluent, so there鈥檚 less surface area for the network security team to keep secure. Making dozens or hundreds of Private Link connections to a single Confluent cluster doesn鈥檛 require any extra coordination with Confluent nor within your organization.
With all these benefits, it鈥檚 not surprising that Azure recommends Private Link as the best method for private connectivity between Azure VNets.
Supporting Private Link in Confluent has been a major effort. Last year, we rolled out support for AWS PrivateLink by introducing a new networking stack to help expose Confluent through AWS PrivateLink using AWS Network Load Balancers. We also introduced self-serve customer AWS account registration with Confluent to enable complete automation of PrivateLink interface endpoint creation from the customer VPC.
We have now extended Private Link support to Azure by adapting the networking stack to run on Azure, and we have integrated it with Azure Standard Load Balancers and Azure Private Link Service. The Confluent and Azure Private Link solution also includes self-serve Azure subscription registration with Confluent to enable complete automation of private endpoint creation from the customer VNet. The outcome: Spin up a Dedicated cluster in Confluent and get an Azure Private Link Service alias in minutes directly through the Confluent UI, totally self-serve, so your clusters can be up and running in no time.
Connect to Confluent securely from your Azure subscription using Azure Private Link Service and enjoy the unique benefits of this connectivity.
As you can see, with just four simple steps, Confluent and self-serve Azure Private Link support provides an enhanced user experience with secure and hassle-free connectivity. We would love to hear about your experience and encourage you to join our Community Forum and connect with industry experts!
Join some of the most security-conscious, highly-regulated companies in the world using Azure Private Link on Confluent Cloud, and get started for free. When you sign up, you鈥檒l receive $400 to spend within Confluent Cloud during your first 60 days. You can also use the promo code CL60BLOG
to get an additional $60 of free Confluent Cloud usage.*
Building data streaming applications, and growing them beyond a single team is challenging. Data silos develop easily and can be difficult to solve. The tools provided by Confluent鈥檚 Stream Governance platform can help break down those walls and make your data accessible to those who need it.
Self-managing connectors come with major time and resource challenges and taking on unnecessary risks of downtime that shift your team鈥檚 focus away from working on more strategic projects and innovations...