Introducing Private Network Interface: Secure Private Networking on AWS for 50% Less

This is the second post in our series exploring the architectural innovations that make Confluent Cloud more cost-effective at scale. Building on our previous post about the operational complexities of Apache Kafka® and our cloud-native architecture's solutions, we'll now dive into how we solved a core challenge for any data streaming workload: high cloud networking costs.

We’re excited to introduce a new option for private networking on Amazon Web Services (AWS): Private Network Interface (PNI). PNI fundamentally enhances security and optimizes costs, solving the difficult trade-off inherent in other networking options like AWS PrivateLink and virtual private cloud (VPC) peering. It leverages the same underlying AWS networking primitives used to power Amazon’s own services, such as Amazon Elastic Kubernetes Service (EKS) or AWS Lambda, and delivers secure, cost-efficient private connectivity with the low latency and high throughput that modern Kafka workloads demand. 

PNI is already available with Freight clusters, and today, we’re excited to announce it’s also supported on Enterprise clusters. For Enterprise clusters that use PNI for private networking, we’re reducing throughput costs by 20%, from $0.05 to $0.04 per GB of data read or written. We’re also announcing an overall 40% reduction in throughput costs for Freight clusters, from $0.05 to $0.03 per GB of data read or written. 

These are massive steps towards our goal to make real-time streaming cost effective for any use case. While compute expenses can be optimized through elastic autoscaling—a capability that inherently reduces infrastructure costs for our customers by over 50%—and storage costs can be managed with object storage, networking frequently remains an overlooked area.

This challenge has become even more critical with the rise of rise-time data pipelines that power modern artificial intelligence (AI) and event-driven use cases. Data-intensive workloads like machine learning (ML) training and inference drive unprecedented data volumes through Kafka. As throughput grows, networking transforms from an operational detail into a fundamental strategic concern. This is especially true with private networking, where throughput increases can lead to disproportionate networking spend. With the traditional private networking options, users are often left with a trade-off between cost and security. 

A New Model for Secure, Cost-Effective Connectivity

Cost vs Security in Traditional Cloud Networking

Traditionally, VPC peering has been the cheapest way to move traffic in the cloud when you’re not crossing availability zones (AZs), but it’s often seen as less secure. It lacks centralized security policy enforcement and presents IP address overlap challenges, and its inability to route traffic through intermediary VPCs restricts architectural choices. 

On the other hand, PrivateLink offers enhanced security—but at a higher cost and complexity. You pay to consume the service via PrivateLink endpoints. And in the case of outbound PrivateLink connections—such as when a managed connector needs to access private resources within your VPC—you, as the user, are responsible for provisioning the PrivateLink service, paying for the Network Load Balancer and managing the connectivity into your VPC.

As a result, teams are often forced to compromise between cost efficiency and preserving their security posture. With PNI, we eliminate this trade-off, with secure cost-optimized private connectivity. 

Learn how to preserve cloud cost efficiency and robust security with Confluent PNI, now available on AWS.

What Is PNI?

Confluent PNI is our new secure, low-cost private networking option. It’s built directly on fundamental AWS networking primitives such as AWS Elastic Network Interfaces (ENI), the same trusted technology that powers popular services such as Amazon EKS and AWS Lambda. This allows it to operate like a native service that leverages the high performance and security of the AWS private network. Today, PNI is supported on Freight clusters and Enterprise clusters on AWS.

By attaching a network interface from your AWS account to a Confluent-managed service, PNI gives you access to Confluent Cloud through an interface directly inside your own VPC. This allows you to apply your security groups to manage all inbound and outbound traffic using familiar tools and workflows.

PNI delivers a single point to define and enforce security policies, control over traffic directionality, freedom from IP address management or routing constraints, and reduced costs for serving and consuming traffic—offering tangible benefits for platform, infosec, and network administrators alike.

The table below compares PNI with other networking configurations. PNI notably offers a more cost-effective approach by:

• Charging for only a portion of traffic that crosses AZs, unlike PrivateLink's data processing and hourly endpoint fees.

• Providing centralized and granular security policies, setting it apart from VPC peering's distributed management and Transit Gateway's more complex firewall integration.

How PNI Lowers AWS Networking Costs

Beyond security and simplicity, PNI offers a strong economic advantage, fundamentally altering the cost curve for your streaming workloads. PNI complements Confluent Cloud’s commitment to providing a highly competitive per-GB cost for Kafka traffic by eliminating intermediary data transfer charges. 

As throughput increases, PNI’s cost efficiencies become more pronounced so that your AWS networking costs don’t penalize your growth. This is how PNI fundamentally alters the cost curve: You pay for the Confluent service you use, not excessively for the networking that delivers it.

Quantifying the Savings

To see these cost savings in action, let’s explore the cost structure for a logging workload. We'll compare this workload using two different configurations: an Enterprise cluster with PNI optimized for low-latency workloads, such as microservices and real-time pipelines, as well as a Freight cluster with PNI optimized for high-throughput, relaxed-latency workloads such as logging, monitoring, and feeding batch pipelines. We’ll compare these to a representative Confluent Cloud Enterprise cluster using PrivateLink.

While Enterprise and Freight clusters are typically optimized for different use cases, the overall percentage of savings remains the same irrespective of the type of workload. For simplicity, we’ll use the same generic logging workload for a direct comparison of the networking economics.

Core Metrics

We also assume the following unit prices, which reflect the new reduced prices we discussed at the beginning of this post:

Unit Prices

Enterprise Clusters With PNI

Enterprise clusters are optimized for any workload or latency. With the launch of PNI for Enterprise clusters, we’re able to reduce the throughput pricing for clusters using PNI to $0.04/GB for Kafka read/write charges. Customers also see an elimination of PrivateLink endpoint charges from their AWS bill. As a result, we’re able to offer the same workload with ~22% reduced networking costs.

Monthly Networking Cost Comparison

Freight Clusters With PNI

Freight clusters make it possible for you to trade sub-100 ms levels of latency for significant cost savings when running relaxed latency workloads such as logging and observability. Coupled with PNI in comparison to traditional PrivateLink networking, the costs are reduced even further. 

Monthly Networking Cost Comparison

For a workload like the one detailed above, Freight’s new pricing and its utilization of PNI delivers 50% in savings when compared to using traditional PrivateLink networking on an Enterprise cluster.

This cost reduction is the direct result of two powerful factors:

1. Elimination of cross-AZ networking costs: Our zone-aligned architecture drastically reduces the most expensive networking fees. First, by writing directly to object storage, we reduce the need for costly data replication between brokers across different AZs. Second, through producer alignment and consumer alignment through follower fetching, we ensure that clients and the brokers they communicate with are located in the same AZ, preventing expensive cross-AZ traffic and driving these networking charges to zero.

2. Reduced data transfer and processing costs: First, Freight clusters leverage newly optimized pricing of $0.03 per GB for Kafka read/write charges. Second, we completely removed the per-GB costs for PrivateLink. This further amplifies your savings by eliminating a significant and often variable component of your bill.

Freight and PNI in Action: Built With Indeed

In 2024, Indeed partnered with Confluent to re-engineer and improve networking cost efficiency. Indeed has a variety of workloads on Confluent Cloud—some that are standard Kafka workloads and some that are uniquely challenging in terms of scale, partitions, and connections. Indeed’s primary goal was to build a more cost-efficient and secure data infrastructure that could scale with its evolving needs. 

To guide this effort, Confluent worked closely with Indeed’s platform engineering team to understand their priorities, which shaped PNI’s architecture.

"Collaborating with Confluent to implement PNI as part of adopting Freight has been fantastic. The solution drives down costs by eliminating private endpoints, writing directly to object storage, and more effectively preventing overprovisioning. The ability to trade unnecessary sub-millisecond latency for large cost savings is crucial for throughput-oriented projects like our logging workload. Moreover, we are able to do this while also improving our overall security posture by integrating with AWS Security Groups to enable fine-grained traffic controls.  Altogether, we expect to see a reduction in infrastructure overhead such as networking by as much as 60%.

Together Freight and PNI are directly aligned with Indeed's goals to create cost-effective and secure infrastructure at scale. Getting managed Kafka at such a cost-effective price point enables us to leverage Kafka in more workloads and use near real-time data to deliver better value for jobseekers and employers."

— Mark Shah, Distinguished Technical Fellow, Indeed

The key objectives were:

• Reduce total cost of ownership by:

  • Cutting network transfer spend by 60% through PNI–particularly around data transfer between client applications and Confluent and Indeed VPC endpoints

  • Slashing replication and cross-AZ write costs by leveraging Amazon S3 for storage

• Strengthen security with tighter egress controls for accessing Confluent’s control plane services, including fully managed private connectors and Apache Flink®.

• Optimize performance by streamlining DNS management, reducing VPC endpoint overhead and refining broker replication—all to foster a more resilient multi-AZ network layer with better observability that is purpose-built for high throughput and low latency.

• Improve resilience by removing dependencies related to VPC endpoints, load balancers, and cross-AZ traffic. With PNI, Confluent and AWS handle much of the routing and connectivity. PNIs provide direct connectivity between VPCs or across AZs, reducing the need to manage complex routing or peering setups. Pre-attach capabilities also enable moving network interfaces across AZs and services, improving failover response times. This simplification means fewer potential points of failure and a more straightforward environment to troubleshoot, enabling faster, more reliable AZ recovery.

• Drop-in solution with no major configuration changes or alternate patterns needed. Behaves like ENI with capability to work cross-VPC and -AZ.

Since adopting Freight clusters and PNI, Indeed has realized meaningful improvements across system resilience, security, and operational efficiency:

• Operational efficiency and cost savings have been amplified through simplification of the architecture. By eliminating unnecessary components like private hosted zone and endpoint processing charges, Indeed has lowered both complexity and cost. This translates to a reduction in overall operational expenses, lower network transfer costs, and fewer engineering resources required to manage unpredictable traffic spikes.

• Secure design improvements have led to a smaller attack surface and strengthened Indeed’s ability to meet the high trust and compliance expectations of its multi-tenant environment. Meanwhile, engineering teams can now respond more quickly to incidents, thanks to better observability and faster detection.

• Resilience and stability have improved. With a reduced blast radius during AZ outages and improved fault tolerance, Indeed is better equipped to scale reliably, even when facing bursty workloads. These improvements have also led to faster recovery times and less downtime, helping the team consistently meet SLOs and avoid SLA penalties.

Get Started Today

With PNI generally available for Freight and Enterprise clusters, now is a great time to see how it can help reduce networking costs without compromising security. Get started and create your cluster today or contact us to ensure that you're using Confluent on AWS in the most cost-effective way.

Check out the PNI documentation to learn more about how to set up PNI, and stay tuned as we expand PNI support across more cloud providers and Confluent Cloud services.

The preceding outlines our general product direction and is not a commitment to deliver any material, code, or functionality. The development, release, timing, and pricing of any features or functionality described may change. Customers should make their purchase decisions based on services, features, and functions that are currently available.

Confluent and associated marks are trademarks or registered trademarks of Confluent, Inc.

Apache®, Apache Kafka®, Kafka®, Apache Flink®, and Flink® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by the Apache Software Foundation is implied by the use of these marks. All other trademarks are the property of their respective owners.