New in Confluent Cloud: Bringing Together the Best of Batch and Stream Processing

Your teams want the immediate insights of stream processing with the scale and historical context of batch processing—but traditional data infrastructure forces you to resort to disparate tooling or manual workarounds to bridge that gap. This quarter’s release, coming to you live from Current London, brings new features in Confluent Cloud that fundamentally change this dynamic by seamlessly unifying stream and batch processing. We’re unveiling innovations such as snapshot queries in Confluent Cloud for Apache Flink® to enable fast, batch-style queries across your Apache Kafka® and Tableflow data—so that you can fuel your artificial intelligence (AI) and analytics workloads with rich historical context in addition to real-time streaming insights. We’re also excited to introduce Tableflow support for Delta Lake tables to power workloads in the Databricks ecosystem, more clouds and cost control for our serverless clusters, and a suite of features for Schema Registry and Apache Flink® designed to simplify your private networking experience.

Keep reading for a full breakdown to get the most out of our new cloud features.

• Unify Batch and Stream With Apache Flink®

• Tableflow Now Supports Delta Lake Tables

• Expanding Secure Networking for Flink and Schema Registry

• Enterprise Clusters on Google Cloud

• Cross-Cloud Cluster Linking

• Additional New Features and Updates

Unify Batch and Stream With Apache Flink®

Snapshot Queries (Early Access)

We’re thrilled to announce the Early Access of Snapshot Queries in Confluent Cloud for Apache Flink®, which enables fast, batch-style queries across Kafka topics as well as Apache Iceberg™️ and Delta Lake tables via Tableflow. With Snapshot Queries, you can now run both batch and streaming workloads on Confluent Cloud using the same Flink SQL interface.

Snapshot Queries enable you to:

• Query historical and real-time data using Flink SQL for unified stream and batch processing

• Tap into long-retention tables on Tableflow at interactive speed for richer historical context

• Accelerate pipeline development and debugging with ad hoc, point-in-time queries

When you set snapshot mode (e.g., SET 'sql.snapshot.mode' = 'now'), data sources are automatically bound, and the execution switches to batch. This allows you to explore, test, or analyze data with queries that compute results once and terminate.

Modern data teams rely on both streaming and batch to react to data in real time and leverage historical context. Today, these workloads are typically split across multiple tools, leading to fragmented developer workflows. Confluent’s serverless Flink breaks through that friction by offering both streaming and snapshot modes.

In particular, snapshot queries on Tableflow are optimized to run at interactive speeds, delivering results up to 50-100x faster than running the same logic as a streaming job over historical data. This makes pipeline development, query testing, and data reprocessing significantly more flexible and convenient. Developers can leverage this feature as part of an iterative workflow for use cases that were previously impractical with long-running, continuous jobs.

This feature currently supports append-only queries across both Kafka and Tableflow. To try Snapshot Queries, sign up for the Early Access program. As we move toward general availability, we plan to expand support to all query modes.

User-Defined Functions (UDFs) in Java on Azure

UDFs let you go beyond the built-in capabilities of Flink SQL, enabling more advanced transformations, filtering, and aggregations tailored to your real-time applications. At the beginning of this year, we announced the availability of UDFs in Java on Amazon Web Services (AWS), including both scalar and table functions. Now we’re extending UDFs to Azure.

Developers working in Azure environments can:

• Customize Flink SQL by implementing custom business logic or complex transformations tailored to their use cases

• Leverage their preferred programming language and existing libraries, streamlining integration with existing systems

• Promote reusability across multiple applications and pipelines, ensuring consistency and reducing development time

You can invoke UDFs directly from the SQL editor, enabling richer, more expressive queries that are tailored to your specific data processing needs. Stay tuned as we expand UDF support to Python and additional cloud platforms.

Confluent for VS Code Adds Support for Confluent Cloud for Apache Flink®

Earlier this year, we announced the general availability of the Confluent for VS Code extension, which makes it easy for developers to connect to any Kafka cluster to develop, manage, and monitor real-time data streams without switching between multiple tools. Today, we’re excited to announce that this extension now supports developing applications with Confluent Cloud for Apache Flink®. Streaming engineers can seamlessly build, manage, and monitor real-time data processing applications powered by Flink—directly from their preferred code editor, Visual Studio (VS) Code.

With this enhancement, developers can:

• Manage Flink SQL deployments alongside Kafka resources, all within a single development environment

• Author, validate, and deploy Flink SQL statements easily, without context switching

• Streamline project setup and development workflows with built-in templates for streaming and stream processing applications

Get started today by installing the Confluent for VS Code extension for free from the Visual Studio Marketplace so that you can experience real-time data development right in your integrated development environment.

Tableflow Now Supports Delta Lake Tables (Open Preview)

Last quarter, we announced the general availability of Tableflow, which represents Kafka topics and associated schemas as open table formats to feed any data lake, warehouse, or analytics engine. We’re glad to announce that Tableflow now supports Delta Lake tables, available in Open Preview. This update simplifies the process of feeding high-quality data streams directly into Databricks and other compatible analytics engines, making it easier to power demanding AI and analytics workloads. And soon you’ll be able to publish materialized Delta Lake tables to Unity Catalog for centralized governance.

For added flexibility, Tableflow also supports dual materialization, allowing a single topic to populate both Delta Lake and Iceberg tables simultaneously so that the underlying data can be shared efficiently.

Expanding Secure Networking for Flink and Schema Registry

CCN Routing for Flink and Schema Registry

We're introducing Confluent Cloud Network (CCN) routing support for Flink and Schema Registry as another step forward in simplifying secure connectivity. While provisioning PrivateLink Attachments is one way to set up dedicated private connections for both products, CCN routing provides an even more streamlined experience. Crucially, this capability allows you to reuse your existing CCN infrastructure to securely connect both Flink and Schema Registry without additional networking setup, thereby enhancing operational efficiency and reducing complexity. Each CCN now includes dedicated Flink and Schema Registry endpoints, enabling secure and flexible access with support for both public and private DNS.

CCN routing for Flink is now generally available on AWS in all regions where Flink is supported, and support for Azure is coming soon. CCN routing for Schema Registry is coming soon to AWS.

IP Filtering for Flink and Schema Registry

Achieving compliance with strict security postures and maintaining fine-grained control over access are simplified when the right tools are in place. That’s why we’re pleased to announce the general availability of IP Filtering for both Flink and Schema Registry. IP Filtering provides enhanced control over how these services are accessed publicly, allowing network administrators to define trusted IP address ranges (CIDR blocks) to explicitly allow or deny access to both Schema Registry and Flink public endpoints. Used in conjunction with features like CCN routing, IP Filtering gives you the flexibility to secure your Confluent Cloud resources with minimal networking overhead.

AWS PrivateLink for Schema Registry

It’s our pleasure to announce the general availability of AWS PrivateLink for Schema Registry, a further enhancement to our security options that allows organizations to connect to Schema Registry through a private endpoint within Amazon Virtual Private Cloud (VPC). This ensures that all traffic remains within their VPCs, eliminating schema metadata exposure over the public internet. See the full list of supported regions here.

Enterprise Clusters on Google Cloud

We’re delighted to announce the availability of Enterprise clusters on Google Cloud, which means they’re now available across all three major cloud providers. Enterprise clusters offer a fully managed, cost-effective solution to meet the needs of streaming workloads of any latency and throughput. These serverless, autoscaling clusters seamlessly handle spikes in demand without requiring complex capacity planning or overprovisioning, ensuring optimal cost efficiency. Enterprise clusters on Google Cloud also provide secure connectivity through Google Cloud Private Service Connect, enabling you to run your Kafka workloads with confidence.

Enterprise clusters are currently supported in 12 Google Cloud regions, with more availability on the way. See the full list here.

Cross-Cloud Cluster Linking (Limited Availability)

Many organizations operate across multiple cloud providers to optimize costs, meet regulatory requirements, and ensure data resiliency. However, linking Kafka workloads and replicating data across these different cloud environments can introduce significant complexity, often requiring custom networking setups, VPN tunnels, or complicated peering agreements. To address these challenges, Confluent is introducing cross-cloud Cluster Linking, currently in Limited Availability and ready for production workloads. This capability enables data replication between Kafka clusters on AWS and Azure that are privately networked through VPC peering or PrivateLink.

This capability is crucial for a range of use cases, including implementing multicloud strategies, facilitating smooth cloud-to-cloud migrations without downtime, and ensuring disaster recovery by maintaining hot standby clusters in different clouds. Cross-cloud Cluster Linking currently allows organizations to replicate data between any combination of Enterprise and Dedicated clusters on AWS and Azure. The feature also supports clusters that are privately networked through VPC peering or PrivateLink, ensuring secure data transit.

Additional New Features and Updates

WarpStream Updates

WarpStream Diagnostics

We’re pleased to introduce WarpStream Diagnostics, which continuously analyzes your clusters to identify potential problems, cost inefficiencies, and ways to make things better. Composed of more than 20 built-in diagnostic checks (with more on the way), WarpStream Diagnostics helps you identify inefficiencies like unnecessary cross-AZ networking, bin-packed or non-optimized instances, and inefficient produce and consume requests.

Metrics can be viewed in the WarpStream console. To learn more, visit the announcement blog.

WarpStream Schema Linking

WarpStream is enhancing its Data Governance suite with the introduction of Schema Linking, a tool to continuously migrate any Confluent-compatible schema registry into a WarpStream Bring Your Own Cloud (BYOC) Schema Registry. A key advantage of WarpStream Schema Linking is its meticulous preservation of your existing schema architecture. It faithfully maintains schema IDs, subjects, subject versions, and compatibility rules.

Natively embedded in the WarpStream Agent, Schema Linking eliminates the need for extra infrastructure. Beyond migration, you can leverage Schema Linking for scalable read replicas, multi-region schema synchronization, and robust disaster recovery. Learn more in the announcement blog.

Snowflake Source Connector

Our new, fully managed Snowflake Source Connector enables users to stream data from Snowflake tables directly into Kafka topics, facilitating reverse ETL use cases. Built on JDBC, this connector supports custom queries and offset management to give you fine-grained control over what data moves and when.

Azure Cosmos DB v2 Source and Sink Connectors

We're happy to introduce the new Azure Cosmos DB v2 source and sink connectors, built to deliver reliable, scalable, and secure data integration. The v2 connectors offer exactly-once delivery semantics, multi-container support, and improved bulk operations for faster, more consistent pipelines. The source connector now eliminates the need for a lease container and supports offset-based change tracking, while the sink connector adds flexible write strategies like patch and delete. These new connectors offer a more scalable, secure, and production-ready integration experience.

Kafka Streams User Interface (UI) for Confluent Cloud

Our first dedicated UI for Kafka Streams provides a UI to monitor the operational health of your Kafka Streams applications. Easily identify application owners, manage workloads with a dedicated view, ensure compatibility with instant version checks, and more!

New Consumer Rebalance Protocol (KIP-848)

The new Consumer Rebalance Protocol (KIP-848), part of the recent Apache Kafka 4.0 release, significantly enhances the reliability and responsiveness of consumer groups, especially in large-scale deployments. This new protocol is available for production usage in Confluent Cloud, with support on Confluent Platform coming soon. KIP-848 shifts more coordination logic to the broker side, enabling faster, incremental, server-assisted rebalances. This results in lighter clients, reduced downtime, and improved overall stability for your streaming applications.   

Start Building With New Confluent Cloud Features

If you’re new to Confluent, sign up for a free trial of Confluent Cloud and create your first cluster to explore the new features. New sign-ups receive $400 to spend within Confluent Cloud during their first 30 days. Use the code CCBLOG60 for an additional $60 worth of free usage.*

The preceding outlines our general product direction and is not a commitment to deliver any material, code, or functionality. The development, release, timing, and pricing of any features or functionality described may change. Customers should make their purchase decisions based on services, features, and functions that are currently available.

Confluent and associated marks are trademarks or registered trademarks of Confluent, Inc.

Apache®, Apache Kafka®, Apache Flink®, Apache Iceberg™️, and the respective logos are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by the Apache Software Foundation is implied by using these marks. All other trademarks are the property of their respective owners.