New in Confluent Cloud and WarpStream: Evolving the Data Streaming Platform for AI, Scale, and Control

In the last few months, Confluent has released over 70 new features to our data streaming platform, ranging from new connectors to better networking connectivity to entirely new ways to build streaming applications.

We are not slowing down. Confluent’s DSP can do more, across every pillar, in ways that are more powerful, more complete, and more connected than ever before. We’re accelerating our AI capabilities for running event-driven agents grounded in fresh context, real-time ML in streaming pipelines, and using AI tools to build and iterate faster. 

Stream: Bring Any (Kafka) Workload, at Any Scale to Confluent Cloud with Confidence and Ease

Stream is the foundation of the data streaming platform. For DSP to fulfill its role as the real-time fabric of your business, that foundation has to support a broader mix of workloads, at greater scale, with strong economics and more production confidence. This way, you can consolidate more of your use cases into one managed platform, instead of dealing with pipeline sprawl. 

Broader workload support

Kafka has evolved from a durable event log into a foundation for many workload patterns: from publish-and-subscribe and stream processing to transactions, exactly-once processing, and queues. Confluent’s goal is to expand support of that full spectrum:

• For teams that want to take advantage of our fully managed diskless Freight clusters for its cost efficiency on high-throughput workloads, we’ve expanded its Kafka protocol compatibility to support transactions and idempotent produce, so you can move over more of your applications where duplicates or partial writes are unacceptable.

• For teams that run Kafka Stream (KStreams) applications on Confluent Cloud, we added KStreams Debug View (as part of KStream add-on) to help you troubleshoot KSreams apps faster with real-time visibility into rebalancing, restoration, and lag right in the UI. That makes it much more straightforward for your on-call teams to understand whether an application is healthy, recovering, or blocked, without jumping between tools or manually correlating logs.

• For teams that want to extend Kafka to queue-based workloads, we’ve previously announced the support of Queues for Kafka (QfK, KIP-932). We’ve since then added share consumers to Python client (preview), bringing QfK to one of Kafka's most relevant languages and the broader client ecosystem. 

Greater scale

As more applications and teams depend on Confluent Cloud, the platform must be able to grow and scale with their businesses without forcing engineers to redesign around infrastructure limits. Confluent Cloud now scales up to 32 eCKUs on Enterprise and 252 CKUs on Dedicated (upon request and AWS only - 152 for Google Cloud and 100 for Azure), delivering up to 15 GB/s ingress and 45 GB/s egress for the most demanding Kafka workloads. 

We’ve also increased the share consumers per share group limit on Dedicated clusters by 5x so teams can start using QFK for broader applications especially process automation without worrying much about upper limits. 

Better economics and cost control

Scaling Kafka should not mean accepting unpredictable networking costs or overprovisioning capacity for peak demand. Follower fetching and Private Network Interface (PNI) networking reduce data-transfer costs across Enterprise and Freight clusters. We recently enabled automatic PNI setup to help you take advantage of this low-cost and secure networking type easier. 

On the cost-control front, max eCKU configuration provides a firm capacity guardrail, while several new metrics in Metrics API around connection count, partition count and throttle clients reveal key drivers of eCKU scaling to support client quota decisions. Together, they help teams understand unexpected usage and manage capacity and spend as adoption grows.

Stronger production confidence

The more workloads teams consolidate, the more important it becomes to protect them during upgrades and infrastructure failures. The new release priority feature in Dedicated clusters gives operators the peace of mind during upgrades by sequencing releases to non-production clusters before production. For broader resilience, Cluster Linking now supports disaster recovery and high availability across public and private networks, including supported cross-cloud private deployments.

Easier migration

Supporting more workloads on Confluent Cloud only matters if you can actually move them here without disruptive cutovers or months of coordination. We’ve been telling you about this magical open-source, command-line migration orchestration tool called KCP, and we’ve recently expanded its support for migration from any flavor of open source Apache Kafka clusters to Confluent Cloud regardless of whether it’s on premises or in the cloud. Client migration, the last (and longest) mile of Kafka migration, is also now supported on the tool, so you can transition applications gradually without weeks-long cross-team coordination for a single, high-risk cutover.

Connect: Connect with 100+ Systems and Save More with Predictable Usage

Over the past two years, our Connect ecosystem has expanded to 100+ fully managed connectors, with enhancements across security, observability, private networking, and migration capabilities to achieve unprecedented enterprise readiness. This quarter, there are more added to the family: the Debezium Google Spanner CDC Source Connector, Salesforce V2 Source Connector, and Google Cloud Firestore Sink Connector. 

Additionally, if you are planning to have predictable usage of our Connectors, we are introducing a new pricing option for you to lock in a minimum 50% discount from day one (up to 80%). The new Connect Advantage packages allow you to commit to a steady-state monthly minimum to reduce pricing variability, simplify budget planning, and keep operating costs efficient as workloads scale. Contact your account team to learn more and optimize costs.

Process: Powerful and Accessible Flink for Both Developers and Data Practitioners

Our goal in designing Confluent Cloud for Apache Flink has always been to make it easy for you to build powerful stream processing applications using the languages, semantics, and tooling you already know, while still getting the performance and operational simplicity of a fully managed runtime. We started with Flink SQL for data engineers, and have since extended the platform with programmatic capabilities like the Table API, User-Defined Functions (UDFs), and Process Table Functions (PTFs) for developers who need more expressive code-first control.

A little bit more about Table API that has recently gone GA for Java on GCP and Python on AWS: Table API brings the best of SQL and code within a single architecture, seamlessly integrating with Flink SQL and Kafka topics. Built specifically for serverless, cloud-native operations, it matches or exceeds lower-level DataStream API performance out of the box by leveraging Flink’s cost-based optimizer. Furthermore, it reduces operational maintenance through pre-built operators that abstract away complex state serialization. Advanced event-driven logic is fully supported through PTFs, providing fine-grained state and timer access without the burden of managing raw cluster infrastructure. With the Table API on Confluent Cloud, engineering teams gain an optimized, production-grade engine that accelerates stream processing from raw events into production. They can also use their familiar IDE workflows, CI/CD practices, and version control without sacrificing the simplicity of SQL.

Furthermore, new Scaling Controls with Baseline CFU (LA) guarantee predictable performance for business-critical pipelines by eliminating rescaling pauses, accelerating the path from development to high-scale production.

For additional details regarding how Flink empowers developers to support mission-critical use cases with programmatic capabilities while accommodating familiar dbt- and SQL-based workflows, please refer to this in-depth product blog covering the latest Flink enhancements.

Govern: Unified, Schematized Governance for Hybrid Confluent Environments

As teams run Kafka across on-premises, private cloud, and public cloud environments, they need a consistent way to understand application health, manage data movement, and respond to issues wherever workloads run. Unified Stream Manager (USM) brings these capabilities together into one unified operational experience, so teams can govern their hybrid Confluent environments with greater confidence. 

In the upcoming weeks, expect to see four more updates that expand what you can monitor and manage on USM:

• Kafka Streams UI allows teams to troubleshoot applications faster with centralized visibility on USM for status, versions, and key performance metrics across your hybrid environment. Teams can view applications connected to each cluster, drill into application and thread states to isolate issues and leverage signals such as end-to-end latency, poll/process/commit ratios, and RocksDB state-store metrics to pinpoint bottlenecks.

• USM will add Azure support via Azure Private Link. Only essential metadata, telemetry, and Schema Registry information are shared with Confluent Cloud through Azure Front Door.

• Native alerts in USM will help teams detect, triage and resolve issues faster across hybrid Confluent environments. Route alerts through email, Slack, PagerDuty, and generic webhooks, and move directly from notification into relevant USM views to take action. Review incidents with ease, spot patterns and support auditability over time with in-product alert history. Additionally, restrict access to who can create or modify alerts with RBAC.

• Monitoring for self-managed and fully managed connectors in one single plane of glass will be in limited availability at the end of August with early access later in Q3. You can register existing self-managed Confluent Connect clusters through the USM UI, API, or Terraform without changing their deployments. USM discovers connectors, synchronizes their status and configuration, and helps you detect problems in minutes with real-time connector status and error visibility across your Connector fleet.

Tableflow: Now Enterprise-Ready at Scale with Powerful New Capabilities

Confluent Tableflow introduces three core capabilities this quarter designed to simplify enterprise deployment and operations. Private connectivity for external catalog integration enables metadata synchronization over AWS PrivateLink and Azure Private Link, ensuring Kafka, Tableflow, storage, and catalog access remain aligned under a unified private network model to streamline security compliance. BYOK for Tableflow on Enterprise clusters allows organizations in regulated industries to maintain custom encryption and governance controls while adopting Tableflow on a scalable cluster architecture. Additionally, enhanced observability features including actionable error messaging, automated alerts, exportable metrics, and native logging integrations provide centralized monitoring and management for enterprise data lake pipelines at scale.

Similar to the new Connect Advantage Packages, our new Tableflow Advantage packages also offer a 50% baseline discount on topics with a minimum monthly commitment, ensuring cost predictability for steady-state usage. Once activated, you can automatically receive the most favorable rate between this baseline and standard tiered pricing, with potential savings reaching up to 90% at scale. Reach out to your Confluent account team to save more today. 

AI: Making Agents Native to the Stream

Confluent Intelligence is our solution for building AI on real-time data: a suite of capabilities spanning both Confluent Cloud and Confluent Platform that helps teams build ML, GenAI, and agentic AI applications directly on top of Kafka and Flink. It brings together AI/ML functions, Streaming Agents, and Real-Time Context Engine to create scalable, intelligent event-driven AI systems. 

The evolution is that Confluent is not only the streaming backbone behind AI workflows – it’s the platform where organizations can build, run, and reliably serve real-time, context-aware AI in production.

AI tools complement this by making Confluent easier to access from the AI assistants and coding agents that developers already use. Through Confluent MCP servers and Agent Skills and now Confluent Copilot, these tools let teams build streaming applications, troubleshoot, and operate in natural language – reducing adoption friction and day-to-day management. 

This quarter, we’re announcing robust anomaly detection and forecasting with expanded model support for IBM Granite Times Series models and TimesFM models. We’ve upgraded the experience of using Real-Time Context Engine with user-facing metrics. And there’s lightning queries in early access, providing a super fast way to query materialized views to get fresh context around the current business state. All of this grounds AI in reliable data and supports enterprise AI use cases in production.

Learn more in the in-depth AI product blog.

Security: Extend Your Existing Security Controls to Confluent Cloud

Security is table stakes, but enterprise security becomes harder when every new platform introduces another identity model, networking pattern, encryption standard, or monitoring workflow. We want to make Confluent Cloud fit the controls your teams already use, so you can secure streaming workloads without creating exceptions or duplicating operational work.

Reduce your reliance on static secrets

Static credentials create exposure and ongoing work: they must be stored, rotated, and revoked across every application. Our new OAuth plugins let Kafka clients authenticate with AWS IAM and Azure Managed Identities, extending your cloud provider’s managed identity lifecycle to Confluent Cloud without storing client secrets. We’ve also rolled out mTLS with certificate revocation to Enterprise, Freight and Dedicated clusters to provide certificate-based authentication, and IP filtering for Kafka to add another safeguard by limiting public cluster access to trusted networks, even if credentials are compromised. 

For other authentication patterns, Secret Manager support keeps fully managed connectors credentials in your existing cloud vault. For workflows that still rely on API keys, Global API Keys reduce credential sprawl across supported Confluent Cloud services and clusters.

Protect data without limiting where workloads can run

Encryption requirements should not force teams onto a particular cluster type or leave them relying on older transport standards. Bring-Your-Own-Key (BYOK) now extends customer-controlled encryption at rest to Freight clusters on AWS, while TLS 1.3 strengthens encryption in transit across Confluent Cloud.. 

Keep more data paths private

A private Kafka cluster is only part of a private architecture. Data can still cross public networks when connectors, processing workloads, governance services, or external systems sit outside that boundary.

PNI egress now lets fully managed connectors privately reach external systems using the cluster’s existing PNI connection, avoiding separate networking infrastructure for each data path. This expands Confluent Cloud’s broader private-networking coverage across Kafka, Flink, Schema Registry, and Unified Stream Manager, alongside PrivateLink support for custom connectors and Flink access to external models and tables on Azure.

Investigate activity in the tools your teams already use

Audit data is only useful when security teams can access and investigate it quickly. Our new Audit Log Monitoring UI brings that workflow directly into Confluent Cloud, while push-based observability integrations deliver logs to Datadog and Splunk (in Early Access) without requiring teams to build and operate custom export pipelines.

WarpStream: BYOC Diskless Kafka Keeps Getting Better

For those who prefer a BYOC solution, WarpStream’s latest batch of updates and feature releases continue to make WarpStream the fastest, most reliable, and easiest-to-use diskless Kafka offering.

Lightning Topics and Ripcord Mode

Lightning Topics let WarpStream acknowledge produce requests as soon as data is durably written to object storage, deferring offset sequencing to an asynchronous background step instead of blocking on it. Combined with S3 Express One Zone, this cuts median produce latency to 33ms, a 70% reduction, making WarpStream viable for latency-sensitive workloads and the fastest diskless Kafka-compatible data streaming product on the market. 

Ripcord Mode lets WarpStream Agents keep accepting produce requests even when the Control Plane is completely unreachable, by journaling records to object storage and committing them asynchronously once connectivity is restored. This eliminates the Control Plane as a hard dependency for writes, so applications that can't backpressure or drop data stay up through networking issues or Control Plane incidents.

Check out the technical deep-dive blog that goes over both these features, as well as this explainer video:

Audit Logs

Audit Logs solve a common pain point: incomplete, scattered visibility into who changed what, which becomes a real problem in regulated environments or during incident response. WarpStream now records every auth decision and cluster- or account-level operation as a queryable, Kafka-native event stream, enabled with a single click and no added infrastructure. 

Learn more about Audit Logs via this blog.

Events

Events lets customers search and visualize Agent logs, ACL decisions, and Managed Data Pipeline execution logs natively in the WarpStream Console, powered by a custom query engine built on top of Kafka itself, with global status and individual event types configurable via the Console, API, or Terraform. Because event data stays inside the customer's own object storage and queries run directly on the Agents, teams get deep, high-cardinality visibility without paying egress costs or standing up a separate observability system. 

Learn more via this blog.

MCP Server

The WarpStream MCP Server connects AI assistants like Claude, Cursor, and Windsurf directly to your clusters, exposing 30+ read-only tools to query cluster health, search event logs, inspect ACL decisions, and explore topics without leaving your IDE. This turns common, operational questions like, "Is anything wrong with my cluster?" or, "Who denied that ACL operation?", into natural-language prompts instead of manual dashboard hunts or hand-written queries. 

Learn more via this blog.

GCP Rapid Buckets

GCP Rapid Buckets support brings low-latency object storage to WarpStream on GCP for the first time, using pre-warmed writer pools and unfinalized file writes to cut P99 file-write latency roughly 3x. This closes WarpStream's last major latency gap across cloud providers, letting GCP customers achieve sub-150ms P99 produce latency to match what's already possible on AWS (S3 Express One Zone) and Azure (premium Blob storage). 

In this blog, learn not only about the feature, but the engineering work that went into creating it.

Orbit Auto Migration

Orbit Auto Migration lets teams cut over Kafka producers from a source cluster to WarpStream without a coordinated maintenance window – producers are repointed at their own pace while WarpStream transparently proxies writes to source until each topic is individually cut over with a single click or API call. This eliminates the traditional all-hands, high-risk migration event (stop every producer, drain lag, restart everyone at once) in favor of a gradual, per-topic, low-risk cutover with automatic rollback if replication lag doesn't drain in time. 

To learn more, read our in-depth blog, as well as watch this explainer video:

Start Building with New Confluent Cloud and WarpStream Features

So many updates and these are just all that we can fit in one blog! The best way to learn about all the new features is to try them out. If you’re new to Confluent, sign up free for Confluent Cloud and create your first cluster to explore the new features. New sign-ups receive $400 to spend within Confluent Cloud during their first 30 days. Use the code CLOUDBLOG60 for an additional $60 worth of free usage.

If you need a BYOC solution, you can sign up for a free WarpStream account. All sign-ups get $400 in credits that never expire and no credit card is required to start.

Get started today:

• Sign up for Confluent Cloud to try new Q3 features.

• Sign up for WarpStream to try the latest. 

• Get Early Access to new AI features to build real-time, context-aware agents faster.

Apache®, Apache Kafka®, Apache Flink®, Flink®, and the respective logos are trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by the Apache Software Foundation is implied by using these marks. All other trademarks are the property of their respective owners.