Confluent Cloud for Apache Flink: Engine for Mission-Critical, Real-Time Operational Systems and dbt/SQL-Native Home for Data Science and AI

Organizations today are under immense pressure to deliver on two critical fronts: building mission-critical, real-time operational systems and powering the next generation data science and artificial intelligence (AI) workflows with analytics-ready data. Historically, achieving both meant navigating a divided, complex architecture. Developers wrestled with distributed stream processing engines to power operational applications, while data scientists and analytics engineers relied on completely separate batch-oriented systems to feed their models, data lakes, and dashboards. The result? Duplicated infrastructure, brittle ETL pipelines, fragmented governance, and data staleness that ultimately held back innovation.

For our Q3’26 Launch, we are announcing a massive evolution of Confluent Cloud for Apache Flink®. We have unified the data streaming platform so it serves as both the robust engine for mission-critical real-time operations and a dbt/SQL-native home for analytics and AI teams. Apache Kafka® and Flink are no longer just integrated; they are seamlessly co-designed into a single, serverless product.

Anchoring to our Q3 launch theme —"Every Pillar of the Platform Has Evolved"—this announcement marks a turning point in stream processing. By breaking down the operational and analytical divide, Confluent is unleashing the next era of data in motion. Whether you are a developer building stateful applications or a data engineer maintaining always-fresh data products, Flink now meets you exactly where you work.

Mission-Critical Power for Developers

To support developer-led teams, we are bringing full programmatic power directly to our serverless runtime. While Flink SQL is fantastic for many analytical use cases, developers often need the flexibility of pure code to express complex business logic. Previously, that meant turning to the lower-level DataStream API, taking on significant operational baggage and cluster management along the way. Not anymore.

Table API General Availability

The Flink Table API is now Generally Available in Java. This gives developers a code-first way to build sophisticated Flink applications while retaining the operational simplicity of a serverless, fully managed runtime. For the vast majority of streaming workloads—roughly 95% to 99% of use cases—the Table API matches or outperforms the DataStream API. It provides a unified, declarative API for both batch and stream processing, allowing developers to focus purely on their business logic rather than the underlying infrastructure.

Process Table Functions (PTFs)

For the critical edge cases that standard SQL simply cannot express, we are introducing Process Table Functions (PTFs). PTFs act as the “stateful brain” of your streaming applications. They allow developers to maintain custom state and configure precise timers within their pipelines. If you are building complex, event-driven architectures for fraud detection, dynamic pricing, or multi-metric anomaly detection, PTFs provide the granular control you need without forcing you to abandon the Table API ecosystem. PTFs are now generally available in Java on all cloud service providers i.e., AWS, Azure and GCP.

External Connectivity and UDFs

We are extending Flink's programmability everywhere developers work. User-Defined Functions (UDFs) now support Java on Google Cloud (GCP) and Python on Amazon Web Services (AWS), effectively closing the multi-cloud UDF gap. This means you can seamlessly bring your custom transformation logic, written in the languages your engineering team already knows, directly into Confluent Cloud.

Serverless Flink Apps and Hardened Resilience

Think of our serverless stateful processing as “AWS Lambda, but with state.” With Flink Apps, developers get exactly-once processing semantics, auto-scaling capabilities, and absolutely zero infrastructure to provision, patch, or manage.

Because we know these systems run your business, we’ve hardened the platform for mission-critical pipelines. We’ve implemented automatic cross-Kubernetes failover, robust Flink SQL recovery mechanisms, failure-pattern detection, and significantly faster checkpoints. We’ve also added new Scaling Controls (Baseline CFU) to give latency-sensitive workloads a guaranteed performance floor, ensuring your most important applications never skip a beat.

The proof is in our customers' success. MasterControl transformed their processing pipelines, drastically cutting data processing time from 3 hours to under 5 minutes. Henry Schein One shifts ownership upstream delivering governed, production-ready data without downstream rework eliminating costly reprocessing. Industry leaders like Fiserv and JPMC trust our platform to handle their most demanding, high-throughput financial workloads.

Building Always-Fresh Data Products for Analytics Engineers

While developers thrive in code, analytics engineers and data scientists need to work in their existing declarative tools without being forced to learn complex streaming paradigms. Over the last year, we are trying to meet data teams directly in their native workflows.

The dbt Adapter and Materialized Tables

Data engineers rely on dbt to manage their batch processing pipelines. When moving to streaming, they shouldn't be forced into unfamiliar CLI tools or manual workflows. With the General Availability of our dbt adapter and Materialized Tables, teams can now manage their entire Flink pipeline lifecycle using the exact same dbt run workflow they use for data warehouses like Snowflake or Databricks.

Materialized Tables transform the way data teams think about streams. Instead of managing ephemeral, short-lived streaming statements, you manage persistent, database-like assets. If you need to evolve a pipeline—add a column, tighten a filter, or change logic—Flink handles the stop, restart, offset management, and catch-up completely under the hood. It’s the "easy button" for data engineering, ensuring that downstream dashboards, data lakes, and lakehouses are fed with always-fresh, analytics-ready data products.

One Engine for Streaming and Batch

Why manage two separate compute engines when one can do it all? Flink natively unifies batch and streaming. With Snapshot Queries and Table API batch mode, you can query historical data and live streams together without rewriting your code.

Snapshot Queries eliminate friction by allowing you to run one-time SQL queries that return a complete, point-in-time view of your data using union reads. This combines the deep historical records stored in Tableflow with the latest real-time events in Kafka. Because these queries run against Apache Iceberg or Parquet formats, they are exponentially faster than scanning raw Kafka streams. This eliminates the necessity for separate batch clusters and standalone real-time nodes. You get one unified platform with the same Flink SQL, the same compute pools, and a single billing model.

AI-Ready Processing: Enriching Data for the AI Era

Generative AI applications and large language models (LLMs) are incredibly powerful, but they suffer from a fundamental limitation: they lack real-time context. An AI model trained yesterday cannot answer questions about a transaction happening right now. To power valuable AI outcomes, organizations need a Real-Time Context Engine, and Flink is the processing layer that makes it happen.

We are making Flink the premier place to enrich data for AI—not just move it. With built-in ML and AI functions, Flink treats models as first-class citizens. Flink AI Model Inference allows you to call remote model endpoints directly from Flink SQL. You can seamlessly enrich your live data streams with real-time inference, embedding generations, and vector search lookups without building brittle, complex data synchronization pipelines. With the introduction of Multivariate Anomaly Detection (GA) and Granite model support, AI-ready data streams are more accessible than ever.

Furthermore, we are moving beyond simple data movement and into agentic decision-making. With Streaming Agents, developers can build event-driven, always-on agents directly on Flink. Flink isn't just processing rows; it is continuously serving fresh context to any AI system via the Model Context Protocol (MCP), ensuring your AI applications have the up-to-the-millisecond data they need to make accurate, trustworthy decisions.

A Unified Platform: Kafka and Flink Co-Designed

The true through-line of the Q3 launch is that every pillar of the Data Streaming Platform has evolved. But more importantly, they have evolved together.

Kafka and Flink are not disparate systems awkwardly stitched together. They are one co-designed product. They share a unified governance model powered by Confluent Schema Registry, ensuring that as data flows from Kafka into Flink, it remains compliant, schematized, and fully trackable through data lineage. Features like Schema IDs in Kafka headers power contextualized AI apps and analytics as data flows through Kafka into Flink, Tableflow, and your lakehouse with consistent, validated schemas.

They also share a comprehensive security posture. Over the last year, we delivered granular Role-Based Access Control (RBAC) at the pool level, Bring Your Own Key (BYOK) encryption for Flink state, and Client-Side Field Level Encryption (CSFLE). We've extended private networking coverage across Kafka, Flink, Schema Registry, and our Unified Stream Manager, giving enterprises the secure boundary they require for highly sensitive workloads.

Finally, they share the same Kora storage engine and the same underlying architecture. When you build on Confluent, you are building on a platform where the stream and the table, the operational and the analytical, the developer and the data scientist, are all finally brought together.

Predictable Flink Pricing with Advantage Packs

As you scale your streaming workloads, unpredictable costs shouldn't slow you down. For production-oriented customers with known steady-state usage, we recently introduced Flink Advantage Packs to provide predictable spend and better price-performance from day one. By committing to a minimum baseline (starting at 25 Confluent Flink Units per cloud provider), customers receive a higher statement allowance—up to four statements per CFU instead of the standard one.

This translates to up to 75% effective savings for your Flink workloads compared to standard pay-as-you-go pricing, making it incredibly cost-effective to run many small statements or expand your stream processing footprint without breaking the bank. Contact us or your Confluent account team to start saving today.

Getting Started

The evolution of Confluent Cloud for Apache Flink represents a fundamental shift in how organizations can harness their data in motion. By delivering full programmatic power for developers, seamless SQL and dbt workflows for analytics engineers, and native AI integrations for data scientists, we are eliminating the compromises of the past. It’s time to stop bridging the gap between your operational and analytical systems and start running them on a single, unified platform.

Ready to experience the power of the evolved Data Streaming Platform? Sign up for Confluent Cloud today. New sign-ups receive $400 to spend during their first 30 days—no credit card required.

The preceding outlines our general product direction and is not a commitment to deliver any material, code, or functionality. The development, release, timing, and pricing of any features or functionality described may change. Customers should make their purchase decisions based on services, features, and functions that are currently available.

Confluent and associated marks are trademarks or registered trademarks of Confluent, Inc.

Apache®, Apache Kafka®, Kafka®, Apache Flink®, Flink®, Apache IcebergTM , IcebergTM , Apache ParquetTM , and ParquetTM are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by the Apache Software Foundation is implied by using these marks. All other trademarks are the property of their respective owners.