Apache Kafka

Confluent, powered by Kafka, is the real-time backbone for agentic systems built with Google Cloud. It enables agents to access fresh data (MCP) and communicate seamlessly (A2A) via a decoupled architecture. This ensures scalability, resilience, and observability for complex, intelligent workflows.

AWS Lambda's Kafka Event Source Mapping now supports Confluent Schema Registry. This update simplifies building event-driven applications by eliminating the need for custom code to deserialize Avro/Protobuf data. The integration makes it easier and more efficient to leverage Confluent Cloud.

Confluent’s Cluster Linking enables fully managed, offset-preserving Kafka replication across clouds. It supports public and private networking, enabling use cases like disaster recovery, data sharing, and analytics across AWS, Azure, Google Cloud, and on-premises clusters.

Confluent Cloud now offers native Kafka Streams health monitoring to simplify troubleshooting. The new UI provides at-a-glance application state, performance ratios to pinpoint bottlenecks (code vs. cluster), and state store metrics.

Learn how to scale Kafka Streams applications to handle massive throughput with partitioning, scaling strategies, tuning, and monitoring.

Learn how to choose the right Apache Kafka® multi-cluster replication pattern and run an audit-ready disaster recovery and high availability program with lag SLOs, drills, and drift control.

Learn how to handle data transformation, schema evolution, and security in Kafka Connect with best practices for consistency, enrichment, and format conversions.

Learn best practices for validating your Apache Kafka® disaster recovery and high availability strategies, using techniques like chaos testing, monitoring, and documented recovery playbooks.

Learn best practices for running Kafka Connect in production—covering scaling, security, error handling, and monitoring to build resilient data integration pipelines.

Announcing the release of Apache Kafka 4.1

Confluent Tableflow unifies operational and analytical data by integrating Kafka with zero ETL, leveraging open table formats such as Iceberg and Delta Lake. It offers advantages over traditional zero ETL by enabling data reuse, schema decoupling, and better scalability, streamlining data sharing.

Big news! KIP-848, the next-gen Consumer Rebalance Protocol, is now available in Confluent Cloud! This is a major upgrade for your Kafka clusters, offering faster rebalances and improved stability. Our new blog post dives deep into how KIP-848 functions, making it easy to understand the benefits.

The users who need access to data stored in Apache Kafka® topics aren’t always experts in technologies like Apache Flink® SQL. This blog shows how users can use natural language processing to have their plain-language questions translated into Flink queries with Confluent Cloud.

For AI agents to transform enterprises with autonomous problem-solving, adaptive workflows, and scalability, they need event-driven architecture (EDA) powered by streaming data.