Confluent: The Real-Time Backbone for Agentic Systems

In the evolving landscape of agentic systems, Confluent and Google Cloud together emerge as critical enablers, providing the real-time infrastructure that underpins efficient, reliable, and intelligent data flow. This powerful synergy addresses key challenges in agent-to-agent (A2A) communication, interaction with external resources, and the overall stability and observability of complex multi-agent environments.

Generative AI (GenAI) has captured the spotlight, but its true power is often limited by a hidden challenge: the freshness of its data. While large language models (LLMs) are trained on vast amounts of information, this data can quickly become outdated, leaving models unable to provide real-time context.

This is where Confluent’s data streaming platform, powered by our cloud-native Apache Kafka® engine, comes in. In this post, we’ll take you through how Confluent and Google Cloud are powering the real-time backbone multi-agent systems.

Ready to try Confluent Cloud? Sign up in the Google Cloud Marketplace and get $1,000 in credits.

Bridging the Gap: Real-Time Data for Intelligent AI Agents

By acting as a central nervous system for data movement, Confluent helps businesses overcome these limitations. It streams fresh, real-time data from various sources directly into powerful AI services like Google Cloud's Vertex AI. But the journey of this data doesn't end there. For a sophisticated multi-agent system, the LLM is just one component. This is where two crucial communication protocols, Model Context Protocol (MCP) and A2A, come into play.

Confluent's platform provides a real-time, trustworthy knowledge base for GenAI applications through a four-step process:

• Stream: Confluent continuously streams data from any environment, providing LLMs, vector databases, and other applications with up-to-the-minute information as it becomes available.

• Connect: With more than 120 pre-built connectors, Confluent simplifies the process of integrating data from various sources, breaking down data silos and creating seamless streaming pipelines.

• Govern: Confluent's Stream Governance ensures data lineage, quality, and traceability, building trust and transparency in AI solutions by providing a clear understanding of the data's origin and usage.

• Process: Using the Apache Flink® protocol, Confluent processes data as it's ingested, allowing transformations and AI model inference to occur with minimal latency. This unifies data processing and AI workflows, enabling models to react to real-time events.

This unlocks the foundation for GenAI LLMs, ensuring that AI agents have the most up-to-date information to complete their tasks before engaging the next agent.

The benefits are clear: improved accuracy, better performance, and reduced costs for training and deploying these models. Furthermore, the A2A protocol leverages this same real-time data backbone to enable agents to talk to each other. Once an agent, using MCP, has identified a customer issue, it can publish an A2A message—a standardized "event"—to a Kafka topic. Another agent, perhaps one specialized in order fulfillment, can then subscribe to that message and take immediate action.

By leveraging Confluent, organizations can deliver richer customer experiences, boost sales with better recommendations, and automate support tasks more effectively—all powered by real-time data that keeps pace with a rapidly changing world and is orchestrated by the powerful combination of A2A and MCP.

A2A and MCP Defined

A2A and MCP are two distinct but complementary protocols that define how agents and systems communicate within a multi-agent environment.

• A2A defines the communication mechanisms specifically for agents to talk to other agents. It enables information exchange, task delegation, and workflow orchestration between different agents in a cooperative system.

• MCP defines how agents integrate with and access external resources and tools. This includes interacting with databases, calling APIs, using legacy applications, or consuming data from streaming sources, allowing agents to gather real-time context and perform actions outside of their own internal logic.

Without a platform like Confluent, A2A and MCP would rely on tightly coupled, point-to-point connections. For A2A, agents would need to know the specific location of every other agent they interact with, creating a fragile system where any change requires widespread updates and messages lack durability. For MCP, agents would have to build custom clients for each external tool, database, or API, leading to protocol mismatches, increased development complexity, and inefficient polling for data—all of which hinder real-time context and scalability.

This direct communication model lacks a centralized system for observability, message durability, and protocol standardization, making the entire multi-agent system difficult to manage, scale, and debug.

The MCP/A2A Proxy: Bridging Communication for AI Agents

A cornerstone of this architecture is Confluent’s MCP/A2A proxy. This vital intermediary streamlines how your AI agents and their tools communicate across diverse systems. It intelligently translates varied communication protocols, such as those defined by MCP and A2A, into standardized, robust messages managed within Confluent Cloud.

Key takeaways of the proxy include the following.

• Unified Registry: The proxy provides a centralized registry, making it easy for agents to discover and connect to available agents, tools, and external resources, eliminating complex point-to-point configurations.

• Protocol Transformation: It seamlessly converts disparate MCP and A2A interactions into a consistent stream of messages within Confluent Cloud. This abstraction simplifies development and ensures reliable data exchange.

• Service Exposure: Crucially, the proxy allows you to readily expose existing microservices, legacy applications, and even real-time Flink jobs as accessible MCP tools and resources or A2A agents. This unlocks immense value from your existing investments by making them consumable by your AI agents.

Decoupled, Event-Driven Design and Asynchronous Communication

Confluent facilitates decoupled communication between agents and external systems and inherently enables an event-driven architecture that is perfectly aligned with the principles of agentic systems. Both A2A interactions and agent-to-external resource operations with MCP can leverage Kafka topics to publish events. Tools and other agents can then subscribe to these topics, ensuring that communication is not tightly coupled.

This asynchronous, event-driven architecture allows agent workflows to operate at their own pace. Confluent reliably delivers messages even if an agent is busy or requires extended processing time, guaranteeing that no critical information is lost. This is particularly crucial for robust agentic systems where individual agents may have varying workloads or processing capabilities.

Agents can react dynamically to other agent messages or external events, fostering a highly responsive and adaptive environment. This event-driven approach promotes loose coupling and scalability, making it easier to design, deploy, and manage sophisticated multi-agent systems.

Stability and Real-Time Context

Complex agents often process massive volumes of data. Confluent ensures the reliable delivery and processing of this data at the pace required by the agent system, even in real-time scenarios. This stability is paramount for applications demanding immediate insights, such as fraud detection, real-time customer service, manufacturing optimization, and dynamic geographic analysis. With Confluent as the data backbone, agents consistently have access to the most up-to-date information, enabling them to make timely and accurate decisions.

A crucial part of building robust agentic systems is enabling agents to communicate effectively with each other and with external resources. MCP and A2A provide the blueprint for these interactions. Confluent acts as the indispensable real-time backbone, transforming these protocols from theoretical designs into scalable, resilient, and observable real-world applications by providing a central nervous system for all data flow.

Expanding on MCP With Confluent: Tools and Data Integration

MCP defines how agents integrate with external resources, such as databases, APIs, and legacy systems. Confluent significantly enhances this integration by providing the real-time conduit for data exchange with the following.

• Tools and Data: Confluent acts as the central nervous system for data flow between agents and diverse external tools and data sources. Whether agents need to query a database, ingest data from a streaming API, or interact with a specialized analytical tool, Confluent ensures that data is moved efficiently and in real time.

• Large Language Models: As LLMs become integral to agentic systems, Confluent can facilitate the real-time feeding of contextual data to LLMs for enhanced reasoning and decision-making. Conversely, it can also distribute the outputs of LLMs to other agents or systems for further action.

• Knowledge Bases: Confluent can power the real-time updates and dissemination of information to and from an agent's knowledge base, ensuring that agents always operate with the most current understanding of their environment and tasks.

Enabling Inter-Agent Communication

The A2A protocol defines the communication mechanisms between agents, and Confluent plays a pivotal role in enabling and enhancing this interaction.

• Inter-Agent Communication With Confluent as Proxy: Confluent serves as a robust and reliable proxy for inter-agent communication. Agents can publish messages to specific Kafka topics, and other agents can subscribe to these topics, facilitating seamless and scalable information exchange without direct point-to-point connections.

• Workflow Orchestration: In complex multi-agent systems, Confluent can function as the central nervous system for workflow orchestration. By defining specific topics for different stages of a workflow and allowing agents to publish and subscribe to these topics, Confluent enables the coordination and sequencing of tasks across multiple agents, creating intricate yet manageable operational flows.

Agentic Observability: Traceable, Auditable, Replayable

Confluent's foundation in the Kafka protocol inherently provides powerful observability features that are crucial for complex agentic systems. Every message flowing through Kafka is traceable, auditable, and replayable. This is invaluable for debugging, understanding agent behavior, and ensuring compliance.

Confluent further enhances this by providing tools for lineage and governance, helping to track the origin and transformation of data, thereby eliminating potentially bad responses from complex agents or meeting stringent regulatory requirements in enterprise environments.

Use Case: Real-Time Financial Fraud Detection and Remediation

Catching fraud in real time is a massive challenge. Traditional systems often struggle, leading to delays and missed threats. What if we could build a system that's fast, accurate, and scalable using intelligent agents and robust messaging?

Let’s explore a streaming architecture that decouples real-time detection from in-depth investigation and remediation, powered by Google's Agent Development Kit (ADK), LangGraph, and Confluent. This architecture leverages Google Cloud's powerful AI agents and Confluent's real-time data processing tools to build a system that’s fast, accurate, and scalable.

• The System

  • Agent 1: The Transaction Monitor

    • Built with Google's ADK, this high-performance agent is the first line of defense. The Confluent client application (in Python or JavaScript) consumes massive, real-time transaction data from Kafka, with the MCP/A2A proxy handling the secure and efficient data exchange. A Flink SQL query quickly flags suspicious patterns. Crucially, it doesn't act itself; it simply identifies and alerts.

  • Agent 2: The Investigation Agent

    • When the Transaction Monitor flags an anomaly, it sends an A2A task message (e.g., {"request": "Investigate suspicious transaction for user X"}) to a fraud-alerts Kafka topic. The Investigation Agent, built with LangGraph for auditable, step-by-step reasoning, consumes this message. It then uses tools to access customer history from the customer relationship management (CRM) system and to check known fraud databases to confirm the fraud.

  • Agent 3: The Remediation Agent

    • After confirming fraud, the Investigation Agent sends another A2A task message (e.g., {"request": "Block account for user X immediately"}) to a remediation-tasks Kafka topic. The Remediation Agent consumes this message, and its sole job is to lock the user's account in the core banking system.

• The A2A and Confluent Workflow

  1. Suspicion Detected: The Transaction Monitor (Agent 1) constantly scans real-time transaction data using a Flink SQL query. When it spots a suspicious pattern, it doesn't act directly. Instead, it generates an A2A task message (e.g., {"task_id": "123", "request": "Investigate suspicious transaction for user X"}) and publishes it to the fraud-alerts Kafka topic. This is the critical first step in decoupling detection from action.

  2. Investigation Initiated: The Investigation Agent (Agent 2), subscribed to fraud-alerts, consumes this A2A message. Leveraging LangGraph for its auditable, step-by-step reasoning, it dives deep. It pulls customer history from a CRM and checks against known fraud databases to confirm if the activity is indeed fraudulent.

  3. Action Requested: If fraud is confirmed, the Investigation Agent again doesn't act directly. It issues a new A2A task message (e.g., {"task_id": "456", "request": "Block account for user X immediately"}) to the remediation-tasks Kafka topic.

  4. Remediation Executed: Finally, the Remediation Agent (Agent 3), listening to remediation-tasks, consumes this message. Its simple, focused job is to lock the user's account in the core banking system, completing the process.

The beauty of this system lies in its seamless A2A communication via Confluent: The Transaction Monitor detects and alerts; the Investigation Agent consumes, analyzes, and confirms; the Remediation Agent executes.

This decoupled, agent-based architecture offers significant benefits: scalability (each agent scales independently), resilience (Kafka ensures message durability), an auditable trail (every A2A message is a clear record), flexibility (agents can be updated independently), and clear responsibilities, simplifying development and maintenance. This approach is a powerful, efficient, and transparent way to tackle real-time fraud, making systems more robust and adaptable.

Start Serving Your AI Agents Real-Time, Context-Rich Data

This powerful synergy between Google's AI capabilities, particularly Vertex AI, and Confluent's real-time data streaming and processing—further enhanced by an A2A communication framework—isn't just theoretical. It's making AI integration immediately practical and profoundly impactful. By providing your AI agents with real-time, context-rich data, you can unlock new levels of responsiveness, personalization, and operational efficiency, all while building a future-proof, continuously intelligent ecosystem without the bottlenecks of traditional batch processing.

To learn more about how Confluent and Google Cloud are building real-time agentic systems, be sure to join our joint session at Current in New Orleans this month. Join the conversation at the Current 2025 session!

Ready to get started? Try Confluent Cloud in the Google Cloud Marketplace today—and get $1,000 in FREE credits when you sign up!

Apache®, Apache Kafka®, Kafka®, Apache Flink®, and Flink® are registered trademarks of the Apache Software Foundation. No endorsement by the Apache Software Foundation is implied by the use of these marks.