Agentic AI is changing how enterprises think about automation and intelligence. Agents are no longer reactive systems. They are goal-driven, context-aware, and capable of autonomous decision-making. But to operate effectively, agents must be connected to the real-time pulse of the business. This is where data streaming with Apache Kafka and Apache Flink becomes essential.
Apache Flink is entering a new phase with the proposal of Flink Agents, a sub-project designed to power system-triggered, event-driven AI agents natively within Flink’s streaming runtime. Let’s explore what this means for the future of agentic systems in the enterprise.
Join the data streaming community and stay informed about new blog posts by subscribing to my newsletter and follow me on LinkedIn or X (former Twitter) to stay in touch. And make sure to download my free book about data streaming use cases, including various real-world examples AI-related topics like fraud detection and generative AI for customer service.
Agentic AI is no longer experimental. It is still in early stage of the adoption lifecycle, but starting to move into production for first critical use cases.
Agents today are expected to:
But these goals face real infrastructure challenges. Existing frameworks like LangChain or LlamaIndex are great for prototyping. But (without the help of other tools) they are not designed for long-running, system-triggered workflows that need high availability, fault tolerance, and deep integration with enterprise data systems.
The real problem is integration. Agents must operate on live data, interact with tools and models, and work across systems. This complexity demands a new kind of runtime. One that is real-time, event-driven, and deeply contextual.
Standards are emerging to build scalable, interoperable AI agents. Two of the most important are:
These standards help define what agents do and how they do it. But protocols alone are not enough. Enterprises need a runtime to execute these agent workflows in production—with consistency, scale, and reliability. This is where Flink fits in.
Similar to microservices a decade ago, Agentic AI and protocols like MCP and A2A risk creating tight coupling and point-to-point spaghetti architectures if used in isolation. An event-driven data streaming backbone ensures these standards deliver scalable, resilient, and governed agent ecosystems instead of repeating past mistakes.
Apache Kafka and Flink together form the event-driven backbone for Agentic AI.
Agentic AI requires real-time data ingestion to ensure agents can react instantly to changes as they happen. It also depends on stateful processing to maintain memory across interactions and decision points. Coordination between agents is essential so that tasks can be delegated, results can be shared, and workflows can be composed dynamically. Finally, seamless integration with tools, models, and APIs allows agents to gather context, take action, and extend their capabilities within complex enterprise environments.
Apache Flink provides all of these natively. Instead of stitching together multiple tools, Flink can host the entire agentic workflow:
A2A and MCP define how autonomous agents communicate and access context. With Kafka as the event broker and Flink as the stream processor, enterprises can build scalable, decoupled, and context-aware Agentic AI systems.
Agents can still communicate point-to-point with each other via protocols like A2A. But their information must often reach many systems, and multiple point-to-point links create brittle architectures. Kafka solves this by acting as the scalable, decoupled event backbone.
More details: Agentic AI with the Agent2Agent Protocol (A2A) and MCP using Apache Kafka as Event Broker.
While this is already possible to implement today, Flink still needs more built-in support for standard protocols like MCP and A2A, along with native AI and ML capabilities, to fully meet the demands of enterprise-grade agentic systems.
FLIP-531: Initiate Flink Agents as a new Sub-Project is an exciting “Flink Improvement Proposal” led by Xintong Song, Sean Falconer, and Chris Meyers. It introduces a native framework for building and running AI agents within Flink.
With FLIP-531, Apache Flink goes beyond orchestration and data preparation in Agentic AI environments. It now provides a native runtime to build, run, and manage autonomous AI agents at scale.
Flink Agents will extend familiar Flink constructs. Developers can define agents using endpoints coming soon to Flink’s Table API or DataStream API. They can connect to endpoints of large language models (LLM), register models, call tools, and manage context—all from within Flink.
Sample APIs are already available for Java, Python (PyFlink), and SQL. These include support for:
The Flink Agents project is moving fast with a clear roadmap focused on rapid delivery and community-driven development:
The team is prioritizing execution and fast iteration, with a GitHub-based development model and lightweight governance to accelerate innovation.
The most impactful agents in the enterprise aren’t chatbots or assistants waiting for user input. They are always-on components, embedded in infrastructure, continuously observing and acting on real-time business events.
Apache Flink Agents are built for this model. Instead of waiting for a request, these agents run asynchronously as part of an event-driven architecture. They monitor streams of data, maintain memory, and trigger actions automatically—similar to observability agents, but for decision-making and automation.
This always-on approach is critical for modern use cases. Enterprises can’t afford delays in fraud detection, equipment failures, customer engagement, or supply chain response. Agents must act instantly—based on the data flowing through the system—not hours later via batch processing or human intervention.
Apache Flink provides the ideal foundation. Its low-latency, stateful stream processing enables agents to:
These aren’t chatbot wrappers—they’re autonomous services embedded in production. They function as part of your business nervous system, adapting in real time to changing conditions and continuously improving outcomes.
This architecture is what enterprises truly need: automation that is fast, reliable, and context-aware. It reduces time to detect and resolve issues, improves SLA adherence, and enables proactive decisions across the organization.
Always-on, embedded agents are the future of AI in business. Apache Flink is ready to power them. Let’s explore a few excellent use case examples across different industries in the next section.
Agentic AI use cases are emerging across industries. These systems demand real-time responsiveness, contextual intelligence, and full autonomy. Traditional REST APIs, manual orchestration, and batch jobs fall short in these environments. Instead, enterprises need infrastructure that is continuous, stateful, event-driven, and always-on (as discussed explicitly in the above section).
Apache Kafka and Apache Flink already power the data backbone of many digital-native organizations. For example:
With Flink Agents, developers can build:
Here are some examples where Flink Agents will shine:
The rise of Agentic AI means a shift in infrastructure priorities.
It’s not enough to invest in model quality or prompt engineering. Enterprises must also modernize their data and execution layer.
Point-to-point communication between agents is fine for direct interaction, but at scale the real value comes from an event-driven backbone like Kafka and Flink that ensures information reliably reaches all required systems.
Flink Agents offers a production-grade, enterprise-ready foundation for agentic systems. It turns brittle demos into reliable applications by providing:
This combination reduces time-to-market, increases system resilience, and lowers operational costs. It gives developers and architects the tools to build agents like real software. Scalable, testable, and observable.
This shift is NOT a replacement for data lakes, lakehouses, or AI platforms. It complements them by enabling real-time, event-driven execution alongside batch and analytical workloads.
To explore how streaming and AI platforms work together in practice, check out my Confluent + Databricks blog series, which highlights the value of combining Kafka, Flink, and lakehouse architectures for modern AI use cases.
The future of Agentic AI is event-driven. Apache Flink is ready to power it.
Join the data streaming community and stay informed about new blog posts by subscribing to my newsletter and follow me on LinkedIn or X (former Twitter) to stay in touch. And make sure to download my free book about data streaming use cases.
Apache Kafka has evolved from a data lake pipeline into the backbone of real-time transactional…
Apache Kafka is the backbone of real-time data streaming. Choosing the right deployment model -…
The automotive industry is rapidly shifting toward a software-defined, data-driven future. Real-time technologies like Apache…
Pinterest uses Apache Kafka and Flink to power Guardian, its real-time detection platform for spam,…
Agentic AI goes beyond chatbots. These are autonomous systems that observe, reason, and act—continuously and…
Global supply chains face constant disruption. Trade conflicts, wars, inflation, and shifting regulations are making…