Latest Insights on Data Streaming, AI, and Enterprise Architecture
Explore Kai Waehner’s latest articles on Apache Kafka, Apache Flink, real-time data streaming, data integration, process intelligence, agentic AI, and modern enterprise architecture. The blog covers practical use cases, technology trends, conference insights, and hands-on perspectives for teams building scalable, future-ready data platforms.
Why Enterprises Need a Multi-Model AI Strategy: Cost, Compliance, and Resilience
Single-vendor AI has become a single point of failure. Five drivers now make the case for a multi-model AI strategy across SaaS frontier models and self-hosted open weights, and the architecture underneath decides whether it works….
Trusted Agentic AI Landscape Q3 2026: Enterprise Vendor Selection, Sovereignty, and Lock-in
The Q3 2026 edition of the Trusted Agentic AI Landscape maps enterprise AI vendors on two axes, trust and lock-in, and explains why sovereignty, open weights, and agent-layer lock-in now decide the vendor choice. Free PDF with the full…
Data Integration vs Workflow Orchestration: Connecting Systems Is Not Coordinating the Work
Data integration and workflow orchestration get confused because both ship hundreds of connectors. This post draws the line: integration moves and reshapes data, orchestration coordinates what runs across categories and recovers when…
Process Intelligence Landscape 2026: Mining, Orchestration, and the Agentic AI Shift
Process intelligence has become three things, not one: mining, orchestration, and a decision gate. Here is how they fit, why agentic AI raises the stakes, and the four choices that keep your options open. Full vendor landscape as a free PDF….
When to Use AMQP, JMS, Kafka, or MQTT: Trade-offs, Not a Winner
AMQP, JMS, Kafka, and MQTT get compared as rivals, but a message broker, a log, and a device protocol sit on different layers. A clear guide to what each is, how they differ, when to use which, and why one platform often supports…
Kafka vs Flink vs Spark: Do You Really Need Real-Time?
Most vendors sell milliseconds, but most enterprise use cases do not need them. A critical look at Kafka, Flink, Spark, Pulsar, and Redpanda, and why the SLA and the estate decide the tool, not the benchmark….
Edge to Cloud and Back: Four Data Movement Problems, and Why One Technology Never Solves All of Them
Edge to cloud is not one integration problem. It is four: telemetry going up, control going down, sites syncing sideways, and data reaching people. This article maps the vendor categories to each pattern and gives you a five-question…
Data Integration Landscape 2026: Event Streaming, API, and Batch in the Era of Agentic AI
The Data Integration Landscape 2026 maps every major vendor across three communication paradigms: request-response, event-driven, and batch. Event streaming has become the architectural foundation for agentic AI. This independent…
Why I Joined Kestra: Enterprise Workflow Orchestration for the Agentic AI Era
Enterprises run separate tools for IT scheduling, data pipelines, business processes, and infrastructure. None talk to each other. Modernization and agentic AI are forcing them into one platform. Here is why I joined Kestra as Global…
My Confluent Chapter: From Apache Kafka Startup to $11 Billion IBM Acquisition
Nine years at Confluent: from a Silicon Valley startup with 100 people to an $11 billion IBM acquisition. A personal reflection on the Apache Kafka and data streaming journey and what the team accomplished together….
YAML vs XML vs JSON: History, Trade-offs, and Where Each Wins in the Age of Agentic AI
XML, JSON, and YAML were built for different jobs in different eras. This post covers where each came from, how they compare, and where each one still wins, including why YAML borrows JSON Schema and how schemas became the contract…
Why Databricks and Snowflake Speak the Kafka Protocol: Ingestion vs. Architecture
Databricks and Snowflake now speak the Kafka protocol. But using the Kafka API to feed a lakehouse is very different from running Kafka as the event-driven backbone of the enterprise. Here is why the two are complementary, not the same….
Choosing an ERP for Manufacturing: How AI Is Reshaping the Vendor Landscape
ERP vendor selection for manufacturing is not a product decision. It is a strategic bet on fit, total cost, and which AI future your vendor is building toward. Three German manufacturers chose SAP, ams.erp, and VlexPlus. All three were…
Process Intelligence Explained: Mining, Orchestration, and the Decision Gate
Process intelligence is not a single tool. It combines process mining, process orchestration, and a decision gate into one architecture that shows how processes really run, governs what happens next, and keeps automation and AI inside…
ERP Migration to SAP S/4HANA and Beyond: Lessons Learned from German Manufacturing
ERP modernization fails when the technology leads and the process work follows. Three German manufacturers ran their migrations differently, on SAP, ams.erp, and VlexPlus. The structural lessons are the same. This post covers what…
Beyond Enterprise Data Lineage: The Case for a Platform-Independent Data Catalog
Most organizations start their data governance journey by asking how to track where data comes from and where it goes. They quickly discover a harder question: why can none of their existing tools answer that across all systems?…
Data Ownership in the Age of Agentic AI: Why SAP’s API Policy Forces a Data Integration Reckoning for Every Enterprise
Every enterprise is being told to go agentic. Meanwhile, the platforms holding your most critical business data are tightening control over how AI agents can access it. SAP made that move explicitly. Other software vendors are doing…
Flink CEP and Agentic AI: Real-Time Pattern Detection as the Foundation for Autonomous Decisions
AI agents fail in production when they are connected directly to raw event streams. Flink CEP is the missing layer between your data streams and your Agentic AI architecture: it detects meaningful event sequences in real time, reduces…
Complex Event Processing (CEP) with Apache Flink: What It Is and When (Not) to Use It
Complex Event Processing is the most underused capability in Apache Flink. It detects meaningful event sequences in real time, fires only when a pattern is confirmed, and even catches events that never arrive. This guide covers what…
MCP vs. REST/HTTP API vs. Kafka: The Architect’s Guide to Agentic AI Integration
MCP, REST/HTTP APIs, and Apache Kafka are not alternatives. They solve different problems at different layers of the architecture. This article maps the decision: what each technology is built for, where the boundaries are, and where…
Enterprise Agentic AI Landscape Q2 2026: Trust, Flexibility, and Vendor Lock-in
The Enterprise Agentic AI Landscape 2026 maps every major AI vendor across two dimensions that matter most: how much you trust their AI, and how much lock-in you accept. An independent, vendor-neutral analysis covering Anthropic,…
The Trinity of Modern Data Architecture: Process Intelligence, Event-Driven Integration, and Trusted Agentic AI
Agentic AI without governed processes is fast but ungoverned. Event-driven integration without process intelligence moves data but not decisions. Process intelligence without live data automates the wrong outcomes. The fix is a…
dbt Meets Apache Flink: One Workflow for Data Engineers on Snowflake, BigQuery, Databricks, and Confluent
Two toolchains, two skill sets, two CI/CD pipelines — that has been the reality for data engineers working across batch and streaming. dbt extending to Apache Flink changes that equation. One workflow, one tool, one engineering team…
The Shift Left Architecture 2.0: Operational, Analytical and AI Interfaces for Real-Time Data Products
The Shift Left Architecture moves data integration logic into an event-driven architecture where governed data products are built once and served to multiple consumers. The original pattern covered two interfaces: operational via…
UFC VIP Experience Worth the Price? Fan Review. Business Perspective. Tech Vision.
The Ultimate Fighting Championship (UFC) held Fight Night London on March 21, 2026, at The O2 Arena. It was a great event inside the octagon. But a 1,200 USD VIP ticket raised real questions about premium customer experience, and about…
Dashboards and Queries for Apache Kafka: Operational, Explorative, and the Role of the Context Engine
Dashboards are a popular way to make streaming data visible and useful, but they are not always the right solution. This blog post explains when dashboards make sense for Apache Kafka data and when other approaches like automation,…
Data Streaming at MWC 2026: How Apache Kafka, Flink and Agentic AI Power Telecom Trends
Mobile World Congress (MWC) 2026 highlights the shift from batch systems to real time data streaming in telecom. AI and agentic automation, network APIs, sovereign cloud, autonomous networks, and 5G monetization all depend on…
From Takeoff to Touchdown: Real-Time Aviation with Data Streaming at Qantas
This blog post explores how data streaming transforms airline operations by enabling real-time visibility, faster decision-making, and improved customer experience. Using Qantas as a leading example, it highlights how a modern data…