Browsing Category

Apache Kafka

253 posts
Data Streaming Meets Data Lake and Lakehouse with Apache Iceberg and Delta Lake
Read More
  • 196 views
  • 5 minute read

Data Streaming Meets Lakehouse: Apache Iceberg for Unified Real-Time and Batch Analytics

Apache Iceberg is gaining momentum as the open table format of choice for modern data architectures. In this blog post, the key takeaways from my talk at Open Source Data Summit are shared, along with the full video and downloadable slides. The session explores how Iceberg fits into real-time data streaming with Apache Kafka and Flink, why streaming into a data lake is complex, and what patterns actually work in production. It also covers technical challenges like schema evolution, compaction, and governance — and how to solve them. Watch the session, review the slides, and learn how Iceberg helps build reliable, streaming-powered data products at scale.
Read More
0
0
0
Social Commerce in Retail with Data Streaming using Apache Kafka Flink and Agentic AI
Read More
  • 408 views
  • 9 minute read

Data Streaming in Retail: Social Commerce from Influencers to Inventory

Social commerce is reshaping retail by merging entertainment, influencer marketing, and instant purchasing into one real-time experience. Platforms like TikTok and Instagram have become active digital storefronts where discovery and transactions happen at once. This article explains how data streaming with Apache Kafka and Flink enables retailers to power social commerce through continuous data flow, real-time inventory updates, and personalized engagement. It shows how streaming unifies marketing, operations, and AI-driven decision-making while helping retailers compete with new AI platforms such as OpenAI that are redefining digital shopping.
Read More
0
0
0
Apache Kafka Proxy Use Cases Benefits Trade-Offs - Open Source vs Cloud
Read More
  • 1.5K views
  • 14 minute read

Kafka Proxy Demystified: Use Cases, Benefits, and Trade-offs

A Kafka proxy adds centralized security and governance for Apache Kafka. Solutions like Kroxylicious, Conduktor, and Confluent enable encryption, access control, and compliance without modifying clients or brokers. This article explores key use cases, best practices, and alternatives such as API gateways and service meshes.
Read More
0
0
0
Stablecoins and Data Streaming with Kafka and Flink for Digital Money and Currency USDC USDT Blockchain
Read More
  • 2.5K views
  • 10 minute read

How Stablecoins Use Blockchain and Data Streaming to Power Digital Money

Stablecoins are reshaping digital money by linking traditional finance with blockchain technology. Built for stability and speed, they enable real time payments, settlements, and programmable financial services. To operate reliably at scale, stablecoin systems require continuous data movement and analysis across ledgers, compliance tools, and banking platforms. A Data Streaming Platform using technologies like Apache Kafka and Apache Flink can provide this foundation by ensuring transactions, reserves, and risk signals are processed instantly and consistently. Together, blockchain, data streaming, and AI pave the way for a new financial infrastructure that runs on live, contextual data.
Read More
0
0
0
Cybersecurity with a Real-Time Digital Twin using Data Streaming with Apache Kafka Flink and Sigma
Read More
  • 1.5K views
  • 10 minute read

Cybersecurity with a Digital Twin: Why Real-Time Data Streaming Matters

Cyberattacks on critical infrastructure and manufacturing are growing, with ransomware and manipulated sensor data creating severe risks. Digital twins combined with data streaming provide real-time visibility, continuous monitoring, and proactive defense across both IT and OT environments. Using technologies like Kafka, Flink and Sigma, organizations can detect anomalies instantly, strengthen resilience, and secure digital transformation.
Read More
0
0
0
The Future of AI-Ready Integration with Data Streaming powered by Apache Kafka and Flink
Read More
  • 1.8K views
  • 4 minute read

How Siemens, SAP, and Confluent Shape the Future of AI Ready Integration – Highlights from the Rojo Event in Amsterdam

Many enterprises want to become AI ready but are limited by slow, batch based integration platforms that prevent real time insight and automation. The Rojo “Future of Integration” event in Amsterdam addressed this challenge by bringing together Siemens, SAP, Rojo, and Confluent to show how event driven and intelligent data architectures solve it. The discussions revealed how data streaming with Apache Kafka and Flink complements traditional integration tools, enabling continuous data flow, scalability, and the foundation for AI and automation. This blog summarizes the key learnings from the event, including my presentation “AI Ready Integration with Data Streaming,” and insights from Siemens, SAP, and Rojo on how enterprises can build truly connected, AI ready ecosystems.
Read More
0
0
0
Scaling Apache Kafka Consumers for High Throughput with Proxy or Client Library for API and Database Integration
Read More
  • 2.1K views
  • 8 minute read

Scaling Kafka Consumers: Proxy vs. Client Library for High-Throughput Architectures

Apache Kafka’s pull-based model and decoupled architecture offer unmatched flexibility for event-driven systems. But as data volumes and consumer applications grow, new challenges emerge; from head-of-line blocking and rising operational overhead to complex failure handling. This post explores real-world lessons from companies like Wix and Uber, highlighting common consumer scalability issues and two main solutions: push-based consumer proxies and enhanced client libraries like Confluent’s Parallel Consumer. It concludes with a vision for a serverless Kafka consumption model that reduces total cost of ownership while preserving Kafka’s core strengths.
Read More
0
0
0
Real Time Point of Sale POS in Retail and eCommerce with Data Streaming using Kafka and Flink at SumUp Square Shopify
Read More
  • 2.6K views
  • 6 minute read

Square, SumUp, Shopify: Real-Time Point-of-Sale (POS) in the Age of Data Streaming

Point-of-Sale systems are evolving into real-time, connected platforms that go far beyond payments. Mobile solutions from Square, SumUp, and Shopify give even the smallest merchants access to integrated sales channels, inventory management, and customer insights. Powered by Apache Kafka and Apache Flink, data streaming is transforming retail with instant decisions, automated actions, and the foundation for Agentic AI in the future of POS.
Read More
0
0
0
Online Feature Store for AI ML with Data Streaming using Apache Kafka Flink FlinkSQL Confluent Cloud at Wix
Read More
  • 3.5K views
  • 6 minute read

Online Feature Store for AI and Machine Learning with Apache Kafka and Flink

Real-time personalization requires more than just smart models. It demands fresh data, fast processing, and scalable infrastructure. This blog post explores how Wix.com rebuilt its online feature store using Apache Kafka and Flink, turning their AI architecture into a real-time powerhouse that supports personalized experiences for millions of users.
Read More
0
0
0
Amazon MSK Forces a Kafka Cluster Migration from ZooKeeper to KRaft
Read More
  • 5.6K views
  • 4 minute read

In-Place Kafka Cluster Upgrades from ZooKeeper to KRaft are Not Possible with Amazon MSK

The Apache Kafka community introduced KIP-500 to remove ZooKeeper and replace it with KRaft, a new built-in consensus layer. This was a major milestone. It simplified operations, improved scalability, and reduced complexity. Importantly, Kafka supports smooth, zero downtime migrations from ZooKeeper to KRaft, even for large, business critical clusters. But NOT with Amazon MSK…
Read More
0
0
0