Kappa Architecture Archives

Reverse ETL Anti Pattern vs Event Streaming with Apache Kafka

753 views
9 minute read

When to Use Reverse ETL and when it is an Anti-Pattern

ByKai Waehner
30. September 2021

This blog post explores why software vendors (try to) introduce new solutions for Reverse ETL, when Reverse ETL is really needed, and how it fits into the enterprise architecture. The involvement of event streaming to process data in motion is a key piece of Reverse ETL for real-time use cases.

Kappa Architecture vs Lambda Architecture for Apache Kafka Pulsar Data Lakes

1.2K views
17 minute read

Kappa Architecture is Mainstream Replacing Lambda

ByKai Waehner
23. September 2021

Real-time data beats slow data. That’s true for almost every use case. Nevertheless, enterprise architects build new infrastructures with the Lambda architecture that includes separate batch and real-time layers. This blog post explores why a single real-time pipeline, called Kappa architecture, is the better fit. Real-world examples from companies such as Disney, Shopify, Uber, and Twitter explore the benefits of Kappa but also show how batch processing fits into this discussion positively without the need for Lambda.