Browsing Tag

Lambda

3 posts
The Shift Left Architecture
Read More
  • 7.0K views
  • 8 minute read

The Shift Left Architecture – From Batch and Lakehouse to Real-Time Data Products with Data Streaming

Data integration is a hard challenge in every enterprise. Batch processing and Reverse ETL are common practices in a data warehouse, data lake or lakehouse. Data inconsistency, high compute cost, and stale information are the consequences. This blog post introduces a new design pattern to solve these problems: The Shift Left Architecture enables a data mesh with real-time data products to unify transactional and analytical workloads with Apache Kafka, Flink and Iceberg. Consistent information is handled with streaming processing or ingested into Snowflake, Databricks, Google BigQuery, or any other analytics / AI platform to increase flexibility, reduce cost and enable a data-driven company culture with faster time-to-market building innovative software applications.
Read More
0
0
0
Top Apache Kafka Use Cases and Architectures for 2022
Read More
  • 2.7K views
  • 2 minute read

Top 5 Apache Kafka Use Cases for 2022

Apache Kafka and Event Streaming are two of the most relevant buzzwords in tech these days. Ever wonder what my predicted TOP 5 Event Streaming Architectures and Use Cases for 2022 are to set data in motion? Check out the following presentation. Learn about the Kappa architecture, hyper-personalized omnichannel, multi-cloud deployments, edge analytics, and real-time cybersecurity.
Read More
0
0
0
Kappa Architecture vs Lambda Architecture for Apache Kafka Pulsar Data Lakes
Read More
  • 24.7K views
  • 17 minute read

Kappa Architecture is Mainstream Replacing Lambda

Real-time data beats slow data. That’s true for almost every use case. Nevertheless, enterprise architects build new infrastructures with the Lambda architecture that includes separate batch and real-time layers. This blog post explores why a single real-time pipeline, called Kappa architecture, is the better fit. Real-world examples from companies such as Disney, Shopify, Uber, and Twitter explore the benefits of Kappa but also show how batch processing fits into this discussion positively without the need for Lambda.
Read More
1
0
3