Browsing Tag

open source

97 posts
Cloud Native SCADA Industrial IoT with Apache Kafka Data Streaming
Read More
  • 8 views
  • 6 minute read

A cloud-native SCADA System for Industrial IoT built with Apache Kafka

Industrial IoT and Industry 4.0 enable digitalization and innovation. SCADA control systems are a vital component of IT/OT modernization. The SCADA evolution started with monolithic applications and moved to networked and web-based platforms. This blog post explores building the 5th generation: A cloud-native SCADA infrastructure with Apache Kafka. A real-world case study explores the journey of a German system operator for electricity to show how such a journey to open and scalable real-time workloads and edge-to-cloud integration progressed.
Read More
0
0
0
Real Time Logistics Transportation Shipping with Apache Kafka
Read More
  • 43 views
  • 7 minute read

Real-Time Logistics, Shipping, and Transportation with Apache Kafka

Logistics, shipping, and transportation require real-time information to build efficient applications and innovative business models. Data streaming enables correlated decisions, recommendations, and alerts. Kafka is everywhere across the industry. This blog post explores several real-world case studies from companies such as Swiss Post, Austrian Post, DHL, and Hermes. Use cases include cloud-native middleware modernization, track and trace, and predictive routing and ETA planning.
Read More
0
0
0
Real-Time Supply Chain Control Tower with Apache Kafka
Read More
  • 59 views
  • 6 minute read

A Real-Time Supply Chain Control Tower powered by Kafka

A modern supply chain requires just-in-time production, global logistics, and complex manufacturing processes. This blog post explores a solution that ingests all information flows into a unified central nervous system. The idea of the Supply Chain Control Tower becomes a reality: An integrated data cockpit with real-time access to all levels and systems of the supply chain.
Read More
0
0
0
Real-Time Sports and Gaming with Data Streaming powered by Apache Kafka
Read More
  • 52 views
  • 6 minute read

Reimagine sports and gaming with data streaming: A table tennis success story built with Apache Kafka

The sports world is changing. Digitalization is everywhere. Cameras and sensors analyze matches. Stadiums get connected and incorporate mobile apps and location-based services. Players use social networks to influence and market themselves and consumer products. Real-time data processing is crucial for most innovative sports use cases. This blog post explores how data streaming with Apache Kafka helps reimagine the sports industry, showing a concrete example from the worldwide table tennis organization.
Read More
0
0
0
Is Amazon MSK Serverless for Apache Kafka a Self-Driving Car or just a Car Engine
Read More
  • 85 views
  • 13 minute read

When NOT to choose Amazon MSK Serverless for Apache Kafka?

Apache Kafka became the de facto standard for data streaming. Various cloud offerings emerged and improved in the last years. Amazon MSK Serverless is the latest Kafka product from AWS. This blog post looks at its capabilities to explore how it relates to “the normal” partially managed Amazon MSK, when the serverless version is a good choice, and when other fully-managed cloud services like Confluent Cloud are the better option.
Read More
0
0
0
The Heart of the Data Mesh Beats Real Time with Apache Kafka
Read More
  • 238 views
  • 5 minute read

The Heart of the Data Mesh Beats Real-Time with Apache Kafka

If there were a buzzword of the hour, it would undoubtedly be “data mesh”! This new architectural paradigm unlocks analytic and transactional data at scale and enables rapid access to an ever-growing number of distributed domain datasets for various usage scenarios. The data mesh addresses the most common weaknesses of the traditional centralized data lake or data platform architecture. And the heart of a decentralized data mesh infrastructure must be real-time, reliable, and scalable. Learn how the de facto standard for data streaming, Apache Kafka, plays a crucial role in building a data mesh.
Read More
1
0
0
Best Practices for Data Analytics with AWS Azure Googel BigQuery Spark Kafka Confluent Databricks
Read More
  • 128 views
  • 10 minute read

Best Practices for Building a Cloud-Native Data Warehouse or Data Lake

The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems. Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Let’s explore this dilemma in a blog series. This is part 5: Best Practices for Building a Cloud-Native Data Warehouse or Data Lake.
Read More
0
0
0
Case Studies for Cloud Native Analytics with Data Warehouse Data Lake Data Streaming Lakehouse
Read More
  • 119 views
  • 7 minute read

Case Studies: Cloud-native Data Streaming for Data Warehouse Modernization

The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems. Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Let’s explore this dilemma in a blog series. This is part 4: Case Studies for cloud-native data streaming and data warehouses.
Read More
0
0
0
Data Warehouse vs Data Lake vs Data Streaming Comparison
Read More
  • 414 views
  • 9 minute read

Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?

The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems. Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Let’s explore this dilemma in a blog series. This is part 1: Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
Read More
0
0
0
Request Response Data Exchange with Apache Kafka vs CQRS and Event Sourcing
Read More
  • 316 views
  • 10 minute read

When to use Request-Response with Apache Kafka?

How can I do request-response communication with Apache Kafka? That’s one of the most common questions I get regularly. This blog post explores when (not) to use this message exchange pattern, the differences between synchronous and asynchronous communication, the pros and cons compared to CQRS and event sourcing, and how to implement request-response within the data streaming infrastructure.
Read More
0
0
0