Browsing Category

Data Streaming

96 posts
Data Streaming with Apache Kafka and Flink in the Cloud in Healthcare and Pharma at Cardinal Health
Read More
  • 6.1K views
  • 7 minute read

Data Streaming in Healthcare and Pharma: Use Cases and Insights from Cardinal Health

This blog explores Cardinal Health’s journey, exploring how its event-driven architecture and data streaming power use cases like supply chain optimization, and medical device and equipment management. By integrating Apache Kafka with platforms like Apigee, Dell Boomi and SAP, Cardinal Health sets a benchmark for IT modernization and innovation in the healthcare and pharma sectors.
Read More
0
0
0
Dynamic Pricing with Data Streaming using Apache Kafka and Flink
Read More
  • 8.6K views
  • 8 minute read

A New Era in Dynamic Pricing: Real-Time Data Streaming with Apache Kafka and Flink

In the age of digitization, the concept of pricing is no longer fixed or manual. Instead, companies increasingly use dynamic pricing — a flexible model that adjusts prices based on real-time market changes to enable real-time responsiveness, giving companies the tools they need to respond instantly to demand, competitor prices, and customer behaviors. This blog post explores the fundamentals of dynamic pricing, its link to data streaming, and real-world examples across different industries such as retail, logistics, gaming and the energy sector.
Read More
0
0
0
Fraud Prevention with Apache Kafka in Real Time in Financial Services and Banking
Read More
  • 4.7K views
  • 4 minute read

Fraud Prevention in Under 60 Seconds with Apache Kafka: How A Bank in Thailand is Leading the Charge

In financial services, the ability to prevent fraud in real-time is not just a competitive advantage – it is a necessity. For one of the largest banks in Thailand Krungsri (Bank of Ayudhya), with its vast assets, loans, and deposits, the challenge of fraud prevention has taken center stage. This blog post explores how the bank is leveraging data streaming with Apache Kafka to detect and block fraudulent transactions in under 60 seconds to ensure the safety and trust of its customers.
Read More
0
0
0
Serverless Data Streaming on Azure Cloud with Apache Kafka Event Hubs Confluent Cloud for OneLake and Microsoft Fabric
Read More
  • 21.1K views
  • 9 minute read

When to Choose Apache Kafka vs. Azure Event Hubs vs. Confluent Cloud for a Microsoft Fabric Lakehouse

Choosing between Apache Kafka, Azure Event Hubs, and Confluent Cloud for data streaming is critical when building a Microsoft Fabric Lakehouse. Each option caters to different needs, and this blog post will guide you in selecting the right data streaming solution for your use case.
Read More
0
0
0
Apache Iceberg Open Table Format for Data Lake Lakehouse Streaming wtih Kafka Flink Databricks Snowflake AWS GCP Azure
Read More
  • 27.9K views
  • 11 minute read

Apache Iceberg – The Open Table Format for Lakehouse AND Data Streaming

An open table format framework like Apache Iceberg is essential in the enterprise architecture to ensure reliable data management and sharing, seamless schema evolution, efficient handling of large-scale datasets and cost-efficient storage. This blog post explores market trends, adoption of table format frameworks like Iceberg, Hudi, Paimon, Delta Lake and XTable, and the product strategy of leading vendors of data platforms such as Snowflake, Databricks (Apache Spark), Confluent (Apache Kafka / Flink), Amazon Athena and Google BigQuery.
Read More
0
0
0
The Shift Left Architecture
Read More
  • 26.7K views
  • 8 minute read

The Shift Left Architecture – From Batch and Lakehouse to Real-Time Data Products with Data Streaming

Data integration is a hard challenge in every enterprise. Batch processing and Reverse ETL are common practices in a data warehouse, data lake or lakehouse. Data inconsistency, high compute cost, and stale information are the consequences. This blog post introduces a new design pattern to solve these problems: The Shift Left Architecture enables a data mesh with real-time data products to unify transactional and analytical workloads with Apache Kafka, Flink and Iceberg. Consistent information is handled with streaming processing or ingested into Snowflake, Databricks, Google BigQuery, or any other analytics / AI platform to increase flexibility, reduce cost and enable a data-driven company culture with faster time-to-market building innovative software applications.
Read More
0
0
0
Data Streaming with Apache Kafka for Industrial IoT in the Automotive Industry at Brose
Read More
  • 3.3K views
  • 3 minute read

Apache Kafka in Manufacturing at Automotive Supplier Brose for Industrial IoT Use Cases

Data streaming unifies OT/IT workloads by connecting information from sensors, PLCs, robotics and other manufacturing systems at the edge with business applications and the big data analytics world in the cloud. This blog post explores how the global automotive supplier Brose deploys a hybrid industrial IoT architecture using Apache Kafka in combination with Eclipse Kura, OPC-UA, MuleSoft and SAP.
Read More
0
0
0
Snowflake with Apache Kafka and Iceberg Connector
Read More
  • 11.2K views
  • 8 minute read

Snowflake Data Integration Options for Apache Kafka (including Iceberg)

The integration between Apache Kafka and Snowflake is often cumbersome. Options include near real-time ingestion with a Kafka Connect connector, batch ingestion from large files, or leveraging a standard table format like Apache Iceberg. This blog post explores the alternatives and discusses its trade-offs. The end shows how data streaming helps with hybrid architectures where data needs to be ingested from the private data center into Snowflake in the public cloud.
Read More
0
0
0
Snowflake and Apache Kafka Data Integration Anti Patterns Zero Reverse ETL
Read More
  • 9.9K views
  • 9 minute read

Snowflake Integration Patterns: Zero ETL and Reverse ETL vs. Apache Kafka

Snowflake is a leading cloud-native data warehouse. Integration patterns include batch data integration, Zero ETL and near real-time data ingestion with Apache Kafka. This blog post explores the different approaches and discovers its trade-offs. Following industry recommendations, it is suggested to avoid anti-patterns like Reverse ETL and instead use data streaming to enhance the flexibility, scalability, and maintainability of enterprise architecture.
Read More
0
0
0
Google Apache Kafka for BigQuery GCP Cloud Service
Read More
  • 16.1K views
  • 8 minute read

When (Not) to Choose Google Managed Service for Apache Kafka?

Google announced its Apache Kafka for BigQuery cloud service at its conference Google Cloud Next 2024 in Las Vegas. Welcome to the data streaming club joining Amazon, Microsoft, IBM, Oracle, Confluent, and others. This blog post explores this new managed Kafka offering for GCP, reviews the current status of the data streaming landscape, and shares some criteria to evaluate when Kafka in general and Google Apache Kafka in particular should (not) be used.
Read More
0
0
0