Browsing Category

Data Integration

5 posts
Snowflake with Apache Kafka and Iceberg Connector
Read More
  • 739 views
  • 8 minute read

Snowflake Data Integration Options for Apache Kafka (including Iceberg)

The integration between Apache Kafka and Snowflake is often cumbersome. Options include near real-time ingestion with a Kafka Connect connector, batch ingestion from large files, or leveraging a standard table format like Apache Iceberg. This blog post explores the alternatives and discusses its trade-offs. The end shows how data streaming helps with hybrid architectures where data needs to be ingested from the private data center into Snowflake in the public cloud.
Read More
0
0
0
SAP Datasphere and Apache Kafka as Data Fabric for ERP Integration
Read More
  • 2.2K views
  • 12 minute read

SAP Datasphere and Apache Kafka as Data Fabric for S/4HANA ERP Integration

SAP is the leading ERP solution across industries around the world. Data integration with other data platforms, applications, databases, and APIs is one of the hardest challenges in the IT and software landscape. This blog post explores how SAP Datasphere in conjunction with the data streaming platform Apache Kafka enables a reliable, scalable and open data fabric for connecting SAP business objects of ECC and S/4HANA ERP with other real-time, batch, or request-response interfaces.
Read More
0
0
0
Business Process Automation and BPM Workflow Engine and Orchestration with Apache Kafka
Read More
  • 7.8K views
  • 15 minute read

Apache Kafka as Workflow and Orchestration Engine

Business process automation with a workflow engine or BPM suite has existed for decades. However, using the data streaming platform Apache Kafka as the backbone of a workflow engine provides better scalability, higher availability, and simplified architecture. This blog post explores case studies across industries to show how enterprises like Salesforce or Swisscom implement stateful workflow automation and orchestration with Kafka.
Read More
0
0
0
Apache Kafka for Data Consistency (and Real-Time Data Streaming)
Read More
  • 4.1K views
  • 6 minute read

Apache Kafka for Data Consistency (and Real-Time Data Streaming)

Real-time data beats slow data in almost all use cases. But as essential is data consistency across all systems, including non-real-time legacy systems and modern request-response APIs. Apache Kafka’s most underestimated feature is the storage component based on the append-only commit log. It enables loose coupling for domain-driven design with microservices and independent data products in a data mesh. This blog post explores how Kafka enables data consistency with a real-world case study from financial services.
Read More
10
0
0
Streaming ETL with Apache Kafka in Healthcare
Read More
  • 2.2K views
  • 4 minute read

Streaming ETL with Apache Kafka in the Healthcare Industry

IT modernization and innovative new technologies change the healthcare industry significantly. This blog series explores how data streaming with Apache Kafka enables real-time data processing and business process automation. This is part three: Streaming ETL. Examples include Babylon Health and Bayer.
Read More
0
0
0