Browsing Category

Cloud

80 posts
My Data Streaming Journey with Kafka and Flink - 7 Years at Confluent
Read More
  • 645 views
  • 11 minute read

My Data Streaming Journey with Kafka & Flink: 7 Years at Confluent

Time flies… I joined Confluent seven years ago when Apache Kafka was mainly used by a few tech giants and the company had ~100 employees. This blog post explores my data streaming journey, including Kafka becoming a de facto standard for over 100,000 organizations, Confluent doing an IPO on the NASDAQ stock exchange, 5000+ customers adopting a data streaming platform, and emerging new design approaches and technologies like data mesh, GenAI, and Apache Flink. I look at the past, present and future of my personal data streaming journey. Both, from the evolution of technology trends and the journey as a Confluent employee that started in a Silicon Valley startup and is now part of a global software and cloud company.
Read More
0
0
0
Snowflake and Apache Kafka Data Integration Anti Patterns Zero Reverse ETL
Read More
  • 1.0K views
  • 9 minute read

Snowflake Integration Patterns: Zero ETL and Reverse ETL vs. Apache Kafka

Snowflake is a leading cloud-native data warehouse. Integration patterns include batch data integration, Zero ETL and near real-time data ingestion with Apache Kafka. This blog post explores the different approaches and discovers its trade-offs. Following industry recommendations, it is suggested to avoid anti-patterns like Reverse ETL and instead use data streaming to enhance the flexibility, scalability, and maintainability of enterprise architecture.
Read More
0
0
0
Google Apache Kafka for BigQuery GCP Cloud Service
Read More
  • 1.0K views
  • 7 minute read

When (Not) to Choose Google Apache Kafka for BigQuery?

Google announced its Apache Kafka for BigQuery cloud service at its conference Google Cloud Next 2024 in Las Vegas. Welcome to the data streaming club joining Amazon, Microsoft, IBM, Oracle, Confluent, and others. This blog post explores this new managed Kafka offering for GCP, reviews the current status of the data streaming landscape, and shares some criteria to evaluate when Kafka in general and Google Apache Kafka in particular should (not) be used.
Read More
0
0
0
Streaming Analytics SQL API with Apache Kafka Confluent ClickHouse Tinybird
Read More
  • 4.7K views
  • 8 minute read

Apache Kafka and Tinybird (ClickHouse) for Streaming Analytics HTTP APIs

Apache Kafka became the de facto standard for data streaming. However, the combination of an event-driven architecture with request-response APIs is crucial for most enterprise architectures. This blog post explores how Tinybird innovates with a REST/HTTP layer on top of the open source analytics database ClickHouse in the cloud. Integrating Kafka with Tinybird, the benefits of fully managed services like Confluent Cloud, and customer stories from Factorial and FanDuel show why Kafka and analytics databases complement each other for more innovation and faster time-to-market.
Read More
0
0
0
The Past Present and Future of Stream Processing
Read More
  • 16.1K views
  • 12 minute read

The Past, Present and Future of Stream Processing

Stream processing has existed for decades. The adoption grows with open source frameworks like Apache Kafka and Flink in combination with fully managed cloud services. This blog post explores the past, present and future of stream processing, including the relation of machine learning and GenAI, streaming databases, and the integration between data streaming and data lakes with Apache Iceberg.
Read More
0
0
0
Data Streaming with Apache Kafka and ARM CPU at the Edge and in the Cloud
Read More
  • 1.0K views
  • 9 minute read

ARM CPU for Cost-Effective Apache Kafka at the Edge and Cloud

ARM CPUs often outperform x86 CPUs in scenarios requiring high energy efficiency and lower power consumption. These characteristics make ARM preferred for edge and cloud environments. This blog post discusses the benefits of using Apache Kafka alongside ARM CPUs for real-time data processing in edge and hybrid cloud setups, highlighting energy-efficiency, cost-effectiveness, and versatility. A wide range of use cases are explored across industries, including manufacturing, retail, smart cities and telco.
Read More
0
0
0
The State of Data Streaming for Insurance in 2023
Read More
  • 1.8K views
  • 5 minute read

The State of Data Streaming for Insurance in 2023

This blog post explores the state of data streaming for the insurance industry in 2023. The evolution of claim processing, customer service, telematics, and new business models requires real-time end-to-end visibility, reliable and intuitive B2B and B2C communication, and integration with pioneering technologies like AI/machine learning for image recognition. Learn from use cases of Allianz, Generali, Policygenius, and more. A complete slide deck and on-demand video recording are included.
Read More
0
0
0
The State of Data Streaming for Energy and Utilities in 2023
Read More
  • 2.0K views
  • 8 minute read

The State of Data Streaming for Energy & Utilities in 2023

The evolution of utility infrastructure, energy distribution, customer services, and new business models requires real-time end-to-end visibility, reliable and intuitive B2B and B2C communication, and integration with pioneering technologies like 5G for low latency or augmented reality for innovation. I look at trends in the utilities sector to explore how data streaming helps as a business enabler, including customer stories from SunPower, 50hertz, Powerledger, and more. A complete slide deck and on-demand video recording are included.
Read More
0
0
0
The State of Data Streaming for Digital Natives in 2023
Read More
  • 1.8K views
  • 8 minute read

The State of Data Streaming for Digital Natives in 2023

This blog post explores the state of data streaming in 2023 for digital natives born in the cloud. Data streaming allows integrating and correlating data in real-time at any scale to improve the most innovative applications leveraging Apache Kafka. I explore how data streaming helps as a business enabler, including customer stories from New Relic, Wix, Expedia, Apna, Grab, and more. A complete slide deck and on-demand video recording are included.
Read More
0
0
0