Browsing Category

Confluent

35 posts
My Data Streaming Journey with Kafka and Flink - 7 Years at Confluent
Read More
  • 610 views
  • 11 minute read

My Data Streaming Journey with Kafka & Flink: 7 Years at Confluent

Time flies… I joined Confluent seven years ago when Apache Kafka was mainly used by a few tech giants and the company had ~100 employees. This blog post explores my data streaming journey, including Kafka becoming a de facto standard for over 100,000 organizations, Confluent doing an IPO on the NASDAQ stock exchange, 5000+ customers adopting a data streaming platform, and emerging new design approaches and technologies like data mesh, GenAI, and Apache Flink. I look at the past, present and future of my personal data streaming journey. Both, from the evolution of technology trends and the journey as a Confluent employee that started in a Silicon Valley startup and is now part of a global software and cloud company.
Read More
0
0
0
When NOT to use Apache Kafka
Read More
  • 794 views
  • 3 minute read

When NOT to Use Apache Kafka? (Lightboard Video)

Apache Kafka is the de facto standard for data streaming to process data in motion. With its significant adoption growth across all industries, I get a very valid question every week: When NOT to use Apache Kafka? What limitations does the event streaming platform have? When does Kafka simply not provide the needed capabilities? How to qualify Kafka out as it is not the right tool for the job? This blog post contains a lightboard video that gives you a twenty-minute explanation of the DOs and DONTs.
Read More
0
0
0
Data Streaming Landscape 2023 with Apache Kafka Flink and much more
Read More
  • 5.1K views
  • 13 minute read

The Data Streaming Landscape 2023

Data streaming is a new software category to process data in motion. Apache Kafka is the de facto standard used by over 100,000 organizations. Plenty of vendors offer Kafka platforms and cloud services. Many complementary stream processing engines like Apache Flink and SaaS offerings have emerged. And competitive technologies like Pulsar and Redpanda try to get market share. This blog post explores the data streaming landscape of 2023 to summarize existing solutions and market trends.
Read More
0
0
0
Best Practices for Data Analytics with AWS Azure Googel BigQuery Spark Kafka Confluent Databricks
Read More
  • 3.1K views
  • 10 minute read

Best Practices for Building a Cloud-Native Data Warehouse or Data Lake

The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems. Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Let’s explore this dilemma in a blog series. This is part 5: Best Practices for Building a Cloud-Native Data Warehouse or Data Lake.
Read More
0
0
0
Data Warehouse vs Data Lake vs Data Streaming Comparison
Read More
  • 6.2K views
  • 9 minute read

Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?

The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems. Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Let’s explore this dilemma in a blog series. This is part 1: Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
Read More
0
0
0
Reverse ETL Anti Pattern vs Event Streaming with Apache Kafka
Read More
  • 4.0K views
  • 9 minute read

When to Use Reverse ETL and when it is an Anti-Pattern

This blog post explores why software vendors (try to) introduce new solutions for Reverse ETL, when Reverse ETL is really needed, and how it fits into the enterprise architecture. The involvement of event streaming to process data in motion is a key piece of Reverse ETL for real-time use cases.
Read More
1
0
0
SIEM and SOAR Modernization with Apache Kafka Elasticsearch Splunk QRadar Arcsight Cortex
Read More
  • 3.2K views
  • 10 minute read

Kafka for Cybersecurity (Part 6 of 6) – SIEM / SOAR Modernization

This blog series explores use cases and architectures for Apache Kafka in the cybersecurity space, including situational awareness, threat intelligence, forensics, air-gapped and zero trust environments, and SIEM / SOAR modernization. This post is part six: SIEM / SOAR modernization and integration.
Read More
0
0
0
De Facto Standard API - Amazon S3 for Object Storage and Apache Kafka for Event Streaming
Read More
  • 5.4K views
  • 13 minute read

Kafka API is the De Facto Standard API for Event Streaming like Amazon S3 for Object Storage

Real-time beats slow data in most use cases across industries. The rise of event-driven architectures and data in motion powered by Apache Kafka enables enterprises to build real-time infrastructure and applications. This blog post explores why the Kafka API became the de facto standard API for event streaming like Amazon S3 for object storage, and the tradeoffs of these standards and corresponding frameworks, products, and cloud services.
Read More
0
0
0
How to choose the right Apache Kafka Offering - Confluent Cloudera Red Hat IBM Amazon AWS MSK
Read More
  • 11.0K views
  • 21 minute read

Comparison of Open Source Apache Kafka vs Vendors including Confluent, Cloudera, Red Hat, Amazon MSK

Apache Kafka became the de facto standard for event streaming. Various vendors added Kafka and related tooling to their offerings or provide a Kafka cloud service. This blog post uses the car analogy – from the motor engine to the self-driving car – to explore the different Kafka offerings available on the market. The goal is not a feature-by-feature comparison. Instead, the intention is to educate about the different deployment models, product strategies, and trade-offs from the available options.
Read More
0
0
0