Real-time data beats slow data. That’s true for almost every use case. Nevertheless, enterprise architects build new infrastructures with the Lambda architecture that includes separate batch and real-time layers. This blog post explores why a single real-time pipeline, called Kappa architecture, is the better fit. Real-world examples from companies such as Disney, Shopify, Uber, and Twitter explore the benefits of Kappa but also show how batch processing fits into this discussion positively without the need for Lambda.
Can and should Apache Kafka replace a database? How long can and should I store data in Kafka?…
At OOP 2018 conference in Munich, I presented an updated version of my talk about building scalable, mission-critical…
I do a lot of presentations these days at meetups and conferences about how to leverage Apache Kafka and Kafka Streams to apply analytic models (built with H2O, TensorFlow, DeepLearning4J and other frameworks) to scalable, mission-critical environments. As many attendees have asked me, I created a video recording about this talk (focusing on live demos).
After three great years at TIBCO Software, I move back to open source and join Confluent, the company behind the open source project Apache Kafka to build mission-critical, scalable infrastructures for messaging, integration and stream processsing. In this blog post, I want to share why I see the future for middleware and big data analytics in open source technologies, why I really like Confluent, what I will focus on in the next months, and why I am so excited about this next step in my career.
Streaming Analytics Comparison of Open Source Frameworks, Products and Cloud Services. Includes Apache Storm, Flink, Spark, TIBCO, IBM, AWS Kinesis, Striim, Zoomdata, …
Build intelligent Microservices by applying Machine Learning and Advanced Analytics. Leverage Apache Hadoop / Spark with Visual Analytics and Stream Processing.
Log Analytics is the right framework or tool to monitor for Distributed Microservices. Comparison of Open source, SaaS and Enteprrise Products. Plus relation to big data components such as Apache Hadoop / Spark.
Closed Big Data Loop: 1) Finding Insights with R, H20, Apache Spark MLlib, PMML and TIBCO Spotfire. 2) Putting Analytic Models into Action via Event Processing and Streaming Analytics.
Slide deck from OOP 2016: Comparison of Frameworks and Products for Big Data Log Analytics and ITOA, e.g. Open Source ELK, TIBCO LogLogic / Unity, Splunk, Papertrail; Relation to Hadoop is also discussed.