This post shares a slide deck and video recording of the differences between an event-driven streaming platform like Apache Kafka and middleware like Message Queues (MQ), Extract-Transform-Load (ETL) and Enterprise Service Bus (ESB).
KSQL – The Open Source Streaming SQL Engine for Apache Kafka => Slides from my talk at Big Data Spain 2018 are online. Check it out!
Apache Kafka Streams to build Real Time Streaming Microservices. Apply Machine Learning / Deep Learning using Spark, TensorFlow, H2O.ai, etc. to add AI. Embed Kafka Streams into Java App, Docker, Kubernetes, Mesos, anything else.
Streaming Analytics Comparison of Open Source Frameworks, Products and Cloud Services. Includes Apache Storm, Flink, Spark, TIBCO, IBM, AWS Kinesis, Striim, Zoomdata, …
Comparison of Open Source IoT Integration Frameworks such as Eclipse Kura (+ Apache Camel), Node-RED, Flogo, Apache Nifi, StreamSets, and others… (slide and video recording)
Introduction to the Open Source IoT Integration Project Flogo to build very lightweight edge applications and microservices for cloud native containers / cloud platforms / serverless architectures.
Log Analytics is the right framework or tool to monitor for Distributed Microservices. Comparison of Open source, SaaS and Enteprrise Products. Plus relation to big data components such as Apache Hadoop / Spark.
Closed Big Data Loop: 1) Finding Insights with R, H20, Apache Spark MLlib, PMML and TIBCO Spotfire. 2) Putting Analytic Models into Action via Event Processing and Streaming Analytics.
See how stream processing / streaming analytics frameworks (e.g. Apache Spark, Apache Flink, Amazon Kinesis) and products (e.g. TIBCO StreamBase, Software AG’s Apama, IBM InfoSphere Streams) are categorized and compared. Besides, understand how stream processing is related to Big Data platforms such as Apache Hadoop and machine learning (e.g. R, SAS, MATLAB).
The article discusses what stream processing is, how it fits into a big data architecture with Hadoop and a data warehouse (DWH), when stream processing makes sense, and what technologies and products you can choose from. Comparison of open source and proprietary stream processing / streaming analytics alternatives: Apache Storm, Spark, IBM InfoSphere Streams, TIBCO StreamBase, Software AG’s Apama, etc.