Apache Kafka Streams + Machine Learning (Spark, TensorFlow, H2O.ai)

Posted in Analytics, Apache Kafka, Apache Spark, Big Data, Confluent, Hadoop, Integration, Kafka Connect, Kafka Streams, Machine Learning, Messaging, Microservices, Open Source, Stream Processing on May 23rd, 2017 by Kai Wähner

I started at Confluent in May 2017 to work as Technology Evangelist focusing on topics around the open source framework Apache Kafka. I think Machine Learning is one of the hottest buzzwords these days as it can add huge business value in any industry. Therefore, you will see various other posts from me around Apache Kafka (messaging), Kafka Connect (integration), Kafka Streams (stream processing), Confluent’s additional open source add-ons on top of Kafka (Schema Registry, Replicator, Auto Balancer, etc.). I will explain how to leverage all this for machine learning and other big data technologies in real world production scenarios.

Tags: , , , , , , , , , , , , , , , , , , ,

Comparison: Data Preparation vs. Inline Data Wrangling in Machine Learning and Deep Learning Projects

Posted in Analytics, Big Data, Business Intelligence, Hadoop on February 13th, 2017 by Kai Wähner

I want to highlight a new presentation about Data Preparation in Data Science projects:

“Comparison of Programming Languages, Frameworks and Tools for Data Preprocessing and (Inline) Data Wrangling  in Machine Learning / Deep Learning Projects”

Data Preparation as Key for Success in Data Science Projects

A key task to create appropriate analytic models in machine learning or deep learning is the integration and preparation of data sets from various sources like files, databases, big data storages, sensors or social networks. This step can take up to 80% of the whole project.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Streaming Analytics Comparison of Open Source Frameworks, Products, Cloud Services

Posted in Analytics, Big Data, Business Intelligence, Cloud, Hadoop on November 15th, 2016 by Kai Wähner

In November 2016, I am at Big Data Spain in Madrid for the first time. A great conference with many awesome speakers and sessions about very hot topics such as Apache Hadoop, Spark Spark, Streaming Processing / Streaming Analytics and Machine Learning. If you are interested in big data, then this conference is for you! My two talks:

  • How to Apply Machine Learning to Real Time Processing” (see slides and video recording from a similar conference talk).
  • Comparison of Streaming Analytics Options” (the reason for this blog post; an updated version of my talk from JavaOne 2015)
Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Machine Learning Applied to Microservices

Posted in Analytics, Big Data, Business Intelligence, Cloud, Docker, Hadoop, Microservices, Middleware on October 20th, 2016 by Kai Wähner

I had two sessions at O’Reilly Software Architecture Conference in London in October 2016. It is the first #OReillySACon in London. A very good organized conference with plenty of great speakers and sessions. I can really recommend this conference and its siblings in other cities such as San Francisco or New York if you want to learn about good software architectures and new concepts, best practices and technologies. Some of the hot topics this year besides microservices are DevOps, serverless architectures and big data analytics respectively machine learning.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Characteristics of a Good Visual Analytics and Data Discovery Tool

Posted in Analytics, Big Data, Business Intelligence, Hadoop on July 28th, 2016 by Kai Wähner

Visual Analytics and Data Discovery allow analysis of big data sets to find insights and valuable information. This is much more than just classical Business Intelligence (BI). See this article for more details and motivation: “Using Visual Analytics to Make Better Decisions: the Death Pill Example“. Let’s take a look at important characteristics to choose the right tool for your use cases.

Visual Analytics Tool Comparison and Evaluation

Several tools are available on the market for Visual Analytics and Data Discovery. Three of the most well known options are Tableau, Qlik and TIBCO Spotfire. Use the following list to compare and evaluate different tools to make the right decision for your project:

Tags: , , , , , , , , , , , , , , , , , , , ,

Streaming Analytics with Analytic Models (R, Spark MLlib, H20, PMML)

Posted in Analytics, Big Data, Business Intelligence, Hadoop, In Memory, NoSQL on March 3rd, 2016 by Kai Wähner

In March 2016, I had a talk at Voxxed Zurich about “How to Apply Machine Learning and Big Data Analytics to Real Time Processing”.

Kai_Waehner_at_Voxxed_Zurich

Finding Insights with R, H20, Apache Spark MLlib, PMML and TIBCO Spotfire

Big Data” is currently a big hype. Large amounts of historical data are stored in Hadoop or other platforms. Business Intelligence tools and statistical computing are used to draw new knowledge and to find patterns from this data, for example for promotions, cross-selling or fraud detection. The key challenge is how these findings can be integrated from historical data into new transactions in real time to make customers happy, increase revenue or prevent fraud.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,