Comparison of Stream Processing and Streaming Analytics Alternatives (Apache Storm, Spark, IBM InfoSphere Streams, TIBCO StreamBase, Software AG Apama)

Posted in Analytics, Big Data, Business Intelligence, Hadoop on September 10th, 2014 by Kai Wähner

The demand for stream processing is increasing a lot these days. Frameworks (Apache Storm, Spark) and products (e.g. IBM InfoSphere Streams, TIBCO StreamBase, Software AG Apama) for stream processing and streaming analytics are getting a lot of attention these days. The reason is that often processing big volumes of data is not enough. Data has to be processed fast, so that a firm can react to changing business conditions in real time. This is required for trading, fraud detection, system monitoring, and many other examples. A “too late architecture” cannot realize these use cases.

Tags: , , , , , , , , , , , , , , , ,

“Hadoop and Data Warehouse (DWH) – Friends, Enemies or Profiteers? What about Real Time?” – Slides (including TIBCO Examples) from JAX 2014 Online

Posted in Analytics, Big Data, Business Intelligence, Cloud, ESB, Hadoop on May 13th, 2014 by Kai Wähner

Slides from my talk “Hadoop and Data Warehouse (DWH) – Friends, Enemies or Profiteers? What about Real Time?” at JAX 2014 (Twitter #jaxcon) in Mainz are online. JAX is a great conference with interesting topics and many good speakers!

Content (Data Warehouse, Business Intelligence, Hadoop, Stream Processing)

Big data represents a significant paradigm shift in enterprise technology. Big data radically changes the nature of the data management profession as it introduces new concerns about the volume, velocity and variety of corporate data. New business models based on predictive analytics, such as recommendation systems or fraud detection, are relevant more than ever before. Apache Hadoop seems to become the de facto standard for implementing big data solutions. For that reason, solutions from many different vendors emerged on top of Hadoop.

Tags: , , , , , , , , , , , , , , , , , , , , , , , ,

Integration of Amazon Redshift Cloud Data Warehouse (AWS SaaS DWH) with Talend Data Integration (DI) / Big Data (BD) / Enterprise Service Bus (ESB)

Posted in Cloud, EAI, ESB on June 26th, 2013 by Kai Wähner

In this blog post, I will show you how to „ETL“ all kinds of data to Amazon’s cloud data warehouse Redshift wit Talend’s big data components. Let’s begin with a short introduction to Amazon Redshift (copied from website):

„Amazon Redshift is [part of Amazon Web Services (AWS) and] a fast and powerful, fully managed, petabyte-scale data warehouse service in the cloud. With a few clicks in the AWS Management Console, customers can launch a Redshift cluster, starting with a few hundred gigabytes and scaling to a petabyte or more, for under $1,000 per terabyte per year.
Traditional data warehouses require significant time and resource to administer, especially for large datasets. In addition, the financial cost associated with building, maintaining, and growing self-managed, on-premise data warehouses is very high. Amazon Redshift not only significantly lowers the cost of a data warehouse, but also makes it easy to analyze large amounts of data very quickly.“

Tags: , , , , , , , , , , , , , ,