In November 2016, I am at Big Data Spain in Madrid for the first time. A great conference with many awesome speakers and sessions about very hot topics such as Apache Hadoop, Spark Spark, Streaming Processing / Streaming Analytics and Machine Learning. If you are interested in big data, then this conference is for you! My two talks:
I had two sessions at O’Reilly Software Architecture Conference in London in October 2016. It is the first #OReillySACon in London. A very good organized conference with plenty of great speakers and sessions. I can really recommend this conference and its siblings in other cities such as San Francisco or New York if you want to learn about good software architectures and new concepts, best practices and technologies. Some of the hot topics this year besides microservices are DevOps, serverless architectures and big data analytics respectively machine learning.
The IT world is moving forward fast. The digital transformation changes complete industries and peels away existing business models. Cloud services, mobile devices and the Internet of Things establish wild spaghetti architectures though different departments and lines of business. Several different concepts, technologies and deployment options are used. A single integration backbone is not sufficient anymore in this era of integration. Therefore, a Hybrid Integration Architecture is getting the new default in most enterprises.
Different user roles need to leverage different tools to integrate applications, services and APIs for their specific need. A key for success is that all integration and business services work together across different platforms in a hybrid world with on premise and cloud deployments.
In March 2016, I had a talk at Voxxed Zurich about “How to Apply Machine Learning and Big Data Analytics to Real Time Processing”.
Finding Insights with R, H20, Apache Spark MLlib, PMML and TIBCO Spotfire
“Big Data” is currently a big hype. Large amounts of historical data are stored in Hadoop or other platforms. Business Intelligence tools and statistical computing are used to draw new knowledge and to find patterns from this data, for example for promotions, cross-selling or fraud detection. The key challenge is how these findings can be integrated from historical data into new transactions in real time to make customers happy, increase revenue or prevent fraud.
I was invited to speak at Microservices Meetup Dublin this week. I updated my slide deck “Microservices – Death of the ESB?” … The meetup was fully booked with a waiting list; around 120 attendees came to Gild‘s office. (see attached link).
If you have not seen the slide deck last year, you should definitely take a look at this updated version with more recent information. I also incorporated valuable information from discussions with attendees in 2015’s sessions about this topic.
Data Warehouses have existed for many years in almost every company. While they are still as good and relevant for the same use cases as they were 20 years ago, they cannot solve new, existing challenges and those sure to come in a ever-changing digital world. The upcoming sections will clarify when to still use a Data Warehouse and when to use a modern Live Datamart instead.
What is a Data Warehouse (DWH)?
A Data Warehouse is a central repository of integrated data from more disparate sources. It stores historical data to create analytical reports for knowledge workers throughout the enterprise. A DWH includes a server, which stores the historical data and a client for analysis and reporting.
TIBCO BusinessWorks and StreamBase for Big Data Integration and Streaming Analytics with Apache Hadoop and ImpalaPosted in Analytics, Big Data, Business Intelligence, Hadoop, In Memory, NoSQL on April 14th, 2015 by Kai Wähner
Apache Hadoop is getting more and more relevant. Not just for Big Data processing (e.g. MapReduce), but also for Fast Data processing (e.g. Stream Processing). Recently, I published two blog posts on the TIBCO blog to show how you can leverage TIBCO BusinessWorks 6 and TIBCO StreamBase to realize Big Data and Fast Data Hadoop use cases.
“Hadoop and Data Warehouse (DWH) – Friends, Enemies or Profiteers? What about Real Time?” – Slides (including TIBCO Examples) from JAX 2014 OnlinePosted in Analytics, Big Data, Business Intelligence, Cloud, ESB, Hadoop on May 13th, 2014 by Kai Wähner
Slides from my talk “Hadoop and Data Warehouse (DWH) – Friends, Enemies or Profiteers? What about Real Time?” at JAX 2014 (Twitter #jaxcon) in Mainz are online. JAX is a great conference with interesting topics and many good speakers!
Content (Data Warehouse, Business Intelligence, Hadoop, Stream Processing)
Big data represents a significant paradigm shift in enterprise technology. Big data radically changes the nature of the data management profession as it introduces new concerns about the volume, velocity and variety of corporate data. New business models based on predictive analytics, such as recommendation systems or fraud detection, are relevant more than ever before. Apache Hadoop seems to become the de facto standard for implementing big data solutions. For that reason, solutions from many different vendors emerged on top of Hadoop.