Apache Kafka + MQTT = End-to-End IoT Integration (Code, Slides, Video)

Posted in Apache Kafka, Big Data, Confluent, EAI, Internet of Things, Kafka Connect, Messaging, Microservices, MQTT, Open Source, Stream Processing on September 10th, 2018 by Kai Wähner

Kafka and MQTT are two complementary technologies. Together they allows to build IoT end-to-end integration from the edge to the data center. No matter if on premise or in the public cloud. I talked about this topic at Kafka Summit SF in San Francisco in October 2018: “Processing IoT Data from End to End with MQTT and Apache Kafka“. The main goal was to discuss different Kafka-native approaches and their trade-offs for integrating Kafka and MQTT.

Tags: , , , , , , , , , , , , , ,

Apache Kafka vs. ESB / ETL / MQ

Posted in Apache Kafka, Big Data, Confluent, EAI, ESB, Integration, Kafka Connect, Kafka Streams, KSQL, Messaging, Microservices, Middleware, Open Source, SOA, Stream Processing on July 18th, 2018 by Kai Wähner

Apache Kafka and Enterprise Service Bus (ESB) are complementary, not competitive!

Apache Kafka is much more than messaging in the meantime. It evolved to a streaming platform including Kafka Connect, Kafka Streams, KSQL and many other open source components. Kafka leverages events as a core principle. You think in data flows of events and process the data while it is in motion. Many concepts, such as event sourcing, or design patterns such as Enterprise Integration Patterns (EIPs), are based on event-driven architecture.

Tags: , , , , , , , , , , , , , , , , ,

Video Recording – Apache Kafka as Event-Driven Open Source Streaming Platform (Voxxed Zurich 2018)

Posted in Apache Kafka, Big Data, Cloud, Docker, EAI, ESB, Integration, Java / JEE, Kafka Connect, Kafka Streams, KSQL, Kubernetes, Messaging, Microservices, Middleware, Open Source, SOA, Stream Processing on March 13th, 2018 by admin

I spoke at Voxxed Zurich 2018 about Apache Kafka as Event-Driven Open Source Streaming Platform. The talk includes an intro to Apache Kafka and its open source ecosystem (Kafka Streams, Connect, KSQL, Schema Registry, etc.). Just want to share the video recording of my talk.

Abstract

This session introduces Apache Kafka, an event-driven open source streaming platform. Apache Kafka goes far beyond scalable, high volume messaging. In addition, you can leverage Kafka Connect for integration and the Kafka Streams API for building lightweight stream processing microservices in autonomous teams. The open source Confluent Platform adds further components such as a KSQL, Schema Registry, REST Proxy, Clients for different programming languages and Connectors for different technologies and databases. Live Demos included.

Tags: , , , , , , , , , , , , ,

Agile Cloud-to-Cloud Integration with iPaaS, API Management and Blockchain

Posted in API Management, Blockchain, Cloud, Cloud-Native, Docker, EAI, ESB, Microservices, Middleware on April 23rd, 2017 by Kai Wähner

Cloud-to-Cloud integration is part of a hybrid integration architecture. It enables to implement quick and agile integration scenarios without the burden of setting up complex VM- or container-based infrastructures. One key use case for cloud-to-cloud integration is innovation using a fail-fast methodology where you realize new ideas quickly. You typically think in days or weeks, not in months. If an idea fails, you throw it away and start another new idea. If the idea works well, you scale it out and bring it into production to a on premise, cloud or hybrid infrastructure. Finally, you make expose the idea and make it easily available to any interested service consumer in your enterprise, partners or public end users.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Cloud Native Middleware Microservices – 10 Lessons Learned (O’Reilly Software Architecture 2017, New York)

Posted in API Management, Cloud, Cloud-Native, Docker, EAI, ESB, Microservices, Middleware, SOA on April 5th, 2017 by Kai Wähner

I want to share my slide deck and video recordings from the talk “10 Lessons Learned from Building Cloud Native Middleware Microservices” at O’Reilly Software Architecture April 2017 in New York, USA in April 2017.

Abstract
Microservices are the next step after SOA: Services implement a limited set of functions; services are developed, deployed, and scaled independently; continuous delivery automates deployments. This way you get shorter time to results and increased flexibility. Containers improve things even more, offering a very lightweight and flexible deployment option.

Tags: , , , , , , , , , , , , , , , , , , ,

Case Study: From a Monolith to Cloud, Containers, Microservices

Posted in API Management, Cloud, Cloud-Native, Docker, EAI, ESB, Java / JEE, Microservices, Middleware, SOA on February 24th, 2017 by Kai Wähner

The following shows a case study about successfully moving from a very complex monolith system to a cloud-native architecture. The architecture leverages containers and Microservices. This solve issues such as high efforts for extending the system, and a very slow deployment process. The old system included a few huge Java applications and a complex integration middleware deployment.

The new architecture allows flexible development, deployment and operations of business and integration services. Besides, it is vendor-agnostic so that you can leverage on-premise hardware, different public cloud infrastructures, and cloud-native PaaS platforms.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , ,

TIBCO’s Hybrid Integration Platform

Posted in API Management, Cloud, Cloud-Native, Docker, EAI, ESB, Microservices, SOA on September 14th, 2016 by Kai Wähner

[Originally posted on the TIBCO Blog]

The IT world is moving forward rapidly. The digital transformation changes complete industries and peels away existing business models. Cloud services, mobile devices, and the Internet of Things establish wild spaghetti architectures through different departments and lines of business. Several different concepts, technologies, and deployment options are used. A single integration backbone is not sufficient in this era anymore.

A hybrid integration platform for core and edge services

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Hybrid Integration Architecture is the New Default

Posted in API Management, Cloud, Cloud-Native, Docker, EAI, ESB, Microservices, Middleware, SOA on August 5th, 2016 by Kai Wähner

The IT world is moving forward fast. The digital transformation changes complete industries and peels away existing business models. Cloud services, mobile devices and the Internet of Things establish wild spaghetti architectures though different departments and lines of business. Several different concepts, technologies and deployment options are used. A single integration backbone is not sufficient anymore in this era of integration. Therefore, a Hybrid Integration Architecture is getting the new default in most enterprises.

Different user roles need to leverage different tools to integrate applications, services and APIs for their specific need. A key for success is that all integration and business services work together across different platforms in a hybrid world with on premise and cloud deployments.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Why TIBCO ActiveMatrix BusinessWorks 6 Is a Leading Integration and Service Delivery Platform

Posted in EAI, ESB, Microservices, Middleware, SOA on July 8th, 2016 by Kai Wähner

TIBCO ActiveMatrix BusinessWorks 6 is a modern integration and service delivery platform ready for modern concepts such Mobile, Cloud, Internet of Things or Microservices. This article is no introduction to integration and service delivery platforms in general, but focuses on differentiators of BusinessWorks 6 compared to other integration solutions available on the market. If you want to see a general introduction to TIBCO ActiveMatrix BusinessWorks 6, explaining its basic concepts and capabilities such as:

  • zero-coding process designer for quick implementation, testing, and maintenance
  • distributed scalability for mission-critical systems and ultra-high performance
  • standards-based service and application integration
Tags: , , , , , , , , , , , , , , , , , , , , , , , ,

Microservices, Containers, Docker and a Cloud-Native Architecture in the Middleware World

Posted in API Management, BPM, Cloud, Cloud-Native, Docker, EAI, ESB, Java / JEE, Microservices, Middleware, SOA on April 22nd, 2016 by Kai Wähner

In April 2016, I had two talks at JPoint in Moscow, Russia. The first talk was an existing talk about “How to Apply Big Data Analytics and Machine Learning to Real Time Processing”. The second talk was a brand new one: “Microservices and Containers in the Middleware World”. This article discusses and shows the new slide deck about how middleware is related to Microservices, Containers, Docker and Cloud-Native Architectures.

Key takeaways of the talk:

  • A cloud-native architecture enables flexible and agile development
  • Modern middleware leverages containers, Microservices and a cloud-native architecture
Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,