Rethinking Stream Processing with Apache Kafka, Kafka Streams and KSQL

I presented at JavaLand 2018 in Brühl recently. A great developer conference with over 1800 attendees. The location is also awesome! A theme park: Phantasialand. My talk: “New Era of Stream Processing with Apache Kafka’s Streams API and KSQL“. Just want to share the slide deck…

Abstract

Stream Processing is a concept used to act on real-time streaming data. This session shows and demos how teams in different industries leverage the innovative Streams API from Apache Kafka to build and deploy mission-critical streaming real time application and microservices.

The session discusses important Streaming concepts like local and distributed state management, exactly once semantics, embedding streaming into any application, deployment to any infrastructure. Afterwards, the session explains key advantages of Kafka’s Streams API like distributed processing and fault-tolerance with fast failover, no-downtime rolling deployments and the ability to reprocess events so you can recalculate output when your code changes.

A demo shows how to combine any custom code with your streams application – by an example using an analytic model built with any machine learning framework like Apache Spark ML or TensorFlow.

The end of the session introduces KSQL – the open source Streaming SQL Engine for Apache Kafka. Write “simple” SQL streaming queries with the scalability, throughput and fail-over of Kafka Streams under the hood.