Apache Kafka and Machine Learning for Real Time Supply Chain Optimization in IIoT

IoT and Supply Chain Optimization with Apache Kafka and Machine Learning
Apache Kafka and Machine Learning for Real Time Supply Chain Optimization: Integrate in real time with the legacy world and proprietary IIoT protocols (like Siemens S7, Modbus, Beckhoff ADS, OPC-UA, et al). You can process the data at scale and then ingest it into a modern database or analytic / machine  learning framework.

I did a webinar with Confluent‘s partner Expero about “Apache Kafka and Machine Learning for Real Time Supply Chain Optimization“. This is a great example for anybody in automation industry / Industrial IoT (IIoT) like automotive, manufacturing, logistics, etc.

We explain how a real time event streaming platform can integrate in real time with the legacy world and proprietary IIoT protocols (like Siemens S7, Modbus, Beckhoff ADS, OPC-UA, et al). You can process the data at scale and then ingest it into a modern database (like AWS S3, Snowflake or MongoDB) or analytic / machine  learning framework (like TensorFlow, PyTorch or Azure Machine Learning Service).

Here is the architecture we use to discuss and implement the supply chain optimization use case leveraging real time stream processing and machine learning:

Apache Kafka TensorFlow IIoT Supply Chain Optimization

We leverage various components from the Apache Kafka ecosystem. This includes:

  • Kafka Connect as scalable and reliable integration framework
  • Kafka Connect connectors like PLC4X – a great Apache framework to integrate with IIoT protocols
  • KSQL for continuous processing (filter, transform, aggregate) of the sensor data
  • Kafka Streams to deploy the trained analytic models to a real time streaming scoring engine

Use Case: Supply Chain Optimization using Real Time and Batch Processes

Automating multifaceted, complex workflows requires hybrid solutions including streaming analytics of IOT data and batch analytics. This includes machine learning solutions and real time visualization. Leaders in organizations who are responsible for global supply chain planning are responsible for working with and integrating with data from disparate sources around the world. Many of these data sources output information in real time. This assists planners in operationalizing plans and interacting with manufacturing output. IOT sensors on manufacturing equipment and inventory control systems feed real time processing pipelines to match actuals productions figures against planned schedules to calculate yield efficiency.

Using information from both real time systems and batch optimization, supply chain managers are able to economize operations and automate tedious inventory and manufacturing accounting processes. Sitting on top of all of these systems is a supply chain visualization tool. This enables users’ visibility over the global supply chain. If you are responsible for key data integration initiatives, join for a detailed walk through of a customer’s use of this system built using Confluent and Expero tools.

What will you learn?

  • See different use cases in automation industry and Industrial IoT (IIoT) where an event streaming platform adds business value
  • Understand different architecture options to leverage Apache Kafka and Confluent Platform in IoT scenarios in the cloud, on premise data centers and at the edge
  • Learn how to leverage different analytics tools and machine learning frameworks in a flexible and scalable way
  • How real time visualization ties together streaming and batch analytics for business users, interpreters, and analysts
  • Understand how streaming and batch analytics optimize the supply chain planning workflow.
  • Conceptualize the intersection between resource utilization and manufacturing assets with long term planning and supply chain optimization.

Industrial Internet of Things (IIoT) in Real Time at Scale with Apache Kafka

Here is the slide deck and video recording. Have fun watching it. Please let me know if you have any feedback or questions:

Slide Deck:

Video Recording:

Apache Kafka IIoT Supply Chain Video Recording

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like

Apache Kafka + MQTT = End-to-End IoT Integration (Code, Slides, Video)

MQTT and Apache Kafka are a perfect combination for end-to-end IoT integration from edge to data center. This post discusses two different approaches and refers to implementations on Github using Apache Kafka, Kafka Connect, Confluent MQTT Proxy and Mosquitto.
Read More
Apache Kafka and PLC4X Architecture for IIoT Automation Industry
Read More

Apache Kafka, KSQL and Apache PLC4X for IIoT Data Integration and Processing

Data integration and processing in Industrial IoT (IIoT, aka Industry 4.0 or Automation Industry). Apache Kafka, its ecosystem (Kafka Connect, KSQL) and Apache PLC4X are a great open source choice to implement this integration end to end in a scalable, reliable and flexible way.
Read More