Kai WaehnerAirlines operate some of the most complex IT environments in any industry. Every flight creates a constant flow of events that must be processed, shared, and acted on immediately. Small delays can cascade across airports, fleets, and customers within minutes. This blog post explains why data streaming has become a foundational capability in the aviation industry. It shows how real-time data supports safety, operational excellence, and passenger experience at scale. Qantas is used as a concrete example in this blog post. The patterns and practices apply across the global aviation industry.
Join the data streaming community and stay informed about new blog posts by subscribing to my newsletter and follow me on LinkedIn or X (former Twitter) to stay in touch. And make sure to download my free book about data streaming use cases, including success stories around Apache Kafka and Flink at Lufthansa and Schiphol Group (Amsterdam Airport).
Data Streaming in the Aviation Industry
The aviation industry runs on real-time information. Aircraft movements, crew activities, baggage handling, and passenger communications all depend on timely and accurate data. Batch based integration is too slow and too fragile for these requirements.
A data streaming platform built on Apache Kafka and Flink enables airlines to connect operational systems, stream events continuously, and react in seconds instead of hours. This shift supports better on time performance, faster recovery from disruption, and more transparent operations across partners.
Several leading airlines already demonstrate this approach in production:
-
Lufthansa shows how real-time event streaming connects legacy systems and improves airline operations and customer experience.
-
Etihad Airways explains how a real-time data mesh enables self service analytics and operational agility across the airline.
-
Cathay Pacific highlights the role of a data streaming platform as the backbone for airline operations and digital services.
- Virgin Australia built a flight state engine and also revamped its loyalty and rewards program.
These examples show a clear industry trend. Airlines are moving away from isolated systems and nightly batch jobs toward shared, governed, real-time data platforms. The Australian airline Qantas follows the same path, with a strong focus on operational reliability, scalability, and simplicity.
Qantas: 100 Years of Aviation
Qantas is the world’s oldest continuously operating airline. Founded in 1920, it carries over 50 million passengers a year with a fleet of more than 310 aircraft. The Qantas Group includes Qantas Domestic, Qantas International, Jetstar, Qantas Freight, and the popular Qantas Loyalty program.
Their operations span commercial flights, cargo logistics, travel planning, and customer engagement. With so many touchpoints, Qantas needs consistent and reliable real-time data across all areas.
This blog post is based on insights shared by Simon Aubury, Principal Engineer at Qantas, during his talk at Current 2025 in New Orleans. Watch “From Cockpit to Kafka: Streaming Design Lessons from Aviation” to see the full session.
Qantas Data Streaming Platform
For Qantas, near real-time information is critical for both operations and customer experience. It supports end-to-end business processes such as flight scheduling, crew coordination, baggage management, and customer communications, ensuring every part of the journey stays connected and responsive.
The company built the Qantas Data Streaming Platform (DSP) to support various business units. This shared group streaming platform enables real-time data flows between systems across Qantas, Jetstar, Loyalty, and Freight.
The DSP provides:
- Data integration through Kafka Connect
- Stream processing with tools like ksqlDB
- Governance for traceability and access control
- Security and privacy enforcement
- Reliable operations via retry policies, dead letter queues, and idempotent processing
Let’s look at two specific use cases that show the value of this platform in action.
Use Case: Airport Collaborative Decision Making (A-CDM) to Enhance Coordination Across Airport Operations
Airport Collaborative Decision Making (A-CDM) improves how airports, airlines, and ground handlers coordinate. Each flight event — like pushback, taxi, takeoff, and landing — is tracked and shared among systems in real time.
With an event-driven architecture leveraging Apache Kafka as the backbone, Qantas integrates timestamps and events for each phase of a flight:
- Target Off-Block Time (TOBT)
- Actual Off-Block Time (AOBT)
- Take-Off and Landing Time
- Arrival Block Time
By streaming these updates, Qantas improves prediction accuracy and reduces delays. Systems can automatically adjust based on real-time data, keeping all stakeholders aligned.
Use Case: Optimizing Ground Operations with the Turn Manager
The turnaround process — the time between an aircraft’s arrival and departure — is one of the most complex parts of airline operations. Dozens of activities happen in parallel: baggage unloading, catering, crew change, refueling, cleaning, and boarding.
Qantas built a Turn Manager powered by Kafka to track all turnaround activities in real time. This provides a “single pane of glass” for each flight:
- Inbound aircraft details
- Passenger transfers
- Baggage load
- Real-time events: doors open, fuel complete, boarding start
If one activity is delayed, the system automatically updates downstream tasks. This improves on-time performance and helps ground staff focus on the highest priorities.
Integrating AIDX Flight Data with Kafka Connect and Single Message Transform (SMT)
Aviation uses complex data formats. A common standard is AIDX (Aviation Information Data Exchange), which uses XML to transmit flight and operational data. This includes everything from flight status to fuel requests and baggage handling.
To integrate AIDX messages into Kafka, Qantas uses Kafka Connect with Single Message Transforms (SMT).
Here’s how it works:
- AIDX XML messages are received from airport or airline systems.
- A custom SMT transforms these into a more usable format (e.g., Avro or JSON).
- The transformed message is published into Kafka topics.
- Downstream consumers can subscribe to specific topics such as “boarding”, “fueling”, or “gate change”.
These transformations also validate data and apply redactions for sensitive information. For example, personal identifiers may be masked to meet privacy rules.
The Business Impact of Real-Time Streaming in Airline Operations
Qantas proves that data streaming is not just a technical upgrade. It is a strategic enabler for airline operations. By moving from siloed systems and slow batch updates to real-time event streaming, the airline has improved on-time performance, streamlined ground operations, and created a consistent flow of trusted data across its entire ecosystem.
The business value is clear. Real-time data allows for faster and more accurate decision-making. Teams can respond to disruptions in minutes, not hours. Passengers receive timely updates. Operational risks are reduced. Each improvement translates to cost savings, better service levels, and stronger customer loyalty.
As airlines continue to evolve in a highly competitive and regulated industry, having a scalable and governed data streaming platform becomes essential. It connects all stakeholders across airports, fleets, and service providers. It helps IT leaders modernize legacy infrastructure without breaking business-critical workflows.
Data streaming has become a foundation for safety, efficiency, and innovation. Qantas is just one example of how this approach creates lasting business impact. The same patterns apply across the aviation industry and beyond.
Join the data streaming community and stay informed about new blog posts by subscribing to my newsletter and follow me on LinkedIn or X (former Twitter) to stay in touch. And make sure to download my free book about data streaming use cases, including success stories around Apache Kafka and Flink at Lufthansa and Schiphol Group (Amsterdam Airport).
