Tangible digital twin with shared visualization for collaborative air traffic management operations

In recent years, digital twin technology has become increasingly popular and been widely applied across different industries including air traffic management (ATM). While previous studies on digital twin in ATM focused on factors such as system architecture, synthetic data generation and transfer, a...

Full description

Saved in:
Bibliographic Details
Main Authors: Chen, Ken, Nadirsha, Thaivalappil N. M., Lilith, Nimrod, Alam, Sameer, Svensson, Åsa
Other Authors: School of Mechanical and Aerospace Engineering
Format: Article
Language:English
Published: 2024
Subjects:
Online Access:https://hdl.handle.net/10356/174715
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-174715
record_format dspace
spelling sg-ntu-dr.10356-1747152024-04-13T16:49:01Z Tangible digital twin with shared visualization for collaborative air traffic management operations Chen, Ken Nadirsha, Thaivalappil N. M. Lilith, Nimrod Alam, Sameer Svensson, Åsa School of Mechanical and Aerospace Engineering Saab-NTU Joint Lab Engineering Mixed reality Air traffic management In recent years, digital twin technology has become increasingly popular and been widely applied across different industries including air traffic management (ATM). While previous studies on digital twin in ATM focused on factors such as system architecture, synthetic data generation and transfer, and real-time simulation, there is currently a lack of research on visualization and interaction design of digital twin from human factors’ perspective. Moreover, recent advances in virtual and mixed reality (VR/MR) technology give potential for designing interactive digital twin in 3D space that can assist human operators in performing their duties. Therefore, this research aims to explore visualization and interaction design of an MR-based digital twin prototype that can assist air traffic controllers (ATCOs) in carrying out ground control tasks. Specifically, MR headsets (Microsoft HoloLens 2) are leveraged to project out-of-tower view of airport traffic onto a 3D printed airport model at a 1:1 scale. The spatially aligned tangible system enables ATCOs to perform typical ATM operations by directly touching the 3D printed airport model. In addition, collaborative features are implemented into the system, which enable multiple ATCOs to have a shared view of the airport traffic and collaboratively perform ATM operations when wearing the MR headsets. Finally, a user study was conducted with the recruitment of ten licensed ATCOs to investigate usability of the system and ATCOs’ workload, situational awareness and trust when using the system. The feedback from the ATCOs in the user study suggested that: (1) The visual and interactive design were acceptable; (2) The ATCOs indicated varied workload under different scenarios; (3) The ATCOs developed a high level of shared situational awareness with shared view of the system; and (4) The ATCOs were open and confident to use the system in its’ mature form. The findings show that the MR-based tangible airport digital tower system has high potential in collaborative ATM operations by improving user performance and user experience. Submitted/Accepted version 2024-04-08T05:50:21Z 2024-04-08T05:50:21Z 2024 Journal Article Chen, K., Nadirsha, T. N. M., Lilith, N., Alam, S. & Svensson, Å. (2024). Tangible digital twin with shared visualization for collaborative air traffic management operations. Transportation Research Part C: Emerging Technologies, 161, 104546-. https://dx.doi.org/10.1016/j.trc.2024.104546 0968-090X https://hdl.handle.net/10356/174715 10.1016/j.trc.2024.104546 2-s2.0-85187139937 161 104546 en Transportation Research Part C: Emerging Technologies © 2024 Elsevier Ltd. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1016/j.trc.2024.104546. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering
Mixed reality
Air traffic management
spellingShingle Engineering
Mixed reality
Air traffic management
Chen, Ken
Nadirsha, Thaivalappil N. M.
Lilith, Nimrod
Alam, Sameer
Svensson, Åsa
Tangible digital twin with shared visualization for collaborative air traffic management operations
description In recent years, digital twin technology has become increasingly popular and been widely applied across different industries including air traffic management (ATM). While previous studies on digital twin in ATM focused on factors such as system architecture, synthetic data generation and transfer, and real-time simulation, there is currently a lack of research on visualization and interaction design of digital twin from human factors’ perspective. Moreover, recent advances in virtual and mixed reality (VR/MR) technology give potential for designing interactive digital twin in 3D space that can assist human operators in performing their duties. Therefore, this research aims to explore visualization and interaction design of an MR-based digital twin prototype that can assist air traffic controllers (ATCOs) in carrying out ground control tasks. Specifically, MR headsets (Microsoft HoloLens 2) are leveraged to project out-of-tower view of airport traffic onto a 3D printed airport model at a 1:1 scale. The spatially aligned tangible system enables ATCOs to perform typical ATM operations by directly touching the 3D printed airport model. In addition, collaborative features are implemented into the system, which enable multiple ATCOs to have a shared view of the airport traffic and collaboratively perform ATM operations when wearing the MR headsets. Finally, a user study was conducted with the recruitment of ten licensed ATCOs to investigate usability of the system and ATCOs’ workload, situational awareness and trust when using the system. The feedback from the ATCOs in the user study suggested that: (1) The visual and interactive design were acceptable; (2) The ATCOs indicated varied workload under different scenarios; (3) The ATCOs developed a high level of shared situational awareness with shared view of the system; and (4) The ATCOs were open and confident to use the system in its’ mature form. The findings show that the MR-based tangible airport digital tower system has high potential in collaborative ATM operations by improving user performance and user experience.
author2 School of Mechanical and Aerospace Engineering
author_facet School of Mechanical and Aerospace Engineering
Chen, Ken
Nadirsha, Thaivalappil N. M.
Lilith, Nimrod
Alam, Sameer
Svensson, Åsa
format Article
author Chen, Ken
Nadirsha, Thaivalappil N. M.
Lilith, Nimrod
Alam, Sameer
Svensson, Åsa
author_sort Chen, Ken
title Tangible digital twin with shared visualization for collaborative air traffic management operations
title_short Tangible digital twin with shared visualization for collaborative air traffic management operations
title_full Tangible digital twin with shared visualization for collaborative air traffic management operations
title_fullStr Tangible digital twin with shared visualization for collaborative air traffic management operations
title_full_unstemmed Tangible digital twin with shared visualization for collaborative air traffic management operations
title_sort tangible digital twin with shared visualization for collaborative air traffic management operations
publishDate 2024
url https://hdl.handle.net/10356/174715
_version_ 1800916413591846912