Mental workload and eye movements during conditionally automated driving in hazardous environments

• Main Author: Du, Bo
• Other Authors: Xu Hong
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Social sciences::Psychology
• Online Access: https://hdl.handle.net/10356/159236
• Institution: Nanyang Technological University

Driving in monotonous road environments may cause drivers to become bored and impair their alertness. This inattention may increase the risk of road accidents. Drivers usually respond to visual signals on the road. A large body of research has shown that the level of a driver’s visual attention is closely related to traffic accidents. Visual attention is often studied by tracking drivers’ eye movements. However, few studies to date have explored eye-movement patterns in different road environments such as open roads and tunnel expressways. This dissertation aims to address that research gap by studying drivers’ eye movements and visual characteristics in different driving conditions in real and simulated environments. In the pilot study, drivers’ eye movement patterns were recorded during high-speed driving in tunnels and on expressways in Singapore. Twenty-two drivers participated in the study; they drove a total of 55 km, which included a 9-km section of tunnel. The results of the study showed that drivers experienced longer fixation duration when driving in tunnels than on the open road but experienced a higher number of fixations, higher instantaneous velocity, acceleration, and dispersion on open roads than in tunnels. These findings suggest that drivers’ eye movements are more concentrated and have longer fixation durations in tunnels than in open road driving, indicating that drivers have a greater mental workload when driving in tunnels. However, testing drivers in a real environment has limitations. For example, weather and road characteristics cannot be manipulated. To address this limitation, Experiment 1 was conducted in a driving simulator using virtual reality simulations in UC-win/road. Experiment 1 sought to investigate drivers’ eye movements in various driving environments. Eighteen drivers participated in the experiment and drove on normal roads, expressways, and through underground tunnels in both sunny and rainy weather. The comparisons of their eye-movement patterns (e.g., saccadic parameters and pupil size) in the simulated environments were consistent with those in real open-road environments and tunnels in sunny conditions. The findings suggest that the eye-movement patterns of drivers in a simulator are comparable to those in real environments, indicating its ecological validity. Previous literature has suggested a relationship between eye movement patterns and mental workload. To verify that relationship, mental workloads in Experiments 2 and 3 were directly measured using questionnaires. Dynamic and static hazardous scenarios were simulated to induce different levels of mental workload. Experiment 2 investigated eye movements, driving performance, and mental workload of drivers who encountered static and dynamic obstacles while manual driving. The results from 23 drivers found that their mental workload in dynamic conditions was higher than in static hazardous conditions; it was also higher before encountering hazards than afterwards. The results indicated that pupil size and saccade numbers are effective indicators of mental workload. Experiment 3 was conducted under conditionally automated driving conditions, where participants were involved in tasks related to or not related to driving and requested to resume control manually when their vehicle was in semi-automated driving mode. Drivers’ eye movements and mental workloads were measured when they were driving, especially during the transitions from semi-automated to manual driving. Similar to Experiment 2, the results from 32 drivers found that mental workload was higher for dynamic hazards than for static hazards and higher before hazards were encountered than after. The results also indicated that pupil size is the only effective indicator of mental workload. Taken together, the results of these experiments reveal that pupil size is the most reliable way to measure drivers’ mental workload. Furthermore, in line with the load theory of attention and De Waard (1996)’s theory of workload and performance, it can be concluded that mental workload (indicated by pupil size) and visual workload (by saccadic dispersion) are the key determinants of driving performance and that drivers can only respond to emergencies during periods of medium levels of mental and visual workload (not underload or overload) with available attentional capacity. The findings from this thesis on eye movement and mental workload may be used to monitor drivers’ real-time attention and workload and thus provide guidance for the design of safe-driving autonomous vehicles that consider drivers’ capacity for information processing when driving. Knowledge about the limitations and capacity of human drivers in tandem with automated systems may also help improve user acceptance of autonomous vehicles in the future.