Portal de Periódicos da CAPES

The Effects of Augmented-Reality Head-Up Display System Usage on Drivers? Risk Perception and Psychological Change

2016; Electronics and Telecommunications Research Institute; Linguagem: Inglês

10.4218/etrij.16.0115.0770

2233-7326

Yoonsook Hwang, Byoung-Jun Park, Kyong-Ho Kim,

Safety Warnings and Signage

ETRI JournalVolume 38, Issue 4 p. 757-766 ArticleFree Access Effects of Augmented-Reality Head-up Display System Use on Risk Perception and Psychological Changes of Drivers Yoonsook Hwang, Corresponding Author Yoonsook Hwang [email protected] Corresponding Author[email protected]Search for more papers by this authorByoung-Jun Park, Byoung-Jun Park [email protected] Search for more papers by this authorKyong-Ho Kim, Kyong-Ho Kim [email protected] Search for more papers by this author Yoonsook Hwang, Corresponding Author Yoonsook Hwang [email protected] Corresponding Author[email protected]Search for more papers by this authorByoung-Jun Park, Byoung-Jun Park [email protected] Search for more papers by this authorKyong-Ho Kim, Kyong-Ho Kim [email protected] Search for more papers by this author First published: 01 August 2016 https://doi.org/10.4218/etrij.16.0115.0770Citations: 6 Yoonsook Hwang (corresponding author, [email protected]), Byoung-Jun Park ([email protected]), and Kyong-Ho Kim ([email protected]) are with the SW & Content Research Laboratory, ETRI, Daejeon, Rep. of Korea. AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Abstract This study was conducted to examine the usage effects of an augmented reality head-up display (AR-HUD) system on the risk perception and psychological changes of drivers. To do so, we conducted an experiment to collect the driver response times for vehicles and pedestrians as their risk perception behavior, and used a driving behavior determinants questionnaire consisting of Problem Evading, Benefits/Sensation Seeking, Anti-Personal Anxiety, Anti-Personal Angry, and Aggression factors for collecting the psychological characteristics of the drivers. Thirty drivers were randomly assigned into an in-vehicle AR-HUD using group and a control group. As a result, the Anti-Personal Anxiety and Anti-Personal Angry factors were negatively correlated with the response time for the control group. In contrast, these results were not shown for the in-vehicle AR-HUD system using group. These results indicate that the in-vehicle AR-HUD system may partially induce a relaxation of tension or stress for drivers with a high level of interpersonal anxiety. Therefore, the in-vehicle AR-HUD system might contribute to not only the visual safety driving behaviors of drivers, but also to their psychological driving safety with specific characteristics. I. Introduction Science and technology have improved the safety of motor vehicles, enabling additional behaviors such the operation of navigation devices or smartphones. In addition, motor vehicles have become an indispensable part of our lives. Although motor vehicles and traffic roads are becoming safer than in the past, traffic accidents owing to driver distractions and additional behaviors still occur. There are large numbers of vehicles and pedestrians on the roads, and drivers are therefore occasionally under stress. Studies on driver stress have therefore been conducted. One such study reported meaningful results on the relationship between the physiological changes in drivers and their driving behaviors, such as steering-wheel angle corrections, velocity changes, and time responses under incremental stress conditions [1]. In another study, the use of an interactive prototype that displays the emotions of the driver indicated that driver stress is reduced when compared to the driver's original state [2]. In [3], speech modeling of the drivers for the classification of stress, including the short speech utterances of the drivers, was studied. In addition, another study attempted to recognize human emotions using physiological signals [4]. The National Highway Traffic Safety Administration reported that driver-distraction accounted for 10% of all fatal crashes, 18% of injury crashes, and 16% of all motor vehicle traffic crashes in 2012 [5]. An adaptive multimodal in-vehicle information system that can be used to estimate driver distractions was recently proposed [6]. Furthermore, for greater convenience, efficiency, and the safe manipulation of drivers, one study proposed a novel human-machine interface (HMI) user-interface/user-experience (UI/UX) system based on the recognition of diverse gestures that is applicable to an in-vehicle infotainment system [7]. Based on the results of these studies, we expect to be able to understand the cause and effect of driver distractions because emotions and stress also affect driver behaviors and states. In particular, a head-up display (HUD) technology is being developed for decreasing the visual distractions of drivers, and such technology is being launched in a number of commercial products. 1. Effects of a Head-up Display on Driver Behaviors Drivers acquire more than 90% of their driving-related information from their visual perception. Unlike other perception types, visual perception can be used to accurately obtain the driving information over a long distance, as well as for the movement direction, trajectory, and speed of the vehicle. Other types of perception can be used to obtain the driving information for only a close distance or narrow range of area [8]. For a visual interface that concentrates the driver's attention on the road ahead, one possible solution is the use of an HUD. In most cases, a driver's visual and cognitive driving-workload can be improved through clever human factor engineering combined with an HUD and obstacle detection technologies [9]. In particular, to deal with obstacles, new approaches for vehicle tracking and nighttime vehicle detection were proposed to improve the stability and robustness [10], [11]. In addition, a two-step pedestrian detection was used to reduce the computation time of the algorithm, and an iterative particle repropagation method was proposed to enhance its tracking accuracy [12]. An HUD shows the driving information exactly where the drivers need it–directly in their line of sight. Therefore, drivers can receive all of their important information, such as the vehicle speed, warning signals, and indicator arrows for navigation, without looking down at the instrument panel or a secondary display [13]. Since General Motors first began developing automobiles equipped with an HUD in 1999, many other companies (for example, BMW, PSA Group, and Cadillac) have also developed vehicles with an installed HUD [14]. The development of HUD technologies has made it possible to provide information within the driver's field of view. The sensitivity of visual cues leads a person to conduct their current task, which is known as selective visual attention [15]. In addition, according to the visual design, searching based on color has been shown to be the fastest search method when compared to size, brightness, or geometric shape [16]. In other words, the various time-to-collision (TTC) indicating colors of AR information appearing on an HUD might be encouraging as a visual cue for the driver's selective attention to be directed toward the item the HUD is highlighting. Therefore, a method for providing information using an HUD will speed up the driver's decision making during a risky situation because the information presented on the HUD operates as a cue. An HUD was suggested for reducing the number of collisions and improving the driver's ability to maintain the proper following distance when compared with a head-down display under low-visibility conditions [17]. In addition, the application of an HUD system can reduce the driver's response time during an emergency situation and allow the driver to more consistently control their driving speed [18]. Moreover, the use of an HUD reduces driver distractions and improves driving safety, and is estimated to contribute to up to 25% fewer vehicle crashes [19]. An augmented reality HUD (AR-HUD) system was recently reported to provide all detected vehicle and pedestrian information through a transparent display installed in front of the driver that fits the driver's view [20]. 2. Effects of Augmented Reality Technology on Driver Behaviors Safe driving requires the driver's attention, visibility, view, optimal traffic signs, a smooth ride, and a number of different available roads [21]. AR-HUD differs from a normal windshield HUD because the reflected information appears to be part of the driving situation itself. Therefore, the drivers will become aware of a critical situation even faster than without the use of an AR-HUD. In addition, it supplements the exterior view of the traffic conditions in front of the vehicle through augmented virtual information provided to the driver [13]. AR can improve an awareness of other road vehicles and pedestrians surrounding the driver. In addition, it can be used to increase the saliency of important elements in the driver's view and has the potential to enhance the driver's situation awareness. All drivers, regardless of age, can benefit through a trickle-down effect [22]. First, the drivers respond more quickly to lane change information when it is directly augmented into their perspective over the road surface, as compared to being displayed through 2D icons [23]. Second, in the context of safety-related applications, augmenting a realistic human-machine interaction view provides drivers with a higher perceived level of safety than with a conventional HMI visualization style, despite the higher visual complexity. In addition, one study found that the more complex the safety recommendation that the human-machine interaction has to communicate, the more the drivers perceive an augmented realistic HMI visualization as a valuable support [24]. In addition, an AR-based navigation system causes drivers to spend more attention looking ahead on the road than does a non-AR system [25]. During a non-augmented traffic situation, information such as lane markings and road signs has proven to be very useful for indicating static traffic aspects. In contrast, AR can be used to provide dynamic markings that can adapt to changing contextual traffic situations to better regulate a driver's traffic behaviors [22]. In this way, automotive AR can potentially enhance a driver's experience by providing a new visual modality that can overlay information over the driver's field of view [26]. 3. Relationship between Driver Characteristics, Risk Perception, and Driving Behaviors In relation to driving behavior, risk perception refers to "the subjective experience of risk in potential traffic hazards [27]." Inexperienced drivers who have low levels of altruism and high levels of anxiety and anger tend to underestimate objective risk factors in a presented traffic environment. This is related to risky behaviors such as speeding [28]. In addition, drivers with low adaptive capability in a physical/social environment and high levels of anxiety have experienced a relatively greater number of traffic accidents [29]. For many drivers, trying to follow digital route guidance in an unfamiliar place during heavy rush-hour traffic can cause sensory overload and high anxiety, and can compromise safety. A new HUD technology is aimed at mitigating and even preventing such scenarios [30]. If we focus on driver behaviors, accidents are likely to occur when the drivers make mistakes or drive improperly when misjudging driving risks. Such driving behaviors can be caused by four main factors: speeding, drunk driving, fatigue, or distractions [31]. However, the driver response time is not only affected by the traffic situation that the drivers experience, but also by their age, psychological state, personality, and attitude [32]. One's state of anxiety is interactively determined by their personal traits and situational stress [33]. It has often been found that anxiety impairs performance, especially when the task being performed is complex and demands attention [34]. In addition, stress directly elicited by driving can be used to predict lapses and violations within a driving environment [35]. Moreover, anxiety and diverse factors such as a lack of experience, emotions, day-dreaming, tension, fatigue, lack of sleep, weather changes, frost, heat, biorhythms, aggression, a bad mood, conflicts, stress (workload), alcohol, drugs, smoking, and pain affect one's level of concentration [36]. However, anxiety is not necessarily bad. Anxious personality types have shown faster reaction times to threatening situations, and at the same time, can quickly process more threatening circumstances [37]. Therefore, this study aims to determine the effect of an in-vehicle AR-HUD system on a driver's response behavior and psychological characteristics. Additionally, we tried to confirm the difference in effectiveness of an in-vehicle AR-HUD system between normal and risky drivers. II. Method 1. Participants Thirty male drivers and one female driver participated in this experimental study. These participants were randomly assigned into two groups: an in-vehicle AR-HUD system user group (16 participants) and a control group (15 participants). However, we excluded the driver data of both the female (in-vehicle AR-HUD system user group) and two of the male (control group) drivers from the final statistical analysis because the female driver had less than one year of driving experience, and the two male drivers showed a response of less than 50% to the total experimental stimuli. The average age and driving experience of the participants are shown in Table 1. There was no statistically significant difference between the two experimental groups. Table 1. Age and driving experience of the participants (standard deviation). AR-HUD group Control group Age 35.27 (6.76) 39.08 (6.46) Driving experience 11.07 (5.90) 14.85 (6.00) 2. Experiment A. Experimental Stimulus and Test-Bed We conducted our experiment using a video clip collected from real traffic environments and a real test vehicle. The data were collected in Yuseong-gu, Daejeon, Korea, as shown in Fig. 1. Figure 1Open in figure viewerPowerPoint Driving scenario path. The playback time of the experimental video clip was about 6 min. The video clip contains experimental stimuli including the risk of a rear-end collision with a preceding vehicle, the risk of a collision with pedestrians on a crosswalk, the risk of a collision with a cutoff vehicle, and a normal driving situation that was not controlled by the researchers, as shown in Table 2. Table 2. Driving events and path positions. Event Position Times Preceding vehicle E1-E2-E5-E6 4 Pedestrian E3-E7 2 Cutoff vehicle E4-E8 2 However, for the risk of a collision with a cutoff vehicle (E4, E8), it seems that a dangerous situation was not properly induced to the participants. In addition, the characteristics of the preceding vehicle stimulus and the pedestrian stimulus remained from the beginning until the end when the drivers completely responded. However, in this experiment, the cutoff vehicle stimulus had both its own characteristics and those of the preceding vehicle stimuli. In addition, the cutoff vehicle initially operated as the cutoff vehicle near an adjacent lane, and after it moved into the same lane, its characteristics changed to those of a preceding vehicle. We found that the response time for the cutoff vehicle stimuli of the participants had two mixed response times for the cutoff and preceding vehicles. The response behavior of the participant was different from our initial experimental purpose of providing cutoff vehicle stimulus. We therefore determined that the mixed responses of the participants were not an accurate response to an accurate stimulus. According to this judgment, we excluded data on the cutoff vehicle stimuli from our final analysis. The drivers from the in-vehicle AR-HUD system using group watched a video clip made by simply adding AR information to an original clip. Danger information regarding detected vehicles and pedestrians was displayed based on the TTC levels as AR technology through an HUD, as shown in Fig. 2. The original video clip was used for the control group. Figure 2Open in figure viewerPowerPoint AR-HUD system presentation information: risk of (a) rear-end collision with a preceding vehicle, (b) pedestrian on a crosswalk, and (c) collision with a cutoff vehicle. In addition, we made a training video clip that allowed portions of the video that did not reflect the intention of our experiment to be cut out. The purpose of the training video clip was to make sure that the participants became familiar with the driving speed and distance presented through a kiosk-HUD device. We conducted the experiment on an indoor test bed that included kiosk HUD devices. The indoor-test bed consisted of a 180-in screen, a beam projector, an HUD device, a life-sized windshield, and a PC for collecting the driver response data and for running the integrated SW, as shown in Fig. 3. Figure 3Open in figure viewerPowerPoint Indoor location test-bed including a kiosk HUD. B. Driving Behaviors: Driver Response Times We collected the response times of the participants under both precautionary and risky situations, including vehicles and pedestrians suddenly appearing in front of them. To do so, we explained to the participants that they should press the spacebar of the keyboard when they wanted to brake to avoid a collision with a vehicle or pedestrian. In addition, we calculated their response time under both the objective precautionary and risky situations during which AR information was displayed in a different color according to each TTC level. C. Driving Behavior Determinants A driving behavior determinants (DBD) questionnaire was used in this study to collect the psychological information of all of the drivers, including their driving characteristics and attitudes. The questionnaire was developed based on the risky behaviors, human factors, psychological characteristics, and general attitudes of Korean drivers. The DBD was divided into two levels: a DBD level and a driver's reckless driving level. The DBD level consists of five factors: Problem Evading, Benefits/Sensation Seeking, Anti-Personal Anxiety, Anti-Personal Angry, and Aggression. The meaning of each factor of the DBD level is as follows. First, a high score for Problem Evading means that the respondent has a tendency to avoid or give up rather than try to solve a problem they are facing. Second, a respondent with a high score for Benefit/Sensation Seeking has a tendency to seek a sense of thrill from dangerous behavior and to benefit from a situation in which a conflict occurs in terms of what they wish to pursue and the societal regulations that they have violated. The respondents who score high on the Anti-Personal Anxiety factor are likely to be reluctant to come forward in front of others and have difficulty in forming new relationships. The Anti-Personal Angry factor is similar to the Anti-Personal Anxiety factor in terms of the inadaptability of interpersonal relationships; however, a distinction in this factor is the expression of negative aspects in an aggressive form. Finally, the Aggression factor has intrinsic characteristics and does not have a specific cause or object. The range of reliability of the DBD level showed a Cronbach's α value of 0.71 to 0.86. In addition, the level uses a seven point Likert scale (strongly disagree to strongly agree) [38]. In addition, the reckless driving level consists of Speeding, Inexperienced Coping, Wild Driving, Drunken Driving, and Distraction. The results of the correlation between two levels indicate that an inappropriate level of DBD is highly correlated with dangerous driving behaviors and the strong possibility of a traffic accident [38]. Based on this, the authors proposed a discriminant function that can be used to discriminate drivers according to the DBD level and predict their reckless driving behaviors through a standardization procedure [39]. Therefore, in this study, we collected the DBD level data of the participants simply to determine whether they are a risky driver or not. D. Experimental Procedure First, we described the purpose of our study to the participants, and then played them the training video clip. The participants watched the training video clip until they became familiar with the speed and distance of the video scene, which was presented using the indoor kiosk HUD test bed. Next, the participants were instructed regarding the response form and allowed to ask questions about the experiment. We provided a range of answers to avoid affecting their response behaviors. When the above procedure was finished, the participants conducted the main experiment. Finally, they filled in a questionnaire set composed of the DBD (37 questions) and basic information (age, driving experience, and so on). E. Data Analysis A data analysis of this study was conducted using data collected from 28 of the participants, with data from both the female (in-vehicle AR-HUD system using group) and two male (control group) drivers described earlier being excluded. In addition, the cutoff vehicle stimuli were also excluded from the data analysis because the average response time and standard deviation of the participants were higher than expected, and the situation seemed to not be particularly risky to the participants. We used the statistical program, SPSS 14.0, to confirm the difference in the response time and psychological characteristics of the participants owing to the use of the in-vehicle AR-HUD system. In addition, the relationship between the response time and psychological characteristics was analyzed through a correlation and regression analysis, and we conducted an analysis of variance (ANOVA) to confirm the difference between the driving conditions and experimental groups. III. Results We conducted this study to confirm the effects of the in-vehicle AR-HUD system usage on a driver's response behavior and psychological characteristics. The average response time and average DBD score of the participants, including the five factors, are presented in Table 3. The results of the ANOVA indicate that there was no statistically significant difference between the two experimental groups. Table 3. Response time and DBD of the participants (standard deviation). AR-HUD Control Objective risky situation Preceding vehicle 0.41 (1.26) 0.58 (0.90) Pedestrian −0.60 (0.66) −0.76 (0.66) Objective precautionary situation Preceding vehicle 1.94 (1.31) 2.11 (0.96) Pedestrian 0.05 (0.49) −0.05 (0.49) DBD 3.30 (0.68) 3.07 (0.78) Factor 1. Problem Evading 2.83 (0.79) 2.60 (1.01) Factor 2. Benefits/Sensation Seeking 3.04 (1.24) 2.61 (0.85) Factor 3. Anti-Personal Anxiety 3.53 (0.86) 3.50 (1.10) Factor 4. Anti-Personal Angry 3.94 (0.70) 3.68 (1.00) Factor 5. Aggression 3.37 (1.10) 3.24 (1.05) A. Results Based on the Relationship between Driver's Response Time and DBD We conducted a correlation analysis to confirm the relationship between the participant's response time and their psychological information for both the in-vehicle AR-HUD system using group and the control group, which are shown in Table 4. Table 4. Relationship between response time and DBD for both the AR-HUD using group and the control group. Conditions DBD Problem Evading Benefits/Sensation Seeking Anti-Personal Anxiety Anti-Personal Angry Aggression AR-HUD R: vehicle 0.41 0.42 0.51 −0.04 −0.09 0.40 R: pedestrian 0.07 0.10 0.11 −0.30 0.01 0.29 P: vehicle 0.43 0.43 0.54* −0.01 −0.14 0.41 P: pedestrian 0.40 0.39 0.40 0.15 0.03 0.31 Control R: vehicle −0.27 −0.27 0.07 −0.59* −0.20 0.04 R: pedestrian −0.40 −0.51 0.12 −0.76** −0.19 −0.06 P: vehicle −0.28 −0.24 0.02 −0.58* −0.20 0.01 P: pedestrian −0.61* −0.42 −0.17 −0.79** −0.57* −0.39 R: risky situation P: precautionary situation * p < 0.05, ** p < 0.01 As a result, during a precautionary situation for the in-vehicle AR-HUD system using group, only the Benefit/Sensation Seeking factor was positively correlated with the response time for the preceding vehicle stimulus (, ). The Anti-Personal Anxiety factor was only negatively correlated with the response time to the preceding vehicle and the pedestrian stimulus for the control group (a risky situation for a vehicle, and , and for a pedestrian, and ; and a precautionary situation for a vehicle, and , and for a pedestrian, and ). In addition, both the DBD level and Anti-Personal Angry factor were only negatively correlated with the response time for the pedestrian stimulus (, ; and , , respectively). Further, we also conducted a regression analysis to confirm how the psychological characteristics of the driver affected their response behavior. As a result, we found that only the DBD level of the drivers had a significant impact on their response time for the control group (, , and adjusted ). In detail, among the five factors, only Anti-Personal Anxiety had a significant impact on the response time (, , adjusted , , and ). B. Results of Response Behavior between Risky and Normal Drivers We tried to confirm the difference in effectiveness of the in-vehicle AR-HUD system between normal and risky drivers. To do so, we divided the participants into two groups, a normal driver group and a risky driver group, for both the in-vehicle AR-HUD group and the control group. We also applied the discriminant function proposed in [39]. The data on the drivers' response time and DBD levels are shown in Table 5. We conducted an ANOVA based on these data. Table 5. Driver response time and DBD for risky and normal driver groups. Normal drivers Risky drivers AR-HUD() Control() AR-HUD() Control() R: vehicle −0.30 (1.22) 0.47 (0.97) 1.22 (0.71) 0.75 (0.85) R: pedestrian −0.81 (0.77) −0.86 (0.71) −0.36 (0.45) −0.59 (0.61) P: vehicle 1.16 (1.24) 2.05 (0.98) 2.83 (0.71) 2.22 (1.03) P: pedestrian −0.13 (0.50) 0.00 (0.61) 0.26 (0.40) −0.15 (0.25) DBD 2.92 (0.46) 2.64 (0.57) 3.73 (0.64) 3.77 (0.50) Factor 1 2.61 (0.85) 2.20 (0.62) 3.07 (0.69) 3.23 (1.27) Factor 2 2.22 (0.55) 2.11 (0.55) 3.98 (1.15) 3.40 (0.60) Factor 3 3.56 (0.73) 3.31 (1.25) 3.48 (1.06) 3.80 (0.84) Factor 4 3.83 (0.69) 3.15 (0.86) 4.07 (0.75) 4.53 (0.46) Factor 5 2.60 (0.78) 2.60 (0.75) 4.24 (0.66) 4.27 (0.45) R: risky situation P: precautionary situation Factor 1: Problem Evading Factor 2: Benefits/Sensation Seeking Factor 3: Anti-Personal Anxiety Factor 4: Anti-Personal Angry Factor 5: Aggression As a result, the risky drivers showed a longer response time to the preceding vehicle than the normal drivers regardless of the use of the in-vehicle AR-HUD system (risky situation, and ; and precautionary situation, and ). However, the response time to the pedestrians showed no significant difference between the normal and risky driver groups. An ANOVA was further conducted to confirm the difference in response time between the in-vehicle AR-HUD system using group and the control group for both normal and risky drivers. For the normal drivers, there was no statistically significant difference between the in-vehicle AR-HUD system using group and the control group. However, the risky drivers from only the in-vehicle AR-HUD system using group had a statistically significant longer response time to the preceding vehicle under both risky and precautionary situations ( and ; and , respectively). In addition, as with the other pedestrian results, the response times to the pedestrians also showed no significant difference between the normal and risky drivers. IV. Discussion We conducted this experimental study to determine the effectiveness of the in-vehicle AR-HUD system usage on a driver's response behavior and psychological characteristics. In addition, we tried to confirm the difference in effectiveness between normal and risky drivers. To do so, we measured the response time and collected the DBD information on the 28 remaining male drivers. One notable result of this study is regarding the pedestrian stimulus. The response times of the drivers who participated in the control group were faster than those tested for the in-vehicle AR-HUD system using group, although the difference was not statistically significant. This indicates that there is a possibility that the drivers relied on the in-vehicle AR-HUD system to provide AR information regarding precautionary and risky situations based on the TTC levels. In other words, the drivers might have conservatively determined the risk of a preceding vehicle or pedestrian when they did not receive additional information. Therefore, a further study will need to be conducted on this topic. Nonetheless, the risk determination regarding pedestrians was not affected by the in-vehicle AR-HUD system usage under any of the conditions applied. This indicates that the drivers seemed to feel that the pedestrians were less dangerous than the preceding vehicles owing to their particular characteristics. Another possibility is that the pedestrians did not stimulate the drivers as much as the preceding vehicles. This issue requires confirmation through a further study. In addition, for a precautionary situation, for the in-vehicle AR-HUD system using group, the drivers with high Benefit/Sensation Seeking characteristics responded more slowly to the preceding vehicle stimulus. This result seems to be due to the fo