Evaluation of Drivers’ Affectability and Satisfaction with Black Spots Warning Application

Since a significant percentage of crashes occur at black spots, different methods have been proposed to prioritize the modification of these spots and prevent crashes. One of these prevention methods in transportation is the hazard warning systems. The purpose of this study is to evaluate the satisfaction of drivers and their affectability from a map-based warning application is evaluated. To this end, black spots were identified on one of the two-way two-lane highways in the North-West of Iran and 32 male drivers were tested in the intervention group (warning state) and the control group (non-warning state). The evaluation of the warning application was done in two steps. In the first stage, drivers’ affectability between the two groups was compared, where average speed and number of speed limit violations were studied in warning and non-warning states. In the second stage, drivers' satisfaction with application features was examined using questionnaires. The findings showed that the difference in mean speeds at black spots between warning and non-warning states was significant with 95% confidence level and the use of warning application was effective in reducing the number of drivers with speed limit violations at black spots. Most drivers were highly content with the warning from car speakers, advisory warnings and warning distance from black spot, and did not have enough satisfaction with visual warnings, the application installation procedure, and warnings from smartphone speakers. Additionally, the results of the questionnaire revealed that not only warnings did not cause distraction for the drivers, they were effective in increasing their caution. These findings can be used to eliminate the shortcomings of the hazard warning application.


Introduction
Traffic crashes are inevitable accidents that occur due to various reasons. In addition to being a threat to society's safety and health, these crashes entail a high amount of costs, such as financial losses, delays in traffic and depletion of national resources. According to the World Health Organization, the damage inflicted on developing countries because of these crashes equals more than 1% of their gross national product (GNP); this value is more than 3% in Iran [34]. Iran, with a population of close to 77 million people and 25,000 annual traffic-related deaths, is one of the most dangerous countries regarding traffic accidents in the world [1] and, as a result, Iranian road safety specialists have recently conducted studies on road and vehicle safety performance [9,17,23,37].
Since the majority of crashes occur at black spots, numerous methods have been employed to prioritize the modification of these spots and prevent crashes [16,35]. One of these prevention methods is the hazard warning system. The driver is warned before reaching the black spot so that he/she can be more cautious to be able to prevent crashes. Some of these systems are map-based warning systems that drivers install on their smartphones, which warn the drivers before reaching specific spots [22,33,38]. Other systems can detect obstacles in front of and at the side of the vehicle and also while exiting the lane, using radar technology or vehicle-mounted cameras, and warn the driver [4,5,10,25,27]. The most commonly used systems include Forward Collision Warning (FCW), Side Blind Zone Alert (SBZA) and Lane Departure Warning (LDW).
In recent years, many studies have been conducted on warning systems and their impact on driver performance. Various criteria have been considered for measuring drivers' performance including speed, driver behavior and braking reaction time. Some studies have also focused on the acceptance of advanced driver assistance systems (ADAS) and driver satisfaction with them. The studies on the affectability and satisfaction of warning systems are presented in the following.
The effect of hazard warning systems on the performance of drivers in two test groups has been evaluated by driving simulators and tests on real roads. Studies conducted in driving simulators have reported reduced rate of collision and reaction time when the warning system was active [3,12,15,18,28,32]. On the other hand, research on real roads has shown lower speed and faster braking by the drivers at the time of warning announcements [2,6,21,22,38].
A group of researchers took into account factors such as usefulness, ease of use, efficiency, affordability and social acceptance for assessing drivers' acceptance and satisfaction with the use of ADAS, and others prepared checklists to compare drivers' satisfaction with new assistant programs and other vehicle systems [14,20,29]. A study in 2008 evaluated drivers' acceptance of various audio and visual warnings for front and side collision warning systems. In their work, the timing of warnings (three different times) and the sound of alarms with and without separate speakers were compared and driver satisfaction and affectability were reported [24].
Studies have also compared different individuals in the acceptance and satisfaction of ADAS. The results of a study showed that older drivers are more satisfied with these systems than young drivers, and are willing to hear warning messages [8,19,26,31]. A study in 2013 evaluated the satisfaction of different drivers with warning systems at intersections and the results showed that all the older drivers were satisfied with the warnings and 60% of the novice drivers did not have enough satisfaction for reasons such as anxiety and lack of proper response upon warning, [36]. A study in 2000 also found that using ADAS was more satisfactory for drivers in foggy conditions compared to normal conditions [30]. Another study focused on the effect of the warning system on young and novice drivers reported that the system improved their turning and inter-lane movements but increased performance near the black spots [11].
Cicchino et al. investigated the effect of audio and visual warnings on reduction of crash rates and reported a 20% decrease in rear-end crashes as a result of these systems [7]. In another study, the effect of audio warning on the drivers' braking reaction was investigated and it was seen that the reaction time of younger drivers is lower than that of older drivers in the speed range of 40 to 60 mph [13].
The black spots warning systems are categorized under the map-based systems that identify the locations on the road that have higher chances of crash and warn the driver before reaching them. While the other warning systems use radar technology or vehicle-mounted cameras to warn about the absolute danger, black spots warning systems announce the high potential of the upcoming danger. Therefore, the investigation of drivers' satisfaction and affectability can be of great importance for this application.

Research Methodology
Firstly, some explanation about the black spots warning application and how it works on rural roads is presented. Then, evaluation of drivers' satisfaction with the different features of the application and drivers' affectability are investigated. The flowchart of the study is shown in Figure 1. The study was performed in five stages. In the first two, crash data were acquired to identify the black spots (the method is explained in the next section). For the third stage, the program for the black spots warning system was written and designed for an Android-running environment. Ultimately, the last two stages involved the assessment of effectiveness and drivers' satisfaction of the program by testing it while driving in a two-way road in Iran.

Black Spots Warning Application Description
Using the data corresponding to the crashes recorded over the past 5 years (2012-2016), black spots were identified on a two-way highway in the North-West of Iran. This highway is 91 km long, and is one of the roads with the highest crash rates in Iran. Crash frequency (CF), crash rate (CR) and kernel density estimation (KDE) were applied to identify black spots. Eight black spots were selected for this highway, and traffic police experts conducted field analyses to identify the precise coordinates of these eight spots. In the end, the coordinates were introduced into the warning system application. Figure 2 shows the location of the black spots.

Figure 2. The location of black spots on Tabriz-Ahar highway
Since in previous studies [38,6,15], warnings in both audio and visual formats were reported to be effective on drivers' affectability, this application also used both methods. The audio warning comprised of the danger type alarm and the visual warning involved displaying a danger sign on the monitor. Fifteen seconds or 600 meters have been suggested as the appropriate time and distance before the danger point to alert the driver [33,18]. In this study, warning at 600 meters to the black spot was implemented. Figure 3 shows the application environment and the danger sign visualization.

Evaluation of Black Spot Warning Application
The evaluation of the application was done in two steps. The first step was the comparison of drivers' affectability in warning and non-warning states. 32 male drivers were randomly selected and tested in the intervention group (warning state) and the control group (non-warning state). Within two months of the test, 160 and 96 speed data points were recorded for the intervention and control groups at the black spots, respectively. Figure 4 illustrates how a driver uses the warning application by a smartphone.

Figure 4. Representation of how drivers use the warning application on the smartphone
The recording of instantaneous and mean speeds using the smartphone GPS was done by a computer code in the application, which could be used to check the speed variations at the black spots with and without warning. Figure 5 shows the speed recording sample in the software memory.

Figure 5. Part of the speed-time diagram for a driver
For the first stage of the assessment, the mean speed in the black spot areas and the risk of exceeding the speed limit at these spots were selected as the evaluation criteria [22,33]. The first speed criterion is the average speed that is recorded with and without warning in the target black spot, and the significance of the difference between these two speeds is evaluated with 95% level of confidence. Equation 1 shows the standard deviation, and Equation 2 indicates the equivalent standard normal distribution value of the observed difference in speeds. The second speed criterion is the relative risk of exceeding the speed limit with and without warning in the target black spot. This criterion can also accurately evaluate the impact of a hazard warning system. Equation 3 shows the relative risk relation used in this study, where the intervention group is the one for which the hazard warning was activated, while the control group represents cases without the warning.
Where =standard deviation; =speed deviation in black spots in the control group; =speed deviation in black spots in the intervention group; N= the sample size in each group; ̅ =the average mean speed in each group; = the equivalent standard normal distribution value of observed difference in speed.
For the second step, a questionnaire for application satisfaction was used that included questions about the features of the warning application ( Table 1). The variables of the questionnaire were warning sound, warning image, warning distance and accessibility, and satisfaction with the application. Some of the variables were in the form of one question, while others involved multiple questions answered by the drivers. The total number of questions was eleven. At the end, drivers were asked to provide suggestions for improving the application. Since this questionnaire was merely a survey for the strengths and weaknesses of the warning application, the questionnaire was analyzed by simple descriptive analysis, and the frequency of answers and percentage of satisfaction with each feature of the application were reported. The volume of sound coming from smartphone speakers is satisfactory. 3 The volume of sound coming from car speakers is satisfactory.

Warning image
The danger sign in the visual warning are appropriate.

Warning distance
The distance from warning announcement to the black spot is appropriate. 6

Accessibility, installation and application
Application installation is easy. 7 The screen-turned-off mode and only using warning sound is appropriate.

8
Online mode is available for you.

Satisfaction
Warnings increase caution in your driving.

11
I am generally satisfied with the warning application.

Results
The evaluation of the black spot warning application was done in two stages. In the first stage, the drivers' affectability from the warnings was studied. For this purpose, speed criteria were analyzed in the intervention group (active warning) and control group (inactive warning). In the intervention group, 160 speed data points with an average of 64.5 km/h and a standard deviation of 8.8 km/h were recorded; while in the control group, 96 speed data points with an average of 68.2 km/h and a standard deviation of 9.3 km/h were recorded at the black spots. The standard normal distribution value was obtained to be ( = 3.14 ≥ 1.96), which confirmed the significant difference in mean speeds between these two groups. Another speed criterion in the first stage of evaluation was the risk of exceeding the speed limit at black spots, which was investigated in the intervention group (warning state) and control group (non-warning state). Among the 160 recorded speeds of the intervention group, eighteen were above the speed limit; while in the control group, out of 96 recorded speeds, twenty exceeded the speed limit. The relative risk of this criterion was 0.54 (confidence interval 0.3 to 0.96), which indicates that danger warning in black spots had a significant effect on reducing the number of speed limit violations.
In the second step of evaluation, in order to determine the strengths and weaknesses of the application, the responses given for each feature (warning sound, warning image, accessibility, installation and application, warning distance and overall satisfaction) were separately analyzed. The results are presented as descriptive charts.
Warning sound: This variable is an important and effective feature of hazard warnings. Due to the high effectiveness of advisory warnings in previous studies [18,22,24], this type of warning was used for the current application. In advisory warnings, depending on the type of danger, an appropriate warning is given to the driver, and warnings are not merely simple alarms. According to the responses regarding the satisfaction of drivers with advisory warnings, the average score obtained for this question was 3.95 out of 5, which indicates a high level of satisfaction with this type of warning. Figure 6 shows that nearly 75% of drivers had high or very high satisfaction with this type of warning.

Figure 6. Satisfaction of warning sound sources and advisory warning
For sound output, two questions were designed about the origin of the sound (phone or car speakers). The satisfaction scores of each of them are reported in Figure 5. None of the 10 drivers who heard the warning from their phone indicated high satisfaction, and about 80% of these drivers had a satisfaction of lower than average. On the other hand, the drivers who heard the warning through their car speakers, reported considerable satisfaction. None of these drivers chose a satisfaction score lower than average, and about 80% of them were very satisfied with this type of playback.

Warning image:
In some studies, drivers have been warned of the danger using warning sounds, without visual warning. But in many other studies, the impact of visual warnings has also been taken into consideration. In the current application, the blinking warning sign appears on the screen before the driver reaches the black spot, and remains there until he/she passes the black spot area. Question 4 in the satisfaction questionnaire was related to the warning sign performance, which garnered only 5% high satisfaction from the drivers. More than 90% of drivers reported moderate and lower than moderate satisfaction with the warning image. Figure 7 shows the results of the visual warning satisfaction.

Q4: Visual warning
Warning distance: This variable is one of the features of the warning application for which, researchers have proposed time and spatial distances. In this study, the distance of 600 meters to the black spot was considered. The number of drivers tested for this feature was 20, among which 80% had a high and very high degree of satisfaction with this feature. Figure 8 shows the results of the satisfaction of the warning distance.
Installation, accessibility and application of the software: Questions about this feature were asked from all 32 drivers. In these questions, the simplicity of application installation was first asked, and the next two questions were about the application of the warning in the screen-off mode and the online application of the software. Figure 9 shows the results of driver responses to these three questions. Nearly 80% of the drivers had moderate and lower than moderate satisfaction with the installation procedure of the application. Regarding the phone-screen-off mode and the audio warning, about 50% of the drivers were content to add this feature to the application, but others were not satisfied to support this decision. For usage of the online mode, about 75% of drivers did not have trouble using the Internet on the road. In this part of the questionnaire, three questions were asked from 20 drivers for whom the warning state was active. In these questions, increased caution, increased distraction, and overall satisfaction with the intended application were considered. The results showed that 70% of the drivers considered the warnings to be effective in increasing their caution. Nearly 75% of the drivers did not consider warnings as a source of distraction, and 60% of drivers reported overall satisfaction with the warning application. Figure 10 shows the results of the drivers' satisfaction with the warning application.

Discussion
The findings of similar studies indicate the higher impact of audio warnings on reduction of reaction time and average speed of drivers compared to visual warning [3,11,13], which is confirmed in the present study, as well. Among the studies that investigate the effect of warning at black spots, works by Ryder et al. 2007 and Zhang et al 2009 are notable [22,38]. The study by Ryder et al. showed that the warnings were effective in reducing the speed of drivers before reaching the black spots, while Zhang et al. reported that the warnings increased the caution of drivers approaching arched bridges. Similarly, the currents study resulted in reduced vehicle speeds and lower number of speed limit violations at black spots.
Regarding the assessment of application features, advisory warning (announcing the danger type) resulted in significant satisfaction by drivers, as did playing the warning from car speakers. The sound of the warning from smartphones was hardly heard due to driving on rural roads and high speeds. For the warning image, the results of the questionnaire showed that the danger sign was not visible because of its small size, and it was decided to display a larger danger sign on the screen. For warning distance, the choice of spatial distance was not a problem for drivers, but according to the drivers' suggestions and for the compatibility of warning distance with speed of drivers, the time interval mode was considered for the application, and therefore the timing of the warning emergence would depend on the vehicle speed, and would not occur at a specific distance from the black spot.
Drivers did not report enough satisfaction with the application installation. The discontent was due to the fact that the map and application files were separate, and arranging the map in the files of the smartphone was time-consuming. The case for the phone-screen-off mode was also present in drivers' suggestions, which was recommended to reduce battery consumption for long distances. The proposal for online application of the software and providing more services, such as real-time danger warnings, was also pleasing to most drivers. Satisfaction results showed that this application increases drivers' caution in black spots without causing distraction.

Conclusion
To assess the affectability of the warning on the performance of drivers, 32 male drivers were randomly selected and put in intervention and control groups. The results of the evaluation showed that the difference in mean speeds at Q11: General satisfaction of the application black spots with and without warning was significant with a 95 % confidence level. Additionally, the warning application was effective in reducing the number of drivers violating the speed limit at black spots.
To evaluate the application in future studies, the effectiveness of warning on drivers' speed can be assessed and analyzed while considering behavioural and demographic characteristics of drivers, weather conditions and different types of vehicles with larger sample sizes, and the results of changes in speed can be reported for different groups. This assessment can also be done at black spots with different geometric characteristics, and therefore the results of different black spots can be compared.
The results of speed criteria for different drivers can vary in different environmental and climatic conditions. For future studies, it is recommended that speed criteria should be evaluated and analyzed in situations like day and night, favourable and unfavourable weather conditions and peak traffic hours. In the case of significant changes in speed criteria, different approaches for improving the warning application should be suggested.
This application can be used in any country after reviewing and correcting its weaknesses according to the country's road crash data. Moreover, with the participation of the car industry, it can be installed in cars or smartphones so that everyone can have easy access to it, and use it to reduce crashes at black spots.