Predicting crash-relevant violations at stop sign–controlled intersections for the development of an intersection driver assistance system

ABSTRACT Objective: Intersection crashes resulted in over 5,000 fatalities in the United States in 2014. Intersection Advanced Driver Assistance Systems (I-ADAS) are active safety systems that seek to help drivers safely traverse intersections. I-ADAS uses onboard sensors to detect oncoming vehicles and, in the event of an imminent crash, can either alert the driver or take autonomous evasive action. The objective of this study was to develop and evaluate a predictive model for detecting whether a stop sign violation was imminent. Methods: Passenger vehicle intersection approaches were extracted from a data set of typical driver behavior (100-Car Naturalistic Driving Study) and violations (event data recorders downloaded from real-world crashes) and were assigned weighting factors based on real-world frequency. A k-fold cross-validation procedure was then used to develop and evaluate 3 hypothetical stop sign warning algorithms (i.e., early, intermediate, and delayed) for detecting an impending violation during the intersection approach. Violation detection models were developed using logistic regression models that evaluate likelihood of a violation at various locations along the intersection approach. Two potential indicators of driver intent to stop—that is, required deceleration parameter (RDP) and brake application—were used to develop the predictive models. The earliest violation detection opportunity was then evaluated for each detection algorithm in order to (1) evaluate the violation detection accuracy and (2) compare braking demand versus maximum braking capabilities. Results: A total of 38 violating and 658 nonviolating approaches were used in the analysis. All 3 algorithms were able to detect a violation at some point during the intersection approach. The early detection algorithm, as designed, was able to detect violations earlier than all other algorithms during the intersection approach but gave false alarms for 22.3% of approaches. In contrast, the delayed detection algorithm sacrificed some time for detecting violations but was able to substantially reduce false alarms to only 3.3% of all nonviolating approaches. Given good surface conditions (maximum braking capabilities = 0.8 g) and maximum effort, most drivers (55.3–71.1%) would be able to stop the vehicle regardless of the detection algorithm. However, given poor surface conditions (maximum braking capabilities = 0.4 g), few drivers (10.5–26.3%) would be able to stop the vehicle. Automatic emergency braking (AEB) would allow for early braking prior to driver reaction. If equipped with an AEB system, the results suggest that, even for the poor surface conditions scenario, over one half (55.3–65.8%) of the vehicles could have been stopped. Conclusions: This study demonstrates the potential of I-ADAS to incorporate a stop sign violation detection algorithm. Repeating the analysis on a larger, more extensive data set will allow for the development of a more comprehensive algorithm to further validate the findings.


Introduction
Accounting for one fifth of all crashes and one sixth of all fatal crashes, intersection crashes are among the most frequent and lethal crash modes in the United States (Kusano and Gabler 2013). Intersection Advanced Driver Assistance Systems (I-ADAS) are emerging vehicle-based active safety systems that aim to mitigate/prevent intersection-related crashes. These systems can alert the driver of an impending collision and in some systems autonomously take evasive action.
One I-ADAS function will be to detect approaching vehicles using onboard sensors, such as radar or lidar. Current technology in the vehicle fleet uses forward-facing radar for cross-traffic object detection (Mercedes-Benz USA 2015; Volvo Car Corporation 2015), but future systems may use side-facing sensors to detect oncoming vehicles from greater distances (Scanlon, Kusano, Sherony, and Gabler. 2015;Scanlon et al. 2016). However, this strategy of detecting oncoming vehicles will be less effective if vehicles fail to yield prior to entering the intersection because less time will be available to detect and avoid an imminent crash. Failure to yield at an intersection is a substantial priority. For straight crossing path (SCP) intersection crashes in the United States, approximately one third (Retting et al. 2003;Scanlon, Kusano, Sherony, and Gabler 2015) of stop sign-controlled crashes and nearly all (95%; Scanlon, Kusano, Sherony, and Gabler (2015) of signalized intersection crashes involved a vehicle failing to yield for the traffic control device.
An alternative strategy may be to alert the driver of an impending traffic signal violation earlier in the intersection approach, while the driver still has time to yield for the traffic control device. Previous work estimates that 90% of drivers who run stop signs failed to detect the traffic control device during the approach (Tijerina et al. 1994). The first challenge is to detect an upcoming stop sign or signalized intersection. Vehicleto-infrastructure communication of vehicle position is possible (Papadimitratos et al. 2009) but may be prohibitively expensive in the case of stop sign-controlled intersections. Current vehicle-based technologies could, however, make this approach feasible. For example, advances in automated traffic sign detection utilize vision-based technology to detect traffic control signs, such as stop signs (Akatsuka and Imai 1987;Bahlmann et al. 2005;De La Escalera et al. 1997;Franke et al. 1997;Garcia-Garrido et al. 2006). Another approach is to couple Global Positioning System (GPS) traces with map data for detecting an upcoming intersection (Mardirossian 1999;Pierowicz et al. 2000;Yamaki and Nishizaka 2004).
There are 2 competing priorities when designing an I-ADAS that accurately notifies a driver of an impending violation. First, the system should aim to maximize the number of correctly predicted violations or the true-positive proportion. The truepositive proportion is the ratio of correctly identified violations to the total number of violations. However, when maximizing correctly identified violations, there is a tradeoff in the number of false alarms that might be delivered to the driver or the falsepositive proportion. The false-positive proportion is the ratio of nonviolations incorrectly identified as violations to the total number of nonviolations. Accordingly, the second priority when designing these systems should be to limit false-positive warnings. Maintaining this balance of identifying violations while not delivering false alarms is important for implementation of a driver warning system. Drivers who receive too many falsepositive alerts could become annoyed and as a result become less reactive when a true alert is given (Dingus et al. 1998;Neale and Dingus 2006) or turn off these alarms altogether (Bliss and Acton 2003). Additionally, for drivers to trust in the warning system, the driver expects to be warned early enough to be able to stop the vehicle (Abe and Richardson 2006).
The objective of this study was to develop a stop sign violation detection algorithm and evaluate its ability to provide an accurate and timely detection. Three research questions were posed in this study. First, how often would drivers be alerted? Second, when would drivers be alerted? Third, could the vehicle be stopped at this earliest violation detection opportunity?

Defining a stop sign violation
By law, drivers must come to a complete stop at stop signs in the United States. Although any noncomplete stop is considered a violation from a legal standpoint, drivers frequently "roll" through stop sign-controlled intersections in real-world scenarios (Doerzaph 2007). Because this stopping behavior is so typical, warning drivers every time they perform a rolling stop could lead to driver annoyance and/or poor acceptance of the technology. In an ideal scenario, drivers would receive warnings only when they are unaware of an approaching stop sign. However, there is no perfect speed threshold that can distinguish between drivers who are aware or unaware of an imminent stop sign. An alternative approach is to select a speed threshold that is atypical during "normal" driving but commonly observed during precrash scenarios. This study defined a stop sign violation as a vehicle that traveled at a speed in excess of 20 mph (32 kph) throughout the entire intersection approach. Previous work indicates that drivers rarely cross stop bars at speeds in excess of 20 mph (0.2%; Doerzaph 2007). Using the 100-Car Naturalistic Driving data set extracted in this study, drivers were similarly found to have traveled in excess of 20 mph throughout the entire intersection approach in only 1.1% of samples. Conversely, in precrash scenarios, drivers frequently travel at speeds in excess of 20 mph. In fact, of the 90 event data recorder (EDR) records considered for this study's analysis that were taken from SCP intersection crashes, 27% of the drivers were traveling at speeds in excess of 20 mph throughout the intersection approach.

Data sources
This study used 2 data sources for generating a single data set of typical driver behavior (100-Car Naturalistic Driving Study, NDS) and violations (EDR downloads from real-world crashes). The following paragraphs detail the data sources. Case inclusion criteria can be found in the Appendix (see online supplement).
The NDS was performed in the Northern Virginia and Washington, D.C., metropolitan areas from 2001 to 2004 by equipping vehicles with a variety of measurement devices, including cameras, inertial measurement systems, GPS, and data collected from the Controller Area Network (CAN; Dingus et al. 2006). This study used previously identified intersection traversals from a prior study (Sudweeks et al. 2007). This data set of stop sign-controlled intersection traversals was found using GPS coordinates from (1) a list of intersections provided by the Virginia Department of Transportation as having elevated crash frequencies and (2) intersections frequently traversed by study participants. Each intersection approach was manually reviewed to verify the event was correctly identified from the GPS trace and to determine the driver's turning behavior, when the vehicle crossed the stop bar (time step when stop bar was last visible), and traffic/road/weather conditions. Precrash records were extracted from EDR modules downloaded from vehicles involved in SCP intersection crashes investigated as part of NASS-CDS years 2000 to 2014. NASS-CDS is compiled annually with records from 4,000-5,000 passenger vehicle crashes investigated by the NHTSA. EDRs can record precrash vehicle speed and driver braking (0 = off, 1 = on) from the CAN bus among other data elements (Gabler et al. 2008).

Overview of methods for developing and evaluating stop sign violation detection model
This study's overall strategy is depicted in Figure A1 (see online supplement). A data set of stop sign violations and nonviolations was first extracted. Vehicle positions through time during the intersection approach were reconstructed for each case. A k-fold cross-validation technique was performed to assess the potential violation detection capacity of the hypothetical stop sign warning algorithm. Using this method, the data set was divided into k-batches. One batch was withheld as a testing data set, and the remaining k − 1 batches were used as a training batch. The training data set was used to develop a stop sign warning algorithm. The developed model was then validated using the testing data set. The process was then repeated for the remaining k − 1 batches until a violation prediction was obtained for every traversal. For this study, each traversal was considered to be a separate batch.

Intersection approach reconstructions
For the 100-Car data set, vehicle speed collected from the vehicle CAN bus was used to reconstruct vehicle position throughout the intersection approach. The time step where the stop bar was last visible was assumed to be the end of the intersection approach. For the EDR data set, a previously developed protocol (Scanlon et al. 2016) was used to trace back the position of the vehicle during the intersection approach. EDR-recorded precrash speed was used to reconstruct the vehicle's position through time. In current EDRs, there is some uncertainty in the timing of precrash speed and braking records with respect to the time of impact (Scanlon, Kusano, and Gabler 2015), which was accounted for using the methods detailed in the Appendix. The scene diagram prepared by the NASS-CDS investigator was used to determine the end of the intersection approach. The stop bar was used as the end of the intersection approach if present. If no stop bar was present, the intersection boundary was used as the end of the intersection approach. Intersection boundary lines were drawn by extrapolating the road edge through the intersection.
Numerical integration was used to estimate the location of the vehicle throughout the reconstructed intersection approach up to 50 m prior to the vehicle entering the intersection. As discussed later in this article, our stop sign detection algorithm was assumed to begin monitoring for potential stop sign running at 50 m from the intersection.

Hypothetical stop sign warning algorithm
This article evaluates the potential of a hypothetical stop sign warning algorithm for an I-ADAS. A unique algorithm was developed for each training batch. The proposed system evaluates the likelihood of a stop sign violation at discrete locations along the intersection approach using a series of logistic regression models. The algorithm begins to monitor the likelihood of a violation beginning 50 m prior to intersection entry and continues to reevaluate the likelihood of a stop sign violation at intervals of 2.5 m. The hypothetical system was assumed to be deactivated if (1) the vehicle's velocity fell below 20 mph (32 kph; i.e., the vehicle was no longer "violating") or (2) the vehicle was within 10 m from entering the intersection. An earliest violation detection distance of 50 m was selected because an earlier analysis of the data set showed that most nonviolating drivers (59.7%) begin to decelerate before this distance before the intersection. The 10-m threshold helps limit false alarm warnings for drivers that have already yielded (slowed or stopped) and began accelerating into the intersection. Early analysis of the nonviolation data set showed that 98.6% of drivers began accelerating within 10 m prior to entering the intersection.
It should be noted that, in a production car, the proposed system would evaluate the likelihood of a stop sign violation continuously through time; that is, at discrete time points. If the system was modeled as assessing violation likelihood at some sampling rate, the algorithm would still need to use a relevant logistic regression model. For example, if the system reevaluated the likelihood of a violation at 18.8 m before the intersection, the system might use the logistic regression model for 17.5 or 20 m, neither of which would be perfectly appropriate. As an alternative, the algorithm developed in this study evaluates likelihood of a violation at discrete 2.5-m increments as a uniform means of assessment. For a vehicle traveling at 30 mph, the system would reevaluate likelihood of a violation once every 0.2 s (5 Hz), and at 60 mph, the system would reevaluate likelihood of a violation once every 0.1 s (10 Hz). The detection refresh rate was considered to be realistic given the refresh rate of current collision avoidance systems (Jansson et al. 2002;Kallhammer 2006;Lindgren et al. 2009).
Three violation detection algorithms were explored in this study, including early detection, intermediate detection, and delayed detection models. The early detection algorithm detects violations at the earliest time point during the intersection approach. However, this algorithm is the most susceptible to false alarms. In contrast, the delayed detection algorithm is the least prone to false alarms but will tend to deliver later alarms. These 3 algorithms were developed by setting different cutoff violation prediction probabilities. If the probability of a violation exceeds this cutoff value at a given location, an alert would be delivered to the driver, and this location would be labeled the "earliest violation detection location. " For each model and training batch, unique violation prediction probability thresholds were selected for each location along the intersection approach. At a given location along the intersection approach, the thresholds selected for the early, intermediate, and delayed detection models were set so that 5, 1, and 0.5% of nonviolations were incorrectly identified as violations.

Logistic regression model development
Logistic regression models were developed in this study using the glm function in the R programming language (R Core Team 2015). Weighting factors were applied to weight the nonviolations and violations according to how often they might occur in the real world. Violations were assumed to represent 0.2% of intersection traversals, which was taken from the results of the previously discussed naturalistic driving study data set (Doerzaph 2007;Doerzaph et al. 2008Doerzaph et al. , 2010. Each violation in this study was given a weight of 1, and the weight for the nonviolation approaches was calculated using Eq. (1).
where Weight nonviolation is a weighting factor assigned to nonviolations, N violation is the number of violations in the data set, and N nonviolations is the number of nonviolations in the data set.
Two measures were used to model likelihood of a stop sign violation. The first measure was RDP, shown in Eq. (2). RDP is defined as the average deceleration magnitude required to stop the vehicle given some velocity and stopping distance. RDP is a useful measure for determining a vehicle's capacity to yield for a stop sign. The second measure was current brake application status, which was used to assess whether the driver was actively attempting to slow the vehicle (i.e., drivers who were braking were more likely to stop). Braking data are stored as a binary variable; that is, 1 = brake pedal depressed and 0 = driver not braking.
where RDP is the required deceleration parameter; v is the current vehicle speed; and D SL is distance from the stop location. The logistic equation was used to generate a single composite metric that can be used to calculate the probability of a stop sign violation at a given distance from the intersection. This relationship is shown in Eq. (3).
where p violation is the probability of a violation (0-1); β 0-3 are the model parameters; RDP is the required deceleration parameter; and brake1 is on/0 is off.

Evaluation of stop sign warning algorithm
The timing of the warning was evaluated using RDP after some driver reaction time, which can be calculated using Eq. (4). RDP after reaction time represents the required deceleration to stop the vehicle after some elapsed perception-reaction time (PRT). This study used a PRT of 1.0 s, which is within the bounds of typical reaction times observed during an experimental study looking at how drivers respond to an in-vehicle alert of a potential intersection violation .
where RDP (after PRT) is the required deceleration parameter after perception-reaction time; v is the current vehicle speed; D SL is the distance from the stop location; PRT is the perception-reaction time; and g is the gravitational constant.
The study evaluated the braking capacity of vehicles with respect to RDP. Maximum vehicle deceleration capacity will play an important role in the effectiveness of stop sign violation warning systems and depend on a number of factors, including vehicle characteristics, surface conditions, vehicle speed, and surface type (Baker 1975;Blau 2008;Franck and Franck 2009). Nominal maximum braking decelerations on wet and dry pavement/asphalt/concrete are 0.7 to 0.8 g, respectively (Kusano and Gabler 2012). In snow, this maximum braking deceleration decreases to 0.4 g. When evaluating RDP in this study, 3 possible surface conditions were considered; that is, poor (maximum braking = 0.4 g), moderate (maximum braking = 0.6 g), and good (maximum braking = 0.8 g) braking conditions.

Data set summary
This study's data set consisted of 696 total intersection approaches. A total of 665 of those approaches came from the 100-Car data set, and 31 approaches came from the EDR data set of violations. Of the 665 100-Car approaches, 7 of the drivers were classified as violators. In total, the final data set consisted of 38 stop sign violations. Violations were weighted using a factor of 1, whereas nonviolations were assigned a factor of 28.8 using Eq. (1). These weighting factors were applied in order to weight violations and nonviolations based on how often they occur in real-world scenarios. A description of the excluded cases can be found in the Appendix.

Statistical models using entire data set
Logistic regression models were fit to the entire data set for various distances prior to intersection entry. Model parameters for 5-m distance increments can be found in Table 1. Additional model regression parameters for 2.5-m steps can be found in Table A1 (see online supplement). The Appendix additionally includes the results from several statistical tests.

Cross-validation evaluation of hypothetical stop sign warning algorithm
The proportion of correctly identified violations and nonviolations at various locations along the intersection approach can be found in Table 2.  The results indicate that all 3 detection algorithms would be able to detect each of the violations at some point during the intersection approach. However, the number of incorrectly identified nonviolations greatly differed. For the early detection algorithm, false alarms would be delivered to 22.3% of nonviolating drivers. For the intermediate detection algorithm, false alarms would be delivered to 5.6% of nonviolating drivers. For the delayed detection algorithm, false alarms would be delivered to 3.3% of nonviolating drivers. Figure 1 provides a graphical depiction of the required braking deceleration magnitude to stop the vehicle at the earliest violation detection time points. Speed versus distance at these time points are shown for each stop sign warning algorithm, and lines are provided to show how these data points correspond to RDP before and after a 1.0 s PRT. Tabulations of the percentage of cases where braking demand (RDP) did not exceed maximum braking capabilities are shown in Table 3.
Given good surface conditions (dry asphalt), maximum braking capabilities of vehicles are around 0.8 g. For all 3 algorithms, 97.4% of violations could have been successfully detected while RDP before PRT was less than 0.8 g. However, RDP after PRT was less than 0.8 g for only 55.3-71.1% of cases. Given poor conditions (snow), maximum braking capabilities fall to 0.4 g. A total of 55.3-65.8% could have been detected while RDP before PRT was less than 0.4 g. Alternatively, RDP after PRT was less than 0.4 g for 10.5-26.3% of drivers.

Summary of results
The overall objective of this study was to evaluate how an I-ADAS warning algorithm might perform in a real-world scenario. All 3 detection algorithms were able to predict an intersection violation at some point prior to the vehicle reaching 10 m from the stopping location, but violation detection did improve with increasing proximity to intersection entry. The early detection algorithm, as designed, was able to detect imminent violations earlier in the intersection approach. However, although the false-positive proportion at each distance was designed to be around 5% for this early detection algorithm, false alarms accumulated throughout the intersection approach, and approximately 1 out of every 4 nonviolations (22.9%) was incorrectly identified. Conversely, the delayed detection algorithm sacrificed some time for alerting the driver but would deliver false alarms in only 1 out of every 30 (3.3%) nonviolations.
The ability of the vehicle to be stopped was assessed by comparing RDP before and after PRT with the maximum braking capacity of vehicles given various surface conditions. For an I-ADAS that alerts the driver of an impending violation, RDP after PRT is a useful measure of what would be required by the driver to completely stop the vehicle. Given good surface conditions (maximum braking capabilities = 0.8 g) and maximum effort, about three fourths of drivers would be able to stop the vehicle by the stop bar regardless of the detection algorithm. In addition to warning drivers, another strategy may be to implement automatic emergency braking (AEB) into I-ADAS technology, which is already present in production vehicles for frontal crash prevention  and I-ADAS (Volvo Car Corporation 2015). If vehicles were equipped with AEB, the results suggest that, given good surface conditions and maximum brake effort, nearly every vehicle could potentially stop. These results highlight the importance of an earlier reaction, which allows lower required braking decelerations, on the potential effectiveness of this technology.

Limitations
There are several limitations with regards to this study's findings. First, it should be noted that 2 unrelated data sets were used to develop this study's violation detection algorithms. The weighting factors used in this study were based on a small sample of stop sign-controlled intersections, and caution should be used if trying to extrapolate the results and models to a larger population. Additionally, each of the violations was weighted equally. In reality, some violations are more likely to occur than others.
The second major limitation of the study relates to the data sets used. The 100-Car study was conducted within a small region and included a limited number of drivers and intersection. This data set may not be representative of the entire U.S. population. A larger scale analysis is required to translate these results to additional regions and demographics. The EDR data set includes only vehicles that were involved in an "airbag deployment-level" collision. Because of this, the data set may be biased toward more severe crashes and may not translate well for (1) lower severity crashes, (2) near-crashes, and (3) violations that did result in a crash or near-crash. It should also be noted that speed and braking data were not sampled at a uniform rate across all cases. Depending on vehicle make, model, and year, EDR data were sampled at 1 to 10 Hz, and 100-Car data were sampled at 3 to 10 Hz. Additionally, EDR precrash data have a limited duration and there is uncertainty associated with the timing of the collected data. Precrash data elements are The graph on the left shows lines representing RDP at the time that the violation was detected; that is, before PRT. The graph on the right shows lines representing RDP after a .-s PRT. Each row of graphs corresponds to the results of each unique violation detection algorithm. recorded in intervals independent of one another and thus synchronicity error between the signals can be present (Wilkinson et al. 2006). In addition, wheel slip during heavy braking could cause an underestimation of vehicle speed and lead to error in the reconstructed position vs. speed profile (Ruth and Brown 2010).