Evaluation of mechanism of injury criteria for field triage of occupants involved in motor vehicle collisions

Abstract Objective The mechanism of injury (MOI) criteria assist in determining which patients are at high risk of severe injury and would benefit from direct transport to a trauma center. The goal of this study was to determine whether the prognostic performance of the Centers for Disease Control’s (CDC) MOI criteria for motor vehicle collisions (MVCs) has changed during the decade since the guidelines were approved. Secondary objectives were to evaluate the performance of these criteria for different age groups and evaluate potential criteria that are not currently in the guidelines. Methods Data were obtained from NASS and Crash Investigation Sampling System (CISS) for 2000–2009 and 2010–2019. Cases missing injury severity were excluded, and all other missing data were imputed. The outcome of interest was Injury Severity Score (ISS) ≥16. The area under the receiver operator characteristic (AUROC) and 95% confidence intervals (CIs) were obtained from 1,000 bootstrapped samples using national case weights. The AUROC for the existing CDC MOI criteria were compared between the 2 decades. The performance of the criteria was also assessed for different age groups based on accuracy, sensitivity, and specificity. Potential new criteria were then evaluated when added to the current CDC MOI criteria. Results There were 150,683 (weighted 73,423,189) cases identified for analysis. There was a small but statistically significant improvement in the AUROC of the MOI criteria in the later decade (2010–2019; AUROC = 0.77, 95% CI [0.76–0.78]) compared to the earlier decade (2000–2009; AUROC = 0.75, 95% CI [0.74–0.76]). The accuracy and specificity did not vary with age, but the sensitivity dropped significantly for older adults (0–18 years: 0.62, 19–54 years: 0.59, ≥55 years: 0.37, and ≥65 years: 0.36). The addition of entrapment improved the sensitivity of the existing criteria and was the only potential new criterion to maintain a sensitivity above 0.95. Conclusions The MOI criteria for MVCs in the current CDC guidelines still perform well even as vehicle design has changed. However, the sensitivity of these criteria for older adults is much lower than for younger occupants. The addition of entrapment improved sensitivity while maintaining high specificity and could be considered as a potential modification to current MOI criteria.


Introduction
The appropriate triage of severely injured patients to trauma centers is essential to optimizing outcomes after motor vehicle collisions (MVCs). There are estimated to be over 6 million MVCs, resulting in over 30,000 fatalities, in the United States each year (National Center for Statistics and Analysis 2021). Transporting severely injured patients directly to trauma centers has been shown to decrease mortality by 25% (MacKenzie et al. 2006). However, transporting patients without severe injuries to trauma centers does not appear to reduce mortality but may result in increased patient care costs and stress trauma center resources (Newgard et al. 2013;Scott et al. 2020). The ability for prehospital providers (e.g., first responders) to determine which patients are likely to have severe injuries is therefore critical to maximizing survival while not overutilizing trauma centers.
The Centers for Disease Control's (CDC) "Guidelines for the Field Triage of Injured Patients" published in 2011 were designed to assist prehospital providers in deciding which patients should be transported directly to a trauma center (Sasser et al. 2012). The algorithm for determining the transport destination is separated into 4 steps, each with a different focus (physiologic, anatomic, mechanism of injury [MOI], and special populations). Within the MOI criteria, 4 criteria are specific to MVCs. A study of the 2006 CDC guidelines, which remained essentially unchanged in the 2011 revision, showed that the MOI criteria had a sensitivity of 44.3% and a specificity of 84.3% ). These MOI injury criteria improve the overall triage accuracy when combined with the other steps of the algorithm Newgard et al. 2011Newgard et al. , 2017. However, studies have demonstrated that the CDC guidelines have an appropriate specificity but the sensitivity is below the goal recommended by the American College of Surgeons Committee on Trauma (Rotondo 2014;Newgard et al. 2016Newgard et al. , 2017.
The performance of the MOI criteria specific to MVCs in real-world collisions has not been evaluated since the release of the 2011 guidelines. Vehicle design and automotive safety technology are continually evolving, and the performance of these criteria may have changed in the last decade. The primary objective of this study was to assess for a change in the prognostic performance of these criteria during the past 2 decades. Secondary objectives were to assess the prognostic performance of each criterion individually, evaluate these criteria in different age groups, and determine the benefit of adding new criteria not currently in the CDC guidelines.

Study design
This study was a retrospective, population-weighted cohort study based on national crash data from 2000 to 2019. Consistent with the criteria outlined in the National Institutes of Health's common rule and the institutional review board at the University of Virginia, this study was considered exempt from institutional review board review because data are de-identified and publicly available.

Data source and case selection
Data were obtained from the NASS-CDS for 2000-2015 and the Crash Investigation Sampling System (CISS) for 2017-2019. These data are collected and made available through NHTSA. Data were downloaded from the NHTSA website (National Highway Traffic Safety Administration 2021).
All crashes in the NASS-CDS and CISS databases between 2000 and 2019 were selected for inclusion in this study. Cases were excluded after 2008 if the vehicle model year was over 10 years earlier than the crash year. These cases were excluded because NHTSA stopped performing crash investigations on older vehicles in 2009. Cases were also excluded if the occupant's injury data were missing. All other missing data were imputed using multiple imputation using chained equations. Five imputation sets were created with different random number generator seeds. An analysis of the missing data is provided in Table A1 (see online supplement).

MOI criteria
The MOI criteria specific to MVC from the latest CDC field triage algorithm were evaluated (Sasser et al. 2012), as well as potential new criteria not currently in the guidelines. The 4 existing criteria included (1) intrusion including the roof: 12 in. at the occupant position or 18 in. at any site, (2) ejection (complete or partial) from the automobile, (3) death in the same passenger compartment, and (4) vehicle telemetry data consistent with a high risk of injury. The last criterion was not examined because there is currently no widely accepted standard for determining whether vehicle telemetry data are consistent with a high risk of injury.
The new criteria were selected based on a review of previously published literature and the data available in both NASS-CDS and CISS Newgard et al. 2016;Stitzel et al. 2016). We also considered which criteria prehospital providers could identify at the scene of a crash. The new criteria we evaluated were (1) multiple collisions, (2) any airbag deployment, (3) entrapment of the occupant, (4) unrestrained occupant, (5) any rollover, (6) 2 quarter turns, (7) 4 quarter turns, (8) posted speed limit 55 mph (89 kph), and (9) posted speed limit 65 mph (105 kph).

Outcome
The outcome of interest was an Injury Severity Score (ISS) 16. There are other outcomes currently in use for determining whether an occupant has a severe injury, such as Maximum Abbreviated Injury Scale 3; however, the ISS 16 is the measure used for severe trauma at U.S. trauma centers and was used in the initial creation and evaluation of the current CDC guidelines (Sasser et al. 2012;Rotondo 2014;Newgard et al. 2016).

Statistical analysis
Characteristics of the sample populations were first compared for the initial data set excluding missing data and the imputed data set. Continuous data were recorded as the mean and interquartile range, and categorical data were recorded as percentages. The characteristics of crashes were then compared between 2000-2009 and 2010-2019.
The performance of the current MOI criteria was compared between 2000-2009 and 2010-2019 using the case weights supplied by NHTSA. Samples of the length of the original data set were randomly selected with replacement (bootstrapped) from the 5 imputed data sets for each decade. The median area under the receiver operator curve (AUROC) was calculated from 1,000 bootstrapped samples. The AUROC was determined by sequentially measuring the sensitivity and specificity after applying each existing criterion. The 95% confidence interval (CI) was obtained by the range that included 95% of the results from the bootstrapped samples. A statistically significant change in AUROC between the 2 decades was determined using a 2-tailed t-test, with alpha ¼ .05 as the threshold for significance. We also performed a sensitivity analysis using the Delong method of comparing ROC curves (DeLong et al. 1988) to assess for a difference in performance between the 2 decades. For this sensitivity analysis, a single imputed data set from both decades was used.
The accuracy, sensitivity, and specificity using case weights for the existing criterion were then determined individually and cumulatively for the criteria in the order they appear in the CDC guidelines. In the context of this study, sensitivity was the number of severely injured occupants who met at least 1 triage criterion divided by the number of severely injured occupants, and specificity was the number of non-severely injured occupants who met none of triage criteria divided by the number of non-severely injured occupants. The new potential criteria were evaluated in the same manner both individually and when added to the existing CDC criteria. The median AUROC and 95% CI were determined based on 1,000 bootstrapped samples from the entire study period.
The performance of the existing criteria in different age groups was then evaluated using accuracy, sensitivity, and specificity. These were also determined by taking 1,000 bootstrapped samples from the occupants in the target age range from the entire data set. The age ranges evaluated were 0-18, 19-55, 55, and 65 years.
Processing of NASS-CDS data was performed using R programming language v3.6.1 (R Core Team 2019). Processing of CISS data and all other analyses were performed using Python programming language (Van Rossum and Drake 2009) v3.8, with packages NumPy, pandas, and SciPy.Stats.

Results
There were 155,666 cases identified in NASS-CDS and CISS between 2000 and 2019, with 4,983 (3.2%) cases excluded due to missing ISS, leaving 150,683 cases for analysis ( Figure  A1, online supplement). Due to decreased case sampling by NHTSA over time and a gap in collection during 2016, there were approximately twice as many cases in 2000-2009 (N ¼ 101,919) compared to 2010-2019 (N ¼ 48,764). Overall, higher severity crashes were investigated in the latter decade, as indicated by the increased intrusion, death in the same compartment, multiple collisions, rollovers, and ISS 16 (Table 1). There was a 41% increase in rollover incidents from 2000-2009 to 2010-2019. This may be due to the increased percentage of sport utility vehicles in the latter decade, which have an increased risk of rollovers (Kallan and Jermakian 2008;IIHS-HLDI 2019). There were also substantially more airbag deployments in the latter decade, likely due to the increasing number of airbags equipped in vehicles. The 5-fold multiple imputation resulted in 753,415 cases used in the analysis. The characteristics of the imputed cases were similar to those of the original data (Appendix-Sensitivity Analysis, online supplement).
The ROC curves for the current CDC MOI criteria were similar for the early (2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009) and late (2010-2019) decades ( Figure 1A). The largest difference in performance was a higher sensitivity in the early decade after evaluating intrusion 12 in. at the occupant's position. The AUROC after 1,000 bootstrapped iterations was slightly lower for the   Figure  1B), which was statistically significant (P < .01). In a sensitivity analysis we compared the ROC curves from the whole data using the Delong method and also detected a statistically significant difference (P < .01).
The criteria from the current CDC guidelines all showed high accuracy and specificity but low sensitivity (Table 2). Ejection had the highest sensitivity, and death in the same passenger compartment had the highest specificity. The addition of each criterion showed significant improvement in the sensitivity to the cumulative performance. The combined criteria had an overall sensitivity of 0.55 (95% CI [0.53-0.57]) and specificity of 0.97 (95% CI [0.97-0.98]).
The accuracy and specificity were relatively consistent between age groups, but the sensitivity differed by age group (Figure 2). There was a small decrease for occupants aged 19-54 years compared to 0-18 years, and then the sensitivity decreased markedly for occupants aged 55 years. There was >40% decrease in sensitivity for occupants 65 years compared to those aged 0-18 years.
The potential new criteria all had lower accuracy than the existing criteria (Table 3). Most of the new criteria had higher sensitivity but lower specificity than the existing criteria. Posted speed limit 45 mph had the highest sensitivity, and entrapment of the occupant had the highest specificity. The addition of each criterion to the existing criteria resulted in increased sensitivity but decreased specificity. The only criterion that maintained a specificity over 95% when combined with the existing criteria was entrapment of the occupant.

Discussion
Our results indicate a small but statistically significant increase in performance of the CDC's MOI criteria for MVCs in the 10 years since they were initially adopted. Additionally, each existing criterion contributes significantly to increasing the overall sensitivity. These findings support the continued inclusion of these criteria in the field triage guidelines. The existing criteria all have high specificity but only moderate sensitivity, which decreases substantially for adults aged 55 years.
The existing MOI criteria performed well in the decade since they were originally evaluated; however, there were some differences in performance compared to the preceding decade. The largest difference was the higher sensitivity of intrusion 12 in. at the occupant position for older vehicles. Improved vehicle design, which better distributes frame loading, has been shown to decrease intrusion (Jakobsson et al. 2013). We surmise that this criterion may have become less sensitive because vehicle improvements in the latter decade may have decreased intrusion into the occupant compartment. This difference in sensitivity between the 2 decades decreased as more criteria were added, and the overall performance in the latter decade was slightly better once all criteria were considered.
There was a substantial decrease in the performance of the MOI criteria for older passengers (55 and 65 years) compared to younger passengers (54 years). These results mean that more individuals over age 55 were severely injured in collisions that did not have any of the MOI features in the CDC algorithm. The greater susceptibility to injury of older adults compared to younger adults in equivalent crashes has been well documented (Kent et al. 2005;Kahane 2013;Kodadek et al. 2015). Additionally, it has been shown that it is more challenging to predict severe injuries in older adults, even using prediction models tailored to these populations (Hartka et al. 2019). There is not a straightforward solution to increasing the sensitivity of MOI for older adults. However, step 4 of the CDC algorithm indicates that the risk of injury and death increases after age 55 year, so transport to a trauma center should be considered. Our findings indicate that this is an important recommendation because the sensitivity of the MOI criteria was markedly lower for occupants 55 years and older. Judgment is still necessary because a previous analysis has shown that it is not cost-effective to transport all trauma patients 55 years to a trauma center (Maughan et al. 2022).
The addition of new MOI criteria for MVCs could improve the performance of the CDC's field triage  guidelines. It is essential to recognize that the MOI only represents 1 step in this algorithm. Though no single step of the algorithm needs to meet the goals put forth by the American College of Surgeons Committee on Trauma, the overall algorithm has been shown to have insufficient sensitivity in practice (Newgard et al. 2016). Adding new criteria could improve the sensitivity but would also result in a decrease in specificity. Therefore, the optimal new criteria would provide a meaningful improvement in sensitivity while maintaining high specificity.
Our results demonstrate that entrapment of the occupant is the most promising new MOI criterion for MVCs. Entrapment increased the overall sensitivity from 0.55 to 0.62 when combined with the existing criteria and was the only new criterion that maintained a specificity above 0.95. The second most promising criterion was rollover with 4 or more quarter turns. However, this criterion resulted in a smaller improvement to sensitivity and a greater decrease in specificity. Additionally, we feel it might be difficult for emergency medical services (EMS) providers to determine the number of quarter turns a vehicle sustained at the scene of a crash in some circumstances, whereas entrapment is more readily apparent.
A study by Lerner and colleagues was foundational to the development of the current MOI criteria . This study examined the association between characteristics of traumatic events and severe injuries for a cohort of patients presenting to the emergency department. Our results confirm the selection of criteria MOI for MVCs; however, our results had some notable differences. Though the specificities were similar, we found sensitivities significantly higher than those reported by Lerner et al. (2011). This might be due to the focus of the NASS-CDS and CISS databases on more severe crashes, which is not entirely compensated for by case weights. Another possibility is that our study included crashes where the occupant died at the scene. We chose to include these patients in our analysis because the time of death was not recorded in many cases. Also, had these patients survived long enough for transport, they would be most likely to benefit from care at a trauma center. Though the sensitivity may be lower when applied in clinical practice because of these deaths, we expect the relative performance of the criteria to remain consistent.
The CDC MOI criteria include vehicle telemetry data consistent with a high risk of injury. We did not include vehicle telemetry in our analysis because these data are not consistently available in real time for trauma triage decisions. Additionally, there are no official recommendations regarding when vehicle telemetry data are consistent with a high risk of injury. There are several promising algorithms to predict injury severity from telemetry data (Bahouth et al. 2004;Stitzel et al. 2016;Weaver et al. 2017;Hartka et al. 2021). It is possible that once these data are available in real time the current MOI criteria will no longer be needed. However, there are still many infrastructure and regulatory challenges to the widespread implementation of such a system, so these criteria will likely be necessary for the next decade.
There are several limitations of this study that should be considered. In addition to the retrospective nature of this research, the vehicle analysis was performed by trained crash investigators rather than EMS providers. EMS providers must perform determinations such as the distance of intrusion with constrained time and limited experience with these tasks. However, prospective validation of the CDC algorithm showed that the MOI criteria, as evaluated by EMS, improve the sensitivity of accuracy of trauma center triage (Newgard et al. 2016). Another limitation is that our data set contained significant missing data that were imputed using multiple imputation using chained equations. However, we repeated this analysis without imputation (Appendix-Unimputed Analysis, online supplement) and observed very similar trends, although individual metrics varied slightly. The data from the latter decade also come from both NASS-CDS and CISS, which have different sampling strategies. An analysis comparing the performance of the MOI criteria in each database showed better results in CISS compared to NASS-CDS (Appendix-Database Analysis, online supplement). This could explain the slight improvement in performance in the latter decade, although this analysis is confounded because CISS contains newer cases. Finally, NASS-CDS and CISS do not have sufficient information to evaluate MOI criteria in the context of the entire CDC algorithm. Prehospital vital signs are often not available, and which injuries were apparent on EMS evaluation are unknown. Further research is necessary to assess how changes to the MOI criteria would alter the algorithm's overall performance.
In conclusion, these results show that the MOI criteria for MVCs in the current CDC field trauma triage guidelines still perform well, even as automotive safety has improved. However, the performance of these criteria for older adults is much lower than that for younger occupants. Entrapment of the occupant could be considered to improve the sensitivity of the criteria.