Resilience in Army STARRS: Evaluating psychometrics of a multi-dimensional resilience measure

ABSTRACT As policymakers and the U.S. military continue to place an emphasis on the resilience of servicemembers, it is critical to utilize psychometrically sound and valid scales to measure resilience. Using two independent samples of Army soldiers-in-training, this study explored the measurement of resilience in the Army Study to Assess Risk and Resilience among Servicemembers (Army STARRS) New Soldier Study Component (NSS). Exploratory factor analysis (EFA) was used to identify the factor structure of a measure of resilience within the Army STARRS NSS. Confirmatory factor analysis (CFA) was then used to confirm the factor structure, then internal reliability was assessed. Convergent validity of the identified resilience factors was examined using two-tailed bivariate correlations. The EFA identified a three-factor structure of a measure of resilience. The CFA confirm the first-order three-factor structure of stress tolerance, positive orientation, and social resources. Each factor was uniquely distinct from measures of the likelihood of generalized anxiety disorder and major depressive disorder, lifetime stressful events, and social network. Findings highlights the utility of a three-factor aggregate measure of resilience in the Army STARRS NSS and provide practitioners with a more nuanced picture of the role of resilience among soldiers-in-training.

Given the increasing focus on the role of resilience among servicemembers by policymakers (U.S.Department of Defense, n.d.), researchers (e.g., Meredith et al., 2011;Nindl et al., 2018), and the U.S. military (U.S.Department of the Army, 2019) it is critical to utilize psychometrically sound and valid scales to measure resilience.In resilience theory, resilience is described as the process or capacity to have successful outcomes or adaptations despite challenging circumstances (Masten et al., 1990).Masten (2015) posited the dynamic process of resilience as one that encompasses individual factors (e.g., genetics, relationships, and environments) and the co-action that exists between one's individual factors and the multiple systems within which individuals live.Within this context, servicemembers are routinely faced with challenging events (e.g., long-term deployments overseas, frequent residential changes; Tong et al., 2018; U.S. Institute of Medicine, 2013); resilience has been identified as a key factor in promoting positive outcomes for these frequently challenged servicemembers (e.g., Britt et al., 2021;Isaacs et al., 2017).Within the workplace, resilience has been described as multi-dimensional selfregulatory processes used to achieve desired outcomes in the face of adversity (Rothstein et al., 2016).
In this study, preexisting items within the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS; Ursano et al., 2015) New Soldier Study (NSS) component, were assessed to identify how resilience can be measured when using the STARRS data.Campbell-Sills et al. (2018) previously examined the predictive validity and correlates of five items of resilience included in the Army STARRS NSS.However, there have been several different approaches to the examination of resilience, highlighting the multifaceted nature of the construct (Fletcher & Sarkar, 2013).Scholars have theorized and operationalized resilience not only as tolerance or management of stress (Campbell-Sills et al., 2018), but also as responding to adversity in adaptive ways (Denckla et al., 2017), positive affect and thinking (Meredith et al., 2011), and connection to social resources (Masten, 2015).Ultimately, although Campbell-Sills et al. (2018) provided a sound foundation for assessing resilience in the Army STARRS NSS, the previously identified questions only capture current responses to stressful stimuli and do not consider other related factors (e.g., positive orientation, social resources).
Assessing the validity of measures can be a challenge when conducting secondary data analyses; identifying validation studies on items of interest within existing data can help inform researchers' decision-making regarding methodology (Smith et al., 2011).This study aims to contribute to this body of literature for those hoping to examine resilience within the Army STARRS by examining additional items within the NSS to determine if a multifaceted measure of resilience was present.(Quinlan et al., 2016) discussed the importance of improving the measurement of resilience based upon (a) construct operationalization, (b) construct application, and (c) system dynamics (i.e., how people, services, and/or institutions interact).With this in mind, guided by theory and existing research, we (a) operationalized resilience as an individual's ability to withstand disruptive life events through reports of perceived stress tolerance, positive orientation, adaptability, and the likelihood of using social resources; (b) applied it to new Army soldiers-in-training during basic combat training; and (c) examined the myriad ways in which individual factors of resilience are related to, yet distinct from, experiences of anxiety, depression, lifetime stressors, and one's social network.

Method
Data from the Army STARRS NSS (Ursano et al., 2015) were used and the study was approved by (U.S.Department of Defense).The NSS encompasses a crosssectional probability global sample of 38,507 new US Army, National Guard, and Reserve soldiers, aged 18-64.Data were collected during the first 2 weeks of basic combat training (BCT) at three Army installations.In order to utilize an analytic sample of soldiers-in-training who were new to the military, individuals who did not report age, were over age 35, or had missingness exceeding 50% on items retained were removed (n = 2,375), yielding an analytic sample of 36,132.To adjust for discrepancies between the NSS sample and the population of individuals who reported to BCT across the US during the study period, combined analysis weights that account for population demographic and service characteristics were used (see, Kessler et al., 2013a for description of weighting and clustering).All subsequent analyses are based on weighted data.

Sample
Of those who reported their enlistment primary component (45.6%), over half (57.0%) indicated regular Army, followed by 32.4% who indicated Army National Guard and 10.6% Army Reserves.Soldiersin-training were a mean age of 20.8 years (SD = 3.37).The sample reflects the sex and racial make-up of the composition of the military (U.S.Department of Defense, 2020); soldiers-in-training were majority male (82.7%) and White (71.4%).Fewer identified as African American or Black (19.5%),American Indian or Alaskan Native (2.9%),Pacific Islander (1.2%), or Other (6.9%).Over half (61.8%) received a high-school diploma or equivalent, followed by a quarter (25.0%) with some post high-school education, 5.7% with an Associate's degree, 7.6% with a Bachelor's degree or higher.The majority were never married (84.3%); 14.0% were married and less than 2.0% were either divorced, separated, or widowed.

Resilience item pool
Based upon a priori understanding of resilience in the military literature and prior use of items in Army STARRS studies (i.e., Campbell-Sills et al., 2018;Choi et al., 2020;Stein et al., 2019), an initial pool of 24 items with face validity were identified and categorized a priori into four hypothesized constructs (stress tolerance; positive orientation; adaptability; social resources) by a reflective team using the operationalized definition of resilience (see above).Items ranged on a five-point Likert scale, although there was some variation in the prompt and value labels (see, Table 1 for the full list of items).

Anxiety and depression
Two screening scales from the World Health Organization (WHO) Composite International Diagnostic Interview Screening Scale (CIDI-SS; Kessler et al., 2013b) were used to assess for generalized anxiety disorder (GAD; 5 items) and a major depressive episode (MDE; 4 items).The CIDI-SS GAD and MDE have demonstrated satisfactory concordance with clinical diagnoses from the DSM-IV (Kessler et al., 2013b).Participants indicated how often within the past 30 days they experienced symptoms, with responses ranging from 0 (None of the time) to 4 (All or almost all of the time).Sample items include "Feel nervous" and "Take little or no interest or pleasure in things" for GAD and MDE screening scales, respectively.Items were averaged; higher scores reflected greater likelihood of GAD (M = .92;SD = 1.03; α =.90) and MDE (M = .55;SD = .75;α =.86).

Lifetime stressful events
Fifteen items from the Joint Mental Health Advisory Team 7 (Joint Mental Health Advisory Team 7, 2011) and Deployment Risk and Resilience Inventory (DRRI; King et al., 2006;Vogt et al., 2008) assessed for highly stressful experiences.Participants indicated how many times they experienced the events, with responses ranging from 1 (0) to 5 (10 or more).Sample items include "Serious physical assault," and "Suicide of a close friend or relative."Items were averaged, with higher scores indicating more stressful events (M = .34;SD = .44;α =.83).

Social network
Four items developed for the Army STARRS were used to assess one's social network.Participants indicated how many people they had in their personal life, with responses ranging from 0 (0) to 9 (31 or more).Sample items include "People who you feel really close to" and "People who really care for you and would be there if you needed them."Items were averaged, with higher scores indicating a larger network (M = .4.69; SD = 1.84; α =.81).

Analyses
Data were weighted, accounting for stratification and cluster, and descriptive statistics, normality, and dimensionality of the item pool were examined.After  determining the item pool was appropriate for factor analysis, a random number generator was used to assign soldiers-in-training to one of two groups: Group 1 (G1) or Group 2 (G2).Independent sample two-tailed t-tests were conducted to compare groups on demographic indicators and the initial item pool.An exploratory factor analysis (EFA) was conducted with the initial item pool using G1 in SPSS 25.Eigenvalues, proportion of variance explained by each factor, factor loadings (cut off < .30),communalities, and scree tests were used to determine the correct number of factors to extract.

EFA
A significant χ 2 (χ 2 [df] = 52,230.31[136],p < .001) on Bartlett's test for sphericity and a Kaiser-Meyer-Olkin (KMO) coefficient of .86 sampling adequacy and homoscedasticity indicated the data were appropriate for factor analysis.An oblique rotation method [Promax] was used, consistent with suggestions for when hypothesized factors are anticipated to be related (Thompson, 2004).Items were atrophied one-by-one, with the model rerun after each removal until a threefactor model (17 items) emerged (Table 1).The extracted structure (explaining 39.18% of the variance) was determined by the overall scree plot's leveling out point, overdetermination of factors, and factor loadings, cross-loadings, and communalities across items.All loadings on associated factors were > .30,no items were cross-loaded (> .30),all factors had at least four items, and findings were replicated across three additional randomized groups.Stress tolerance consisted of six items and involves the capacity, ability, or adaptability to deter or deviate from, or manage stressful stimuli.Positive orientation consisted of seven items and reflects an intrapersonal state of mind and approach to life consisting of positive affect, optimism, persistence, and high personal standards.Social resources consisted of four items and reflects a willingness to seek out interpersonal support from informal or formal domains.

CFA
Missing data on items within Group 2 ranged from less than 1-28%.CFA was conducted using the G2 sample to test the 17 item, three-factor structure emergent from the EFA and a second-order factor of global resilience.Items with large correlations (i.e., ≥ .05)were allowed (a priori) to covary.Model fit was good (x 2 [df] = 1752.22-[109],p < .001;CFI = 0.97; RMSEA = 0.03 [0.03, 0.03], p = 1.00;SRMR = .03).However, second-order loadings were inconsistent with expectations (Stress Tolerance λ = 0.40, p < .001;Positive Orientation λ = 1.27, p < .001;Social Resources λ = 0.24, p < .001)and the residual variance of Positive Orientation was negative, indicating model misspecification.Due to this inconsistency (i.e., one high factor loading, two low factor loadings), misspecification issue, and low correlations with social resources, the second-order factor was removed and the first-order factors were instead allowed to covary.The first-order model demonstrated good fit (x 2 [df] = 1752.23[109],p < .001;CFI = 0.97; RMSEA = 0.03 [0.03, 0.03], p = 1.00;SRMR = .03).See, Figure 1 for factor loadings.The relationship between resilience factors varied.Stress tolerance and positive orientation were highly correlated, positive orientation and social resources were moderately correlated, and a low correlation coefficient was found between stress tolerance and social resources.The findings were replicated across the three additional randomized groups.

Convergent validity
Finally, correlations were used to examine the relationship between the resilience factors and the likelihood of GAD and MDE, lifetime stressful events, and social network.The relationships between stress tolerance and GAD (r = −.31),MDE (r = −.32),lifetime stressful events (r = −.03), and social network (r = .13)demonstrated low to moderate correlations, suggesting related but distinct constructs.The relationships between positive orientation and GAD (r = −.13),MDE (r = −.22), and social network (r = .26)demonstrated low correlations, however, positive orientation and lifetime stressful events (r = .01)were not significantly correlated, indicating unrelated and separate constructs.The relationships between social resources and MDE (r = −.04),lifetime stressful events (r = −.03), and social network (r = .21)demonstrated low correlations, whereas social resources was not related to likelihood of GAD (r = −.01).

Discussion
The current study sought to build upon a preexisting set of resilience items in the Army STARRS NSS, examine the psychometric properties of those items and, in turn, uncover a multifaceted structure that can be used to capture multiple related, yet distinctive domains of the resilience, as purported by Fletcher and Sarkar (2013).From our analyses, three dimensions emerged and were confirmed: stress tolerance, positive orientation, and social resources, with items on each on factor demonstrating strong factor longings, adequate communalities, and with minimal cross-loadings.Additionally, the three factors demonstrated good face validity and internal reliability within acceptable ranges (Onwuegbuzie & Daniel, 2002).Although the three factors are interrelated, a higher order unidimensional factor was determined not to be optimal, indicating it may be inappropriate to combine all 17 items into an overall resilience variable.Rather, the three resilience factors should be treated as unitary, yet interrelated, constructs.
The relationships between the three resilience factors varied.Stress tolerance and positive orientation were moderately correlated, whereas there was a small correlation between the two factors and social resources.Such differences may reflect factor contexts, as stress tolerance and positive orientation were intrapersonal in nature, whereas social resources reflected intrapersonal and interpersonal components.Additionally, it may be that a person's ability to manage stress is more connected to their sense of optimism and high personal standards, rather than a willingness to use social resources.For example, behavioral and cognitive self-regulatory components of resilience have been linked with soldiers' perceptions of effectiveness (McLarnon et al., 2021).Perhaps having an optimistic outlook on stressors may make challenges seem more manageable, working to reduce the negative impact of stress (e.g., Nichter et al., 2020;Vanhove et al., 2016).
Although scholars agree upon the centrality of positive adaptation in the conceptualization of resilience (Fletcher & Sarkar, 2013), there is little consensus within the academic community in the defining of resilience.Some argue an individual must experience adversity in order to be resilient while other scholars argue resilience does not need to be necessitated by adversity (Seery et al., 2010).Findings from this study suggest perhaps both conceptualizations are correct and that it may be important to consider how different facets of resilience are tied to adversity whereas others are not.Positive orientation was not related to lifetime stressful events, indicating support for the latter approach (resilience independent of adversity) to the conceptualization resilience.Conversely, stress tolerance and social resources were linked to fewer lifetime stressful events, reflecting the former approach to resilience conceptualization, albeit in the opposite direction as might be anticipated.This difference in finding may also provide further insight into the differentiation between constructs, with positive orientation reflecting a more innate resilience function, which may be omnipresent independent of life circumstances.In contrast, stress tolerance and social resources may be more malleable aspects of resilience that can be influenced by external circumstances.
All three factors of resilience were related to depression, consistent with prior research indicating resilience is a protective factor for mental health issues among servicemembers (e.g., Campbell-Sills et al., 2009).However, there were mixed regarding resilience factors and anxiety.Both stress tolerance and positive orientation were significantly and negatively related to anxiety, consistent with expectations.However, social resources was not, and further demonstrated the smallest correlation coefficient of all factors with depression, despite its statistical significance.It could be that stress tolerance and positive orientation are more closely aligned with internalizing health problems.Future research may consider testing the relationship between the social resources subscale and externalizing health conditions to better understand its utility in the broader health landscape.
Finally, each factor of resilience was related to one's social network in the expected ways.The connection between positive orientation, social resources, and social network is largely intuitive; if one has a general positive outlook on life, they may be more apt to identify others as important references in their social network.Additionally, being willing to seek out help from social resources implies that social resources are available; having a larger social network may enhance one's willingness to reach out as they have more options.However, it is important to note social resources and social network were distinctly different constructs, given the small, yet significant, correlation.The relationship between stress tolerance and social network may be reflective of the theoretically and empirically supported notion that having extensive and available social networks (similar to how social network was operationalized herein) can act as a critical psychosocial resource that can aid in one's ability to counteract existing stressors and prevent the manifestation from occurring over time (Pearlin, 1983;Pearlin & Bierman, 2013).Although the size of one's social network is not interchangeable with the level or quality of support one receives from that network, if one has a larger network in which they can seek support from they may be better equipped to manage stress when it arises.
Although multiple groups were utilized, both groups came from the same study.Participants were soldiers-intraining, limiting generalizability.The identified items were not designed to operate together and thus items may not reflect the full range of subdomains that would be considered in the broader resilience literature; future research may consider rewordings and uniformed scaling in the interest of improving internal reliability and interpretability.Additionally, there may be other ways of operationalizing resilience within the Army STARRS; other scholars may come to a different consensus regarding what items within the data reflect resilience.This study did not explore external supports beyond one's social network, thus it is not clear to what extent other dynamics (e.g., quality of support) may relate to resilience.
Findings can be used to guide helping professionals in thinking about resilience in more nuanced and multifaceted manner when working with servicemembers.As Army training is considered a stressful experience for many U.S. soldiers (Britt et al., 2016;Burchett et al., 2015), soldiers-in-training may benefit from targeted efforts to foster resilience.Stress tolerance may be a resilience factor naturally targeted in training but fostering positive help-seeking attitudes may not be.If soldiers-in-training have access to tools that help them think about their training experience, and future careerrelated challenges, in a positive way, they may be better equipped to manage challenges as they arise, subsequently enhancing their mission readiness.
Furthermore, this multifaceted view of resilience in the Army STARRS NSS can guide future scholars in the examination of the more nuanced underpinnings of resilience among soldiers-in-training.This is important as Army STARRS is one of the few large datasets that includes samples of soldiers-in-training.The transition from civilian to military life is a critical point in which individuals may benefit from more tailored training to bolster their resilience and ultimately their ability to be mission ready, a key priority of the U.S. military (U.S.Department of Defense, 2018).
b I am a pretty passive person 2 * b I am very self-conscious 2 * b I am pretty set in my ways 2 * b There are many things I would just never do because I believe they are wrong 2 * b There are times when my future looks very dark 2 b I am pretty comfortable with emotional closeness, but I am also fine being alone.I don't worry much about being accepted or rejected 2 b I am usually very optimistic about the future 2 EEVA = eigenvalue; M = Mean; SD = standard deviation.Items in italics were hypothesized under a 4 th construct of Adaptability; all other items loaded on the a priori hypothesized construct.*reverse scored.a Rate ability to handle stress item prompt.b How well statement describes you item prompt.c Likely to talk/seek help from item prompt.

Table 1 .
Three-factor solution from the exploratory factor analysis of an aggregate measure of resilience.