The Interactive Effect of Anger and Disgust on Moral Outrage and Judgments

The two studies reported here demonstrated that a combination of anger and disgust predicts moral outrage. In Study 1, anger toward moral transgressions (sexual assault, funeral picketing) predicted moral outrage only when it co-occurred with at least moderate disgust, and disgust predicted moral outrage only when it co-occurred with at least moderate anger. In Study 2, a mock-jury paradigm that included emotionally disturbing photographs of a murder victim revealed that, compared to anger, disgust was a more consistent predictor of moral outrage (i.e., it predicted moral outrage at all levels of anger). Furthermore, moral outrage mediated the effect of participants’ anger on their confidence in a guilty verdict—but only when anger co-occurred with at least a moderate level of disgust—whereas moral outrage mediated the effect of participants’ disgust on their verdict confidence at all levels of anger. The interactive effect of anger and disgust has important implications for theoretical explanations of moral outrage, moral judgments in general, and legal decision making.

elicited by purity violations of body or soul (Horberg, Oveis, Keltner, & Cohen, 2009). Disgust cues (e.g., the presence versus absence of offensive odors) result in harsher moral judgments (Schnall et al., 2008), and neural regions associated with disgust are also associated with moral judgments (Moll et al., 2005). In fact, among emotional reactions to moral violations, disgust is sometimes more consistent than anger (Hutcherson & Gross, 2011).
Determining whether moral outrage arises from a combination of anger and disgust requires testing both their independent and interactive effects. Assessing the independent roles of anger and disgust in moral judgments is difficult given that they are likely to covary (Marzillier & Davey, 2004), and people sometimes use the term disgust interchangeably with anger (Nabi, 2002;Olatunji et al., 2012). Researchers have thus highlighted the need to differentiate between their antecedent appraisals and consequences (Hutcherson & Gross, 2011), to measure them simultaneously (Gutierrez & Giner-Sorolla, 2007), and to examine their independent effects on moral judgments by controlling for each statistically (e.g., Horberg et al., 2009). These investigations have revealed that anger and disgust can operate differently. For example, violations of body norms and taboos elicit disgust, whereas harm elicits anger (Gutierrez & Giner-Sorolla, 2007), and disgust is more resistant than anger to change in the face of new, mitigating information (Russell & Giner-Sorolla, 2011).
Despite the frequent co-occurrence of anger and disgust, and the efforts to investigate their independent effects on moral judgments, we are unaware of any studies that have focused on either the unique effect of disgust or the interactive effect of disgust and anger on moral outrage (or on moral judgments in general). Why might anger and disgust interact to produce moral outrage? Both anger and disgust are associated with certainty appraisals (Smith & Ellsworth, 1985); as a result, both anger (Bodenhausen, Sheppard, & Kramer, 1994) and disgust (Tiedens & Linton, 2001) can diminish information processing. Thus, anger and disgust might each exacerbate the effect of the other. For example, increasing anger might lead to increased reliance on visceral gut reactions (disgust) when making moral judgments, and increased disgust might lead to increased reliance on anger when making moral judgments. Thus, perhaps moral outrage represents a unique combination of anger and disgust in which the two emotions interact so that each exacerbates the other's influence on judgments.
Our primary goal in these studies was to investigate the interactive effect of anger and disgust on moral outrage and subsequent judgments. We hypothesized that anger and disgust would predict moral outrage only when each emotion co-occurred with at least a moderate level of the other. Because moral outrage has been identified as an important mediator of anger on downstream social consequences (Skitka et al., 2004), we also tested whether moral outrage would mediate the effect of anger and disgust on verdicts in a murder case, and whether each indirect effect would be moderated by the other emotion. Specifically, we hypothesized that (a) moral outrage, only when co-occurring with at least moderate disgust, would mediate the effect of anger on murder verdicts, and (b) moral outrage, only when cooccurring with at least moderate anger, would mediate the effect of disgust on murder verdicts.

Study 1
We assessed anger, disgust, and moral outrage toward two moral transgressions to ensure that the predicted anger and disgust interaction would generalize across different moral transgressions. Disgust can be elicited by different types of purity violations (e.g., purity violations of body or soul; Horberg et al., 2009). Thus, to ensure that our conclusions about the hypothesized anger-bydisgust interaction would not be limited to only one type of moral transgression, we asked each participant to read one of two vignettes: One vignette included a body violation (sexual assault) and another did not (Westboro Baptist Church funeral picketing).

Method
Participants (N = 102; mean age = 34 years, SD = 11; 50% women), recruited through Amazon's Mechanical Turk (http://www.mturk.com), read a vignette describing either a sexual assault by a man named "David" (n = 39) or funeral picketing by the Westboro Baptist Church (n = 63). After they read the vignette, participants completed simultaneous measures of how disgusted and angry they felt about the offender's actions, using a scale from 1 (not at all) to 5 (extremely), on a grid designed to discourage people from using the word disgust to label their anger (Fig. 1). Participants in the sexual-assault condition were given the following instructions: Please use this grid to indicate how angry and disgusted you feel by David's actions. David's actions can make you feel high in both, low in both, or high in one and not the other. Along the bottom of the grid is how disgusted you feel about David's actions, with low disgust on the left through high disgust on the right. Along the left side of the grid represents how angry you feel about David's actions, from low anger on the bottom to high anger at the top. Please enter the number of the box that lines up with your level of disgust and your level of anger at David's actions.
Participants in the funeral-picketing condition were given the same instructions, except that "David's" was replaced with "Westboro Church's." Participants also responded to four items assessing moral outrage (e.g., "I feel morally outraged toward the defendant"; α = .62) on a scale from 1 (not at all) to 5 (very much). These statements have been used previously to capture the cognitive, affective, and behavioral components of moral outrage (Skitka et al., 2004

Results and discussion
Bivariate correlations. We examined bivariate correlations among disgust, anger, and moral outrage ( Table 1). The significant correlation between anger and disgust was expected given that they have previously been shown to covary (Marzillier & Davey, 2004). These two correlations were of only moderate size, however, indicating thatas expected-anger and disgust are not redundant constructs. 1

Regression analyses.
Step 1 in a hierarchical regression revealed that anger was a significant predictor of moral outrage, b = 0.23, t(94) = 2.66, p < .01, and disgust was a marginal predictor of moral outrage, b = 0.19, t(94) = 1.85, p = .07; R 2 = .19, F(2, 94) = 11.08, p < .001. In Step 2, we entered the interaction between anger and disgust; this was also significant, b = 0.17, t(94) = 2.58, p = .01, R 2 = .24, ΔR 2 = .04. (See Fig. 2.) To probe the interaction, we applied the Johnson-Neyman technique (Hayes & Matthes, 2009), which is an alternative to more traditional techniques that require the researcher to select two arbitrary values of the moderator at which to assess the significance of the focal predictor (e.g., ±1 SD from the mean; see Aiken & West, 1991). The Johnson-Neyman approach avoids this potentially arbitrary choice by revealing all ranges of the moderator (i.e., disgust) in which the focal predictor (i.e., anger) is a significant or nonsignificant predictor of the outcome (i.e., moral outrage). Thus, this approach provides a more complete picture of moderation patterns than do traditional methods. Specifically, we used this technique to determine the value of disgust when anger becomes a significant predictor of moral outrage and again to determine the value of anger when disgust becomes a significant predictor of moral outrage (Tables 2 and 3).
Anger became a stronger predictor of moral outrage as disgust increased, becoming significant when participants' level of disgust reached 3.94 on the 5-point scale. Likewise, disgust became a stronger predictor of moral outrage as anger increased, becoming significant when participants' level of anger reached 3.16 on the 5-point scale. Thus, the combination of disgust and anger predicted  moral outrage, rather than each emotion predicting moral outrage independently. Anger predicted moral outrage only when it co-occurred with at least a moderately high level of disgust, and disgust predicted moral outrage only when it co-occurred with at least a moderately high level of anger. 2

Study 2
Study 2 extended Study 1 in two ways. First, we tested the interactive effect of anger and disgust on moral outrage in another scenario likely to elicit anger and disgust in our participants-a gruesome murder. Second, we tested the potential downstream consequences of participants' experiencing anger, disgust, and moral outrage on a novel outcome: legal verdicts. Although verdicts are ideally driven by rational weighing of facts, jurors' negative emotions can increase guilty verdicts (e.g., Bright & Goodman-Delahunty, 2006; for reviews of potential mechanisms, see Feigenson & Park, 2006;Salerno & Bottoms, 2009). We tested one potential mechanism: A combination of anger and disgust might increase moral outrage, which in turn might increase participants' confidence in a guilty verdict. In support of this hypothesis, moral outrage has been identified as an important mediator of the effect of anger on other outcomes. For example, moral outrage mediates the effect of anger on political tolerance (Skitka et al., 2004) and the effect of offense severity on punishment (Carlsmith et al., 2002;Salerno et al., 2010). Furthermore, given that anger predicts both increased moral outrage (Skitka et al., 2004) and guilty verdicts (e.g., Bright & Goodman-Delahunty, 2006), we hypothesized that anger would increase moral outrage, which in turn would increase confidence in a guilty verdict. There is no previous research, to our knowledge, that has tested whether moral outrage mediates the effect of disgust on subsequent judgments-perhaps because moral-outrage research has focused heavily on anger, rather than disgust. Trait disgust sensitivity predicts participants' estimates of the likelihood that a defendant committed a crime ( Jones & Fitness, 2008); therefore, we hypothesized that disgust would also increase moral outrage, which in turn would increase confidence in a guilty verdict. We predicted that these mediation effects for each emotion (e.g., the indirect effect of disgust on verdicts through moral outrage) would be moderated by the other emotion (e.g., anger). This is consistent with the prediction that disgust and anger would moderate each other's effect on moral outrage, which was supported by Study 1. That is, because anger and disgust are associated with certainty (Smith & Ellsworth, 1985) and diminished information processing (Bodenhausen et al., 1994;Tiedens & Linton, 2001), we hypothesized that anger and disgust might each exacerbate the effect of the other on subsequent judgments. Increasing anger, for example, might lead to increased reliance on visceral gut reactions (disgust) in verdict judgments, and increasing disgust might lead to increased reliance on anger in making verdict judgments. Thus, following our hypotheses in Study 1, we predicted a moderated-mediation pattern such that moral outrage would mediate the effects of anger and disgust on confidence in a guilty verdict-but only when each emotion co-occurred with at least a moderate level of the other emotion. We thus hypothesized that the unique interactive pattern of anger and disgust would predict not only moral outrage but also downstream consequences of moral outrage (i.e., confidence in a guilty verdict).

Method
In a mock-juror study, 118 jury-eligible undergraduates (mean age = 19 years; 63% women) arrived in a laboratory and viewed a 20-min presentation of evidence; this presentation was adapted from an actual murder case (R v. Valevski, 2000), including opening and closing statements, testimony from witnesses, photographs and descriptions of knife wounds on the victim's throat, and real jury instructions. Participants completed two measures that were combined to create the verdict-confidence variable for analyses. Specifically, they reported (a) their preferred verdict and (b) their confidence in that verdict on a scale from 0% confident to 100% confident (see Methodological Details in the Supplemental Material for the measure used to assess verdict confidence). We recoded and combined these two measures into one bipolar scale, which allowed responses that ranged from 100% confident in a not guilty verdict to 100% confident in a guilty verdict. The measure of disgust and anger was identical to that used in Study 1, except that it was a paper-and-pencil version without numbers in the boxes, and participants responded by marking an "X" in the appropriate box. The instructions were the same as those described in Study 1, except that "David's" was replaced with "the defendant's." Participants rated their moral outrage on the same 5-point scale used in Study 1 (α = .91).

Results and discussion
Bivariate correlations. We again examined bivariate correlations among disgust, anger, and moral outrage ( Table 1). As expected-and consistent with Study 1the moderately sized correlation coefficient indicated that anger and disgust were related but not redundant constructs.

Regression analyses. In
Step 1 of a hierarchical regression, disgust, b = 0.31, t(106) = 4.38, p < .001, and anger, b = 0.22, t(106) = 3.38, p = .001, predicted moral outrage, R 2 = .31, F(2, 106) = 23.68, p < .001. In Step 2, we entered the interaction between anger and disgust and found that it was again significant, b = 0.16, t(105) = 2.33, p = .02, ΔR 2 = .04. To probe the interaction, we again used the Johnson-Neyman test, which revealed that, consistent with Study 1 results, anger was a stronger predictor of moral outrage as disgust increased, becoming significant when participants' level of disgust reached 2.87 (Tables 4  and 5). In contrast, disgust significantly predicted moral outrage at all levels of anger. We also applied the Johnson-Neyman technique to the moderated-mediation analyses (Preacher, Rucker, & Hayes, 2007)  In sum, the combination of anger and disgust again predicted moral outrage. Disgust, however, predicted moral outrage more consistently-from not at all angry to extremely angry-whereas anger required at least moderate levels of disgust before predicting moral outrage. Critically, as predicted, anger and disgust both increased confidence in a guilty verdict through moral outrage-but the strength of the indirect effect of each emotion depended on the level of the other.

General Discussion
Two studies confirmed that moral outrage is distinguishable from pure anger by demonstrating that moral outrage results from a combination of anger and disgust-even when the transgression did not include a body-disgust violation (funeral picketing). Despite often being characterized as the central emotional component of moral outrage, anger predicted moral outrage only when it co-occurred with at least a moderate level of disgust, and disgust predicted moral outrage only when it co-occurred with at least a moderate level of anger. In fact, disgust was a more consistent predictor of moral outrage in Study 2-it significantly predicted moral outrage at all levels of anger (and even in the absence of anger). This may be the case because disgust (vs. anger) is more resistant to mitigating evidence (Russell & Giner-Sorolla, 2011), which was presented in Study 2 (i.e., the defense's case) but not in Study 1.
Furthermore, moral outrage mediated the effect of both disgust and anger on judgments with serious reallife consequences: murder verdicts. Anger increased moral outrage, which in turn increased participants' confidence in a guilty verdict-but only when it co-occurred with at least moderate levels of disgust. Disgust predicted confidence in a guilty verdict through moral outrage, however, at all levels of anger. Because both anger and disgust are associated with certainty appraisals that decrease cognitive processing, each emotion might encourage greater reliance on the other when making judgments.
This conceptualization of moral outrage has an important theoretical implication: Moral outrage is distinguishable from pure anger (cf. Batson et al., 2007). Prior conclusions that the two are indistinguishable might be a result of operationalizing moral outrage using only anger measures, thereby failing to take disgust into account. Furthermore, disgust and anger have different eliciting cues (Russell & Giner-Sorolla, 2011), underlying cognitive appraisals (Rozin et al., 1999), and action tendencies (Gutierrez & Giner-Sorolla, 2007), which suggests complex antecedents and processes behind moral outrage that await future investigation.
The implications of our findings extend beyond moral outrage. Despite the effort to differentiate the effects of anger and disgust, this was the first demonstration of an interactive effect of disgust and anger, not only on moral outrage but also on judgments more generally. Given that anger and disgust have competing action tendencies (approach for anger, and punishment and avoidance for disgust; Gutierrez & Giner-Sorolla, 2007), future research could investigate what behavior the combination yields in both moral and nonmoral judgments.
Finally, these findings have implications for important real-world issues, particularly for the role of emotion and moral outrage in legal decision making and in how people assign blame for moral transgressions. Although verdicts are ideally driven by rational weighing of evidence, mock jurors' moral outrage toward the defendant explained the effect of both anger and disgust on guilt judgments. Evidence that elicits both anger and disgust (e.g., gruesome photographs; Bright & Goodman-Delahunty, 2006) might affect jurors' emotions so as to increase their likelihood of voting guilty by increasing their moral outrage-without necessarily providing any additional information-and therefore should be carefully regulated in court.
Contrary to traditional conceptualizations of moral outrage, these studies suggest that moral outrage is an emotional experience distinguished from anger by the coexperience of disgust. Anger, disgust, and their interactive effect should be included in theoretical explanations of moral outrage and moral judgments in general as well as in exploration of their real-world consequences.  (Hayes & Matthes, 2009) was again used to probe the anger-by-disgust interaction on moral outrage. To test for moderated mediation, we used Preacher, Rucker, and Hayes's (2007) MODMED SPSS macro that incorporates the Johnson-Neyman technique as an alternative to more traditional moderated-mediation analyses that use arbitrary points of the moderator at which to test conditional indirect effects. This analysis reveals all ranges of the moderator (i.e., disgust) in which the conditional indirect effect of the focal predictor (i.e., anger) on the outcome (i.e., verdict confidence) through the mediator (i.e., moral outrage) is significant. Values in boldface indicate that the conditional effect was a significant predictor of moral outrage.

Notes
1. Because our independent variables were moderately correlated, we conducted collinearity analyses. The results did not merit further investigation (tolerance ≥ .45, variance inflation ≤ 2.23). 2. Although we were interested in generalizing across vignettes, for the interested reader, we present separate analyses by vignette. We got similar results when we analyzed the vignettes separately. Specifically, the disgust-by-anger interaction was significant for the sexual-assault vignette, b = 0.24, t(34) = 2.94, p = .01. Analyses of simple slopes revealed that anger predicted moral outrage at high levels of disgust (0.5 SD above the mean), b = 0.90, t(34) = 3.90, p < .001-but not at low levels of disgust (0.5 SD below the mean), b = −0.03, t(34) = −0.21, p = .81. Likewise, disgust predicted moral outrage at high levels of anger (0.5 SD above the mean), b = 0.87, t(34) = 3.10, p < .01, but not at low levels of anger (0.5 SD below the mean), b = −0.22, t(34) = −1.36, p = .18. We found consistent results for the funeral-picketing vignette-although the anger-by-disgust interaction was marginally significant, b = 0.18, t(56) = 1.72, Note: The Johnson-Neyman technique (Hayes & Matthes, 2009) was again used to probe the anger-by-disgust interaction on moral outrage. To test for moderated mediation, we used Preacher, Rucker, and Hayes's (2007) MODMED SPSS macro that incorporates the Johnson-Neyman technique as an alternative to more traditional moderated-mediation analyses that use arbitrary points of the moderator at which to test conditional indirect effects. This analysis reveals all ranges of the moderator (i.e., anger) in which the conditional indirect effect of the focal predictor (i.e., disgust) on the outcome (i.e., verdict confidence) through the mediator (i.e., moral outrage) is significant. Values in boldface indicate that the conditional effect was a significant predictor of moral outrage. p = .09 (most likely because the separate vignette analysis was underpowered). Analyses of simple slopes revealed that anger predicted moral outrage at high levels of disgust (0.5 SD above the mean), b = 0.50, t(56) = 2.22, p = .03-but not at low levels of disgust (0.5 SD below the mean), b = −0.16 t(56) = −0.72, p = .48. Likewise, disgust predicted moral outrage at high levels of anger (0.5 SD above the mean), b = 0.94, t(56) = 2.25, p = .03, but not at low levels of anger (0.5 SD below the mean), b = 0.16, t(56) = 0.94, p = .35. (We broke down the interaction at ±0.5 SD from the mean for this analysis because doing so 1 SD above and below the mean resulted in values that extended beyond the original 5-point scale that are therefore not interpretable.) Furthermore, the anger-by-disgust interaction remained significant with vignette included as a covariate, b = 0.18, t(93) = 2.69, p < .01. Thus, we are confident that our collapsed results for the combined data set that we report in the main results represent the pattern of results in each separate vignette.