figshare
Browse
interpreting unexpected significant.pdf (308.16 kB)

Interpreting unexpected significant findings

Download (0 kB)
journal contribution
posted on 2014-05-18, 10:09 authored by Dorothy V M BishopDorothy V M Bishop

Many people assume that if an ANOVA output is significant at the .05 level, there's only a one in twenty chance of it being a spurious chance effect. We have been taught that we do ANOVA rather than numerous t-tests because ANOVA adjusts for multiple comparisons. But this interpretation is quite wrong. ANOVA adjusts for the number of levels within a factor, so, for instance, the probability of finding a significant effect of group is the same regardless of how many groups you have. ANOVA makes no adjustment to p-values for the number of factors and interactions in your design. The more of these you have, the greater the chance of turning up a "significant" result.

History