MetaLab: Supporting Power Analysis and Experimental Planning in Developmental Research
MetaLab (metalab.stanford.edu) is a web-based tool that aggregates meta-analyses across different domains in language acquisition. MetaLab can be used for power analysis, experimental planning, and theory development. Currently, MetaLab includes meta-analyses on five topics (phoneme discrimination: Tsuji & Cristia, 2014; word segmentation: Bergmann & Cristia, 2015; infant directed speech preference: Dunst, Gorman, & Hamby, 2012; pointing and language development: Colonnesi, et al., 2010; and mutual exclusivity). In each meta-analysis, effect sizes are extracted from the primary research literature, providing a standardized difference metric for comparing across studies. Currently, MetaLab includes 575 effect sizes from 165 mostly published and peer-reviewed papers and data from 11,627 children. MetaLab is completely open and encourages contributions from researchers to expand and stay up to date (Tsuji, Bergmann & Cristia, 2014).
Reproducible research requires experimental designs with appropriate statistical power. But since effect sizes are often unknown, sample sizes are difficult to determine prospectively. Language development research is particularly sensitive to this issue: both sample sizes and effect sizes are relatively small and so research is typically underpowered and thus unlikely to reliably measure an effect even if it is really there (for example, a typical study in word segmentation from native speech has n = 24 and Cohen’s d = .2). MetaLab allows researchers to estimate effect sizes across studies and to select experimental design parameters that increase their likelihood of success. For instance, based on an estimated effect size, MetaLab determines the appropriate sample size for observing an effect at a given level of desired power (Fig. 1). Critically, researchers can customize their query to the particular phenomenon, age, and method of their planned study. For example, while 16-24 children are sufficient for many studies of mutual exclusivity in word learning, such sample sizes would be insufficient to reliably observe a true effect in a study of word segmentation.
Better effect size estimates are also important for theoretical progress. Existing meta-analyses reveal developmental trends within individual phenomena, but comparing these trends across phenomena is more difficult because of the wider range of tasks and ages. MetaLab provides a synthesis across different meta-analyses by visualizing the relationship between the developmental trajectories of different phenomena in language development (Fig. 2). This analysis provides an empirical analogue to classic “ages and stages” charts that show how different abilities overlap in developmental time, and highlights the interactive nature of language learning.
MetaLab is constantly growing; the MetaLab team is currently involved in 3 additional meta-analyses and we invite submissions from authors of independent meta-analyses and offer supporting material (for example https://sites.google.com/site/infantdbs/create-your-own-cama). We thus expect to arrive at an increasingly detailed picture of infants' developing abilities.
To sum up, using MetaLab allows researchers to take into account a growing body of existing developmental data to plan experiments and explore theoretical questions about the developmental trajectories of different phenomena. The promise of this work is an empirically driven synthesis of our knowledge about early language development.
