A Comparison of Population-Averaged and Cluster-Specific Approaches in the Context of Unequal Probabilities of Selection

Sampling designs of large-scale survey studies are typically complex, involving multiple design features such as clustering and unequal probabilities of selection. Single-level (i.e., population-averaged) methods that use adjusted variance estimators and multilevel (i.e., cluster-specific) methods provide two alternatives for modeling clustered data. Although the literature comparing these methods is vast, comparisons have been limited to the context in which all sampling units are selected with equal probabilities (thus circumventing the need for sampling weights). The goal of this study was to determine under what conditions single-level and multilevel estimators outperform one another in the context of a two-stage sampling design with unequal probabilities of selection. Monte Carlo simulation methods were used to evaluate the impact of several factors, including population model, informativeness of the design, distribution of the outcome variable, intraclass correlation coefficient, cluster size, and estimation method. Results indicated that the unweighted estimators performed similarly across conditions, whereas the weighted single-level estimators tended to outperform the weighted multilevel estimators, particularly under nonideal sample conditions. Multilevel weight approximation methods did not perform well when the design was informative. An empirical example is provided to demonstrate how researchers might investigate the implications of the simulation results in practice.