A Comparison of Different Approaches to Bayesian Hierarchical Models in a Basket Trial to Evaluate the Benefits of Increasing Complexity

A basket trial is a trial that investigates a single experimental therapy simultaneously across multiple different patient population groups. When groups are heterogeneous, the optimal analysis is independence but when groups are homogeneous the optimal analysis strategy is pooling. A Bayesian Hierarchical Model (BHM) analysis strategy offers a compromise between independence and pooling. There are multiple different parameterizations of BHMs in the literature offering different ways to account for possible lack of exchangeability and heterogeneity across the groups in the basket. We selected 4 different BHMs that fit under the same general structure but have increasing levels of complexity to accommodate group heterogeneity for comparison. Because the choice of prior for a BHM affects the behavior of the model, we first optimize each model’s prior selection in terms of maximizing the expected number of correct decisions across six efficacy scenarios. After optimization, differences between the performance of the models were small. In the context we evaluated, a simpler model may provide similar operating characteristics to a more complex model even when groups are truly heterogeneous.