Fragility of evidence for efficacy of anti-fracture medications-Supplementary documents

Context: A P-value and statistical significance, conventionally considered for assessing an intervention’s effectiveness are usually misused and misinterpreted.

Objective: To quantify fragility of randomized controlled trial (RCT) evidence for anti-fracture efficacy.

Design: This retrospective analysis included 27 phase 3/4 RCTs in high-profile medical journals which assessed anti-fracture efficacy, allocated participants in a 1:1 ratio to pharmacological intervention or control and reported a statistically significant result. Fragility of the results were assessed using the Fragility Index (FI) and Fragility Quotient (FQ). FI is the minimum number of participants in a positive analysis result for whom reversing the reported status would eliminate statistical significance, while FQ is a function of FI to the sample size.

Results: The median FI was 9 (IQR: 4, 19), indicating that adding 9 fracture patients (~0.51% of the study size) to the intervention group would eliminate the documented evidence of anti-fracture efficacy. Notably, the number of participants lost to follow-up exceeded the corresponding FI in 60% of analyses. The most robust evidence for anti-fracture efficacy was documented for romosozumab (FI: 19.5; IQR: 7.0, 31.5); whereas the least found for denosumab (4; 3, 17) and calcium/vitamin D supplementation (7.0; 2.3, 16.8). Anti-fracture efficacy evidence improved among the results that considered fractures the primary endpoint measure (14; 11, 33) or those with P-value< 0.001 (26; 18, 42).

Conclusion: The existing RCT evidence of anti-fracture efficacy is highly fragile. The FI, its comparison with loss to follow-up and FQ should be incorporated into clinical guideline development and doctor-patient risk communication.