Improving access in rheumatology: Evaluating the validity of a paper triage process involving an advanced practice physiotherapist through a retrospective chart review

Objectives: This study evaluated a standardized paper triage process conducted by an advanced practice physiotherapist (APP) at a rheumatology center. The aims were to (1) determine the concordance between paper triage priority assignment and the rheumatologist’s diagnosis; (2) determine the sensitivity and specificity of the paper triage process; and (3) assess reasons for incorrect priority ranking. Methods: Referrals were triaged by a formally trained APP into one of the three priorities, guided by a priority referral tool. A retrospective review of 192 charts was performed. Raw proportion of agreement between paper triage and rheumatologist’s diagnosis was supplemented by a prevalence-adjusted bias-adjusted kappa (PABAK). Priority categories were collapsed to calculate sensitivity and specificity. For discordant cases, additional information was collected from the referral and chart to identify potential features leading to discrepancy. Results: Overall agreement was 76%. The PABAK was 0.80 [95% confidence interval 0.70–0.90]. Sensitivity ranged 0.64–0.92 and specificity ranged 0.81–0.94, depending on the priority category. Forty-six cases were discordant, with the APP choosing a higher priority in 37 cases. An incorrect diagnosis from the family physician with no supporting information for the paper triage led to discordance in 16 cases. Conclusion: A standardized paper triage process conducted by an APP showed substantial concordance, sensitivity, and specificity.