Knowledge translation intervention increased the use of outcome measures by physical therapists in inpatient rehabilitation

ABSTRACT Background and Purpose Outcome measures (OMs) have been emphasized by healthcare professions to optimize patient examination; however, a lack of regular use of OMs exists. The purpose of this study was to describe the outcome of a knowledge translation (KT) intervention to increase the use of OMs by physical therapists in an inpatient rehabilitation setting. Methods A quasi-experimental pre-post study design was used. A multi-component KT intervention including education, organizational support, documentation, and environmental changes to increase the use of five OMs was implemented. Audit and feedback (A&F) was added to the KT intervention at month 6. Documented use of OMs was determined through manual chart audit (n = 864) and electronically (n = 2599). Regression analyses were used to identify factors associated with OMs use across time and diagnoses. Results Following the addition of A&F to the KT intervention at month 6, there was a significant increase in the odds of OMs use across all time intervals (months 6–12, 12–18, 18–24)(Odds Ratio (OR) 5.9, 95% Confidence Interval (CI) 4.1–8.5; OR 8.5, 95% CI 6.0–12.1; OR 10.8, 95% CI 7.6–15). There was also a significant increase in the odds of documenting OMs on individuals with neurological diagnoses (OR 0.3, 95% CI 0.5–0.8). Conclusions This KT intervention increased and sustained OMs use over 24-months. This intervention can be replicated to improve the evidence-based practices of physical therapists.


Introduction
Outcome measures (OMs) are standardized instruments used to evaluate actual or perceived ability of an individual to perform activities and participate in daily life (Jette et al., 2009). Professional associations (American Physical Therapy Association, 2020) and guidelines (Moore et al., 2018b;Sibley et al., 2015), recommend that rehabilitation providers implement OMs into routine clinical practice to assess a patient's initial status and response to treatment, provide information about the overall outcomes of an organization, and compare outcomes of clinicians and organizations (Institute of Medicine, 2013). In physical therapy (PT), standardized OMs are a required component of examination (American Physical Therapy Association, 2020). The need for routine use of OMs is widely recognized and led to the development of resources such as Rehabilitation Measures Database (RMD) (https://www.sralab.org/reha bilitation-measures), Evaluation Database to Guide Effectiveness (EDGE) (ptresearch.org/home/edge-taskforce), and Task Forces (ANPT) (http://www.neuropt. org/practice-resources/neurology-section-outcomemeasures-recommendations). These initiatives gather psycho-metrics, report on ease of use, and guide physical therapists regarding OM selection. Despite these recommendations and resources to support the use of OMs, they are not integrated regularly in PT practice (Duncan and Murray, 2012;Jette et al., 2009;Salbach, Guilcher, and Jaglal, 2011;Sibley et al., 2011).
Physical therapy practice has identified many barriers to the use of OM that are related to clinicians, organizations, patients, and specific outcome measures, which may account for limited use of OM in practice (Duncan and Murray, 2012). Due to the multi-faceted challenges, knowledge translation (KT) research is needed to identify efficient and effective strategies to overcome barriers and promote the adoption of OMs in rehabilitation. KT research investigates the process of moving evidence into practice and aims to decrease the evidence-topractice gap, which includes identifying the gap between research recommendations and routine practice (e.g. characteristics of current use of OMs by clinicians) (Graham et al., 2006;Straus, Tetroe, and Graham, 2013). Research indicates the effectiveness of KT may be increased by using frameworks to guide projects, selecting multi-component implementation strategies that target barriers and are tailored to the local setting, and actively engaging the stakeholders of the organization in the project (Duncan and Murray, 2012;Field, Booth, Ilott, and Gerrish, 2014;Graham et al., 2006;Hudon, Gervais, and Hunt, 2015;Straus, Tetroe, and Graham, 2013).
Research studies in PT have sought to increase the use of OMs in pediatric (Russell et al., 2010;Schreiber, Marchetti, Racicot, and Kaminski, 2015); outpatient orthopedic (Käll, Larsson, and Bernhardsson, 2016); acute care (McDonnell, Stillwell, Hart, and Davis, 2018); and rehabilitation settings (Romney, Salbach, Parrott, and Deutsch, 2019) as well as for individuals with stroke (Bland et al, 2013;Van Peppen et al., 2009). A recent systematic review on the use of OMs in rehabilitation identified that commonly used KT strategies, including education, may improve OM use. However, the authors reported that most studies did not use of frameworks to guide the KT process, lacked details of tailored interventions to target specific barriers, and suggested more active multi-component intervention strategies were needed (Colquhoun et al., 2017a).
A paucity of literature has described the development of tailored multi-component KT interventions to address clinician-specific and measure-specific barriers to increase the use of OMs in PT (Bland et al, 2013;McDonnell, Stillwell, Hart, and Davis, 2018;Romney, Salbach, Parrott, and Deutsch, 2020). Audit and feedback (A&F) is a promising KT strategy that addresses clinician barriers by monitoring clinician behavior and providing direct feedback on their performance (Colquhoun et al., 2017b;Ivers et al., 2012). Bland et al. (2013) developed a KT intervention using A&F to improve the use of OMs across rehabilitation disciplines for individuals with stroke. Several OMs were selected across discipline and setting to address measure-specific barriers and audits were performed to determine adherence. Managers were provided with clinician-specific and measure-specific feedback to disseminate among their staff. Bland et al. (2013) found that OM use was less than desired and concluded that new processes needed to be developed to address the clinician barriers and improve adherence. McDonnell, Stillwell, Hart, and Davis (2018) selected several recommended OMs and used A&F in an acute care setting and reported improvement in the use of OMs. However, the authors recommended a more robust A&F system and the use of a framework to guide the KT intervention. Romney, Salbach, Parrott, and Deutsch (2019) performed a survey and focus group with PTs to determine clinician-specific barriers using outcome measures and mapped barriers to strategies. The Knowledge-to-Action Framework (Graham et al., 2006) and Theoretical Domains Framework (Michie et al., 2005) guided their work. One outcome measure was selected by the PT staff, with a goal of 50% adherence. The authors used a multi-component KT intervention that included A&F. This study resulted in improved use of a gait speed measure in a sub-acute rehabilitation hospital over 10 months.
The overarching goal of this study was to increase the use of OMs by physical therapists working in inpatient rehabilitation. Specifically, a tailored multi-component KT intervention, including A&F, was implemented over 24-months to increase the use of OMs and reporting of patient OMs in clinical documentation. The research questions were as follows: 1) Did the KT intervention increase OMs use; 2) Did the physical therapists meet and sustain use at the recommended adherence levels over 24-months; and 3) Did differences exist in documented use of OMs for individuals with medically complex issues in comparison to neurologic diagnoses? The final question was included because the EDGE Task Forces and RMD focus more on neurologic diagnoses.

Design
A pre-post quasi-experimental study was completed to describe the change in documented use of OMs following the KT intervention. Use was measured through manual chart audits and an automated report from the electronic medical record.

Sample of physical therapists
The sample included all physical therapists who worked at the hospital. In this facility, physical therapists typically work in teams based on diagnosis; however, they examine and treat individuals with many different diagnoses. In February 2015, there were 14 full-time physical therapists (Table 1). Throughout the study, there were four full-time resignations, three who remained parttime employees and four new full-time hires. The Workgroup for Intervention Development and Evaluation Research (WIDER) was used to report the KT intervention (Albrecht, Archibald, Arseneau, and Scott, 2013) (Supplementary Material 1: WIDER recommendations). The Sacred Heart University and Gaylord Specialty Healthcare Institutional Review Boards approved this study.

Procedures
The action cycle of the Knowledge to Action (KTA) framework was used to design the KT study (Table 2) (Graham et al., 2006). Several physical therapists at the facility identified the problem of inconsistent use of OMs and discussed the problem with therapy supervisors and director. Simultaneously, upper-level management was interested in a formalized outcome report from therapy to document and share overall outcomes of clinical care. Therapy management identified the inability to systematically aggregate and analyze the data which led to the creation of an Outcome Measure Committee (OMC). The OMC members were opinion leaders, senior clinicians that serve as role models to help implement change (McDonnell, Stillwell, Hart, and Davis, 2018;Moore et al., 2018a;Perry, Zeleznik, and Breisinger, 2014). These clinicians were tasked with improving the use and documented use of OMs within the facility. The OMC also included the manager of therapy services and two physical therapists. The OMC evaluated the evidence presented in existing KT tools (Moore, Raad, Ehrlich-Jones, and Heinemann, 2014) as well as EDGE and RMD to select a battery of performance-based activity-level OMs that could be used across the patient populations. The following criteria were used to select the OMs: 1) rating of "recommended" or "highly recommended" by a minimum of two APTA EDGE Task Forces (i.e. Spinal cord injury, stroke, Parkinson's disease, traumatic brain injury, multiple sclerosis, and vestibular), which included good or excellent psychometrics and clinical utility (easy to administer and score); and 2) applicability to a variety of functional levels.
The OMC selected four OMs including 1) 10-Meter Walk Test (Bohannon, 1997); 2) Berg Balance Scale (Berg, Wood-Dauphinee, Williams, and Maki, 1992); 3) Timed Up and Go (Mathias, Nayak, and Isaacs, 1986); and 4) Functional Reach Test (FRT) (standard or modified) (Katz-Leurer et al., 2009;Weiner, Duncan, Chandler, and Studenski, 1992). Approximately six months after starting the study, the Five Times Sit to Stand Test (FTSTS) was added as a fifth OM to accommodate better those with limited ability to ambulate (Csuka and McCarty, 1985). The clinicians were asked to achieve the goal of performing at least one outcome measure for every patient in the hospital at initial examination and discharge.
The OMC also identified barriers and facilitators to using these OMs in clinical practice. It developed an intervention plan to use social support and engagement to address the identified barriers ( Table 2). The multicomponent KT intervention plan included the use of opinion leaders, educational in-service, documentation changes in the electronic medical record and paper examination templates, environmental changes to serve as reminders, organizational and therapy administrative support, goal setting, social engagement, and A&F ( Table 2). The OMC committee provided a one-hour interactive educational in-service to 14 members of the PT staff (Table 2, Supplementary Material 2: Educational In-Service). Physical therapists not present at this session were trained by an OMC member including the 14 part-time employees and four new hires. An additional 15-minute in-service on the FTSTS was completed at month 6. Reminders were conducted weekly at staff meetings, and at month 6 feedback was provided monthly (Supplementary Material 3: Audit and Feedback Intervention Design). Informal meetings with the OMC occurred on a regular basis. Audit and feedback was the primary strategy used to improve the use and documentation of the OMs (Ivers et al., 2012). At four months, management set a goal of ≥90% adherence to either documenting one OM at the initial examination for every patient or documenting a reason why an OM was not completed in a designated comment field. Before this study, therapy managers at the hospital were completing 36 monthly chart audits (six charts from six different units: stroke, medically complex, cardiac, traumatic brain injury, spinal cord injury, and general medical), to determine adherence to other documentation standards including the Centers for Medicare and Medicaid Services (CMS) Quality Indicators, examination documentation completed within 72 hours of admission, and documentation of pain and patient/family goals. Outcome measure adherence on the initial examination was added to these monthly chart audits after four months. The physical therapy manager provided feedback from the chart audits to the physical therapists at monthly staff meetings beginning at month 6. The feedback was delivered to the group for social engagement, motivation, and to allow the physical therapists to discuss barriers to use. Individual feedback was provided on an as-needed basis to the physical therapists who dropped below the goal. This feedback was intentionally collaborative and constructive. After month seven, the managers added OMs as a component of the documentation section of the physical therapists' annual performance review (Supplementary Material 3: Audit and Feedback). • OMC met and discussed barriers and facilitators to current outcome measure use ○ Identified barriers included: lack of knowledge, lack of outcome measure field on electronic medical record and paper documentation, lack of designated physical space to perform OMs, lack of accountability ○ Identified facilitators: organizational support, physical therapist engagement/motivation Adapt Knowledge to the Local Context Month 0 • OMC developed a one hour educational in-service to review four outcome measure protocols, psychometric properties, clinical decision-making strategies, and provided demonstration. The therapists then practiced the OMs.
• Developed resource materials on how to perform and interpret the OMs and placed in therapy gym Select, Tailor, Implement Interventions Months 0-6 OMC: • Placed equipment to perform OMs around therapy gym (Reminders for Physical Space) • Adapted initial examination paperwork and weekly coverage treatment card to include OMs (Reminder for Lack of Use) • Documentation changes in EMR (Reminder for Lack of Use) Management: • Set expectation that clinicians perform at least one OM within 3 days of initial examination and discharge for every patient (months 0-3) • The OMC and management planned sustainability activities into the KT intervention. These included chart audits and verbal feedback about documentation adherence goals which remained as part of the monthly staff meetings. Open discussions about barriers and new KT strategies were reviewed by the OMC when adherence was less than desired. The OMC continually evaluated the selected OM and research for changes. In addition, the documentation procedures in performance reviews were made with the intent of conveying the leadership team's ongoing expectation of sustained behavior change related to physical therapists' use and documentation of OMs.

Data collection
Thirty-six monthly chart audits were conducted by a team of six supervisors and managers in the PT department. Data from the 36 monthly chart audits over the 24 months were collected to determine if a subset of patient charts had either documentation of OMs at initial examination or reasons why OMs were not documented on the initial examination. In addition, a report was developed using the electronic medical record (EMR) system to determine OM use for all patient charts over 24 months. Data reported on to the automated EMR report included patient admission and discharge dates, diagnosis, and OM type and score at initial examination and discharge. The EMR report did not capture the open comment fields where physical therapists may have documented more information about the OMs, including reasons why the OMs were not performed and reasons for deviations from testing protocols.

Data analysis
Data from the manual chart audit were analyzed descriptively (frequency and percentages) to determine documentation of OMs on the initial examination and if documentation met the 90% adherence goal. Data from the EMR were downloaded to Excel and coded based on OM documentation at initial examination and discharge. Reliability was ensured by having the researchers re-code a subset of 200 charts from the 0-6 month time period. The second code demonstrated 100% consistency. The FRT, using the modified and standard instructions, was combined for analysis because it was unclear from the EMR which test was selected. The Excel spreadsheet was uploaded and analyzed using SPSS version 24. Descriptive statistics were used to determine the frequency and percentage of use of OMs at the initial examination, discharge, and both across time points and OMs. Descriptive statistics were used to compare documentation of OMs on the initial examination between the manual chart audit and the EMR report and OM utilization across diagnostic groups. Logistic regression was used to determine if A&F improved the documented use of OMs. The time of OM administration was the primary factor of interest. Charts were divided into four, six-month time-periods: 0-6 months (baseline), 6-12, 12-18, and 18-24 months. The baseline of 0-6 months was selected because feedback started at the 6-month time point (Table 2). Patient diagnostic groups were divided dichotomously: individuals with neurological diagnoses and individuals with medically complex issues. Adjusted odds ratios and 95% confidence intervals were provided for each factor.

Results
A multi-component KT intervention guided by the action cycle of the KTA framework was used to increase the use of OMs into clinical practice at an inpatient rehabilitation setting over 24 months. The monthly chart audits (n = 864) found baseline (0-6 month) frequency (percentage) of documented use of OMs on the initial examination ranged from 53%-94% (18-34 out of 36 charts). After A&F began at month 6, the manual chart audits exceeded the 90% adherence goal and it was sustained for the study duration (months 6-24) (Figure 1).
There were 2599 patient charts included from the EMR report during the 24 months. Documentation of OMs use on the initial examination and discharge increased after A&F began at month six and remained higher than baseline for the duration of the study (Figure 2). The FRT (standard and modified) was the most frequently used OM at each time period across all diagnostic groups (Supplementary Material 4: Frequency of Outcome Measure Use across Major Diagnostic Groups at Initial Examination, Discharge, and Both). The logistic regression found a significant (p < .001) increase in the odds of documenting OMs at both initial examination and discharge across all time periods following baseline (0-6 months) OR 6-12 5.9, 95%CI 4.1-8.5; OR 12-18 8.5, 95%CI 6.0-12.1; OR 18-24 10.8, 95%CI 7.6-15.3. There was also a significant (p < .001) increase in the odds of documenting OMs at both initial examination and discharge for individuals with neurological diagnoses (n = 315, 35.5%) (OR .6, 95% CI .5-.8) as compared to individuals with medically complex diagnoses (n = 415, 24.2%).
A comparison of the manual chart audit and EMR report for the initial examination determined that manual chart audits had higher documentation adherence on the initial examination at each time point (Figure 3). The manual audit recorded documentation of an OM or documentation of reasons why an OM was not performed, while the EMR report only determined documentation of an OM.

Discussion
The design and implementation of a multicomponent KT intervention increased and sustained the use of OMs by physical therapists over 24 months. The use of the KTA framework guided the implementation of measurement recommendations made by the APTA EDGE Task Forces (ANPT Outcome Measures Recommendations (EDGE), 2018) and changed clinical practice at an inpatient rehabilitation facility. Experienced physical therapists collaborated with managers and recognized the inconsistent use of OMs. An initiative was introduced by critical stakeholders to introduce a battery of OMs and increase the use of OMs by the physical therapists. Strategies used to implement the OMs initially included opinion leaders, stakeholder engagement, education, documentation changes, environmental changes, and administrative support. Audit and feedback was the primary strategy used to monitor and ensure adherence to the OM recommendations. During the study, adherence to OM administration recommendations increased from 53% to >90%, which was sustained over the course of 24 months.
Several important findings resulted from this KT study. The documentation of OMs was higher at discharge, the FRT either standard or modified was used most frequently, and individuals with neurological diagnoses were more likely than individuals with medically complex issues to have OMs documented. We hypothesized that physical therapists documented OMs more frequently on discharge due to patient's improved functional ability. Although a three-day window to collect initial examination data was intended to allow time to perform a comprehensive examination. It is possible that the patients may not have been functionally able to perform an OM, and the physical therapists may not have recorded zero to indicate the inability to perform the test. In addition, we speculate that the FRT was used more frequently than any other OM because of the ability to modify the test to complete in sitting and its administrative ease. Lastly, individuals with neurological diagnoses were more likely to have OMs completed on the initial examination and discharge compared to those with medically complex diagnoses. This may be due to the increased volume of EDGE Task Force recommendations and RMD OM summaries that targeted neurologic disorders as compared to others. Previous research reports mixed success with A&F (Bland et al, 2013;McDonnell, Stillwell, Hart, and Davis, 2018;Romney, Salbach, Parrott, and Deutsch, 2020). Positive results from this study may be due to the systematic approach using A&F and the KTA framework. Management used strategies supported by the literature including stating an adherence goal, promoting social support and engagement, and providing face-to-face feedback at monthly time intervals which improved the success of the intervention (Colquhoun et al., 2017b). We hypothesize that A&F with clear adherence expectations from the management were key factors contributing to the successful increase in the use of OMs.
The comparison of the manual audits to the EMR report provided insight into the use of OMs. The manual chart audit tracked the adherence goal, which included OMs documented on the initial examination or documented reasons for not completing OMs within the three-day examination window. The EMR report tracked adherence by collecting OM type and documented score, but it did not capture open comment fields with documentation for reasons of nonuse. Adherence data from the EMR report (50.6-56.1%) was similar to reported OM use in previous literature (48%-88%) (Bland et al, 2013;Jette et al., 2009;McDonnell, Stillwell, Hart, and Davis, 2018;Romney, Salbach, Parrott, and Deutsch, 2020). This approach enhances existing literature on adherence because the manual chart audit had adherence rates >90% as compared to the EMR report 50-56% and demonstrated that even if the OM was not completed, physical therapists documented reasons for why an OM was not performed. Documentation of OMs at initial examination and discharge is required to demonstrate a patient's response to treatment. The OMC may consider additional training to include documentation of "0" for patients with goals for balance or ambulation to provide an opportunity to compare initial examination and discharge change scores. Including an open field comment in the electronic medical record for "reasons not performed" may be beneficial for investigating additional reasons for nonuse. Exploring reasons for nonuse should be evaluated and discussed with the PT staff for future barrier assessment and intervention development to increase OM usage.
Organizational support is vital to improve KT and behavior change (Colquhoun et al., 2017b;Duncan and Murray, 2012;Romney, Salbach, Parrott, and Deutsch, 2019;Straus, Tetroe, and Graham, 2013). This organization, both executive and middle management, valued the physical therapists input as demonstrated by the creation of the OMC who served as opinion leaders and engaged the physical therapists in the intervention (Moore et al., 2018a;Perry, Zeleznik, and Breisinger, 2014). The OMC added designated testing space, equipment and the addition of the FTSTS as a result of clinician input. The organization also supported changes to documentation which facilitated the development of the EMR report. The management plans to share the EMR report with organizational leadership, as well as publicly display these data to share patient outcomes at the facility to support the sustainability of this practice.
Physical therapist engagement was a useful strategy to increase buy-in (Leeman et al., 2015;Straus, Tetroe, and Graham, 2013). The engagement of stakeholders has been used to adapt KT interventions (Stevens and Beurskens, 2010), or as a strategy for development and implementation of the entire KT process in the literature (Moore et al., 2018a;Perry, Zeleznik, and Breisinger, 2014;Romney, Salbach, Parrott, and Deutsch, 2019). Perry, Zeleznik, and Breisinger (2014) used physical therapist opinion leaders as role models and mentors to facilitate engagement in the KT intervention. Moore et al. (2018a) trained "champions" in the KT intervention and provided time for the champions to train other physical therapists in the evidence-based practice implemented in the large rehabilitation hospital. Romney, Salbach, Parrott, and Deutsch (2019) engaged the physical therapists to select the outcome measure and design the intervention. This research, as well as the current study, provide examples of the use of engagement to increase evidence-based practice behavior change in KT. More research should be conducted to determine factors that lead to enhanced engagement and more effective and efficient implementation.
Future research should enhance the design and further explore the use of administrative support, opinion leaders and A & F to encourage OM use. More KT research is needed to move evidence into practice by training physical therapists on selecting and using OMs that are psychometrically sound for the population and setting of interest, documenting zero when appropriate, following the standardized procedure, and using the newly developed Core Set of OMs for Adults with Neurologic Conditions Clinical Practice Guideline (Moore et al., 2018b). Also, planning and recording cycles of barrier assessment, intervention development and evaluation of behavior change will help build the sustainability of this KT study. In addition, reliability testing across physical therapists and OMs would enhance comparisons of physical therapy interventions.

Limitations
This KT study was conducted at a single inpatient rehabilitation facility with a small sample size and it lacked a control group or randomization. The physical therapists at the facility most frequently used and adapted the modified FRT, which has limited psychometric properties. More synthesized resources and psychometric properties on OMs for patients with medically complex issues, lower functional levels and impaired cognitive status are needed. It is possible that physical therapists incorrectly entered OM data when transferring it from paper to computer record. Manual chart audits were extracted by managers who were not part of the research study and not checked by the research team for reliability. This KT study investigated adherence to OM, as indicated by OM documentation. It is unknown if the clinician used the data to inform clinical decision-making or patient education.

Conclusion
Physical therapists at this inpatient rehabilitation facility used synthesized resources to select a battery of OMs. They successfully used a KT intervention, guided by the KTA framework, to improve adherence to the OMs. Administrative support with A&F, as well as goal setting, not only improved OM documentation but it was sustained over the 24-month duration. This KT study can be replicated and used by other facilities to improve documented the use of OMs. In this study, KT intervention strategies including A&F changed OM practice among physical therapists. As evidence-based practice is a foundational aspect of the PT profession, using OMs is one step toward becoming a more evidence-based practitioner.