Reproducibility of the items on the Stroke Specific Quality of Life questionnaire that evaluate the participation component of the International Classification of Functioning, Disability and Health.

Abstract Purpose: To evaluate the reproducibility of the Stroke Specific Quality of Life (SS-QOL) items that address the participation component of the International Classification of Functioning, Disability and Health (ICF) and analyse the correlation between the subscore of these 26 items and the total SS-QOL score. Methods: Seventy-five stroke survivors participated in this study. Reproducibility was evaluated using the intraclass correlation coefficient (ICC2,1), standard error of measurement (SEM), minimum detectable change (MDC) and the Bland–Altman plot. The correlation between the subscore of the 26 items and the total SS-QOL score was analysed using Spearman’s correlation coefficients (rho) and simple linear regression. An alpha risk ≤ 0.05 was considered for all analyses. Results: The SS-QOL items that address the participation component of the ICF demonstrated excellent reliability (intra-rater ICC2,1 = 0.96; inter-rater ICC2,1 = 0.95). The SEM and MDC were adequate. The Bland–Altman plot demonstrated satisfactory agreement. A significant and strong correlation (rho = 0.83) was found between the 26 SS-QOL items that address participation and the total SS-QOL score. Moreover, the evaluation of participation was found to explain 73% of the evaluation of health-related quality of life. Conclusion: The 26 SS-QOL items that address the participation component of the ICF demonstrated adequate reproducibility. Thus, participation, which represents the social aspects of functionality, can be adequately evaluated with these items. Implications for Rehabilitation The 26 Stroke Specific Quality of Life items that address participation proved to be reproducible for the analysis of social participation following a stroke. The findings can lead to a better understanding of the social participation of individuals with chronic hemiparesis and assist in the establishment of adequate treatment for such individuals. The rehabilitation process can be directed towards more specific goals focused on the patient expectations, thereby contributing to greater humanization and effectiveness of treatment to improve social participation following a stroke.


Introduction
The incidence and prevalence of cerebrovascular accident (stroke) are high worldwide. [1,2] It is estimated that a stroke occurs every 40 s, affecting approximately 795 000 Americans per year. [3] In a study on the disease burden throughout the world, stroke was reported to be the third major cause of disability-adjusted life years in developed countries and the sixth major cause worldwide. [4] Thus, stroke was one of the first adverse health conditions to receive the attention of different research groups studying functionality using the International Classification of Functioning, Disability and Health (ICF), [5,6] the model of which is based on the biopyschosocial approach, which is used to address the biological, individual and social dimensions of health. [7] Among the three components of the ICF model (body structures and functions; activity; and participation), the latter is considered the most complex to characterize using existing assessment tools. [8,9] It has therefore been suggested that assessment tools developed to measure healthrelated quality of life (HRQoL) could be used to evaluate CONTACT  social participation following a stroke. [5,10] According to Geyh et al., [11] both HRQoL and participation are patientoriented, which favours the interposed use of these concepts and may be one of the reasons why HRQoL assessment tools are the most indicated for the evaluation of social participation. Silva et al. [12] demonstrated that the Stroke Specific Quality of Life (SS-QOL) questionnaire is the most complete assessment tool for the evaluation of the participation component of the ICF among all HRQoL assessment tools adapted and validated for Brazilian Portuguese, as this questionnaire addresses the greatest number of categories and concepts related to participation in comparison to generic HRQoL assessment tools. [12][13][14] However, only 26 of the 49 items on the SS-QOL address participation, [12] whereas the other items address other ICF categories. The reliability of the 26 SS-QOL items has been tested in a preliminary study, in which adequate measures were reported. [15] However, the study cited involved a small sample and no concordance analysis was reported. Therefore, it is necessary to perform and accurate evaluation of reproducibility (reliability and agreement) to determine whether the measurement properties of these 26 items are adequate.
The primary aim of the present study was to evaluate the reproducibility of the SS-QOL items that address the participation component of the ICF. The secondary aim was to evaluate the correlation between the subscore of the 26 SS-QOL items and the total SS-QOL score to determine whether a relationship exists between the evaluation of participation and HRQoL. The intention is to give scientific credibility to the evaluation of participation and provide important data for the establishment of adequate treatment plans aimed at improving social participation among stroke survivors.

Study design
A cross-sectional study was conducted. Reproducibility procedures were based on the Guidelines for reliability reports and studies Agreement (GRRAS). [16] The sample size was determined in accordance with the guidelines of the Quality Criteria for Health Status Questionnaires, [17] which suggests a minimum of 50 individuals for an appropriate analysis of reproducibility.

Participants
Individuals with chronic hemiparesis stemming from a stroke were recruited from the physical therapy outpatient clinic of University Nove de Julho (Sao Paulo, Brazil).

Eligibility criteria
The inclusion criteria were a clinical diagnosis of primary or recurring stroke more than six months earlier and weakness and/or spasticity in the affected half of the body. Individuals with clinical conditions other than hemiparesis stemming from a stroke, those with motor aphasia and those with impaired cognition, which was screened using the Mini Mental Health Examination and the cut-off points described by Bertolucci et al., [18] were excluded from the study.

Ethical aspects
This study was conducted in accordance with the principles of the Declaration of Helsinki and the regulating guidelines for studies involving human subjects stipulated by the National Health Board of the Brazilian Health Ministry established in December 2012. All participants signed a statement of informed consent and were ensured the possibility of dropping out of the study at any time with no negative consequences. This study received approval from the institutional review board of University Nove de Julho (São Paulo, Brazil) under process number 313.776/13.

Assessment tools and procedures
Due to the sociocultural characteristics of the sample, the decision was made to administer the SS-QOL in interview form with the questions always read in the same order, as recommended in a previous study involving a Brazilian population. [19] Two examiners who had undergone training process involving the theoretical and practical use of the questionnaires carried out the assessment. During the interview, the volunteers also answered a questionnaire addressing social, demographic and clinical aspects for the characterization of the sample with regard to age, sex, schooling, time elapsed since the stroke, type of stroke, number of strokes, affected side of the body and marital status. Each participant then answered the questions regarding participation and HRQoL.
The items on the SS-QOL scale that are specific to the participation component of the ICF were based on a study conducted by Silva et al., [12] who identified 26 of the 49 items on the scale that evaluated participation, distributed among eight domains (family roles, language, mobility, self-care, social roles, thinking, upper extremity function and work/productivity) (supplementary material). The score of each item was performed the same way as the conventional scoring and the total score ranged from 26 to 130 points, with higher scores denoting a greater degree of participation. [12] Reproducibility For the evaluation of inter-rater reproducibility, two examiners independently administered the 26 times that evaluate the participation component of the ICF, with each examiner unaware of the results of the other examiner. For the evaluation of intra-rater reproducibility, one of the examiners administered the 26 items on two separate occasions with a minimum period of 7 d between administrations to avoid the memorization of the answers and a maximum period of 15 d to avoid possible changes in the participation of the individuals. In the period between tests, the participants reported events that could have altered the participation results, such as the occurrence of a fall or the loss of a loved one. Such individuals were automatically excluded from the study to avoid interference with the measurement of reproducibility.

Statistical analysis
Descriptive statistics (mean and standard deviation for quantitative variables and frequencies for categorical variables) were performed for the characterization of the sample and the distribution of the scores. The Kolmogorov-Smirnov normality test was used to determine the distribution of the data.
Reliability and agreement between the measures at the three different evaluation periods were used to analyse the reproducibility of the 26 items that address the participation component of the ICF. [20] The intraclass correlation coefficient (ICC 2,1 ), [21] with respective 95% confidence intervals (CI), was used to analyse reliability and was interpreted as follows: less than 0.40 ¼ poor reliability, 0.40-0.75 ¼ moderate reliability, 0.75-0.90 ¼ good reliability and greater than 0.90 ¼ excellent reliability. [17] For the analysis of intrarater and inter-rater agreement, the standard error of measurement (SEM) and minimum detectable change (MDC) were used. [17] The SEM reflects the error of the assessment tool and was calculated by the ratio between the standard deviation (SD) of the mean of the differences and the square root of 2 (SD of differences/ˇ2). The MDC is the minimum change of the measure that can be interpreted as a real change and was calculated using the following equation: MDC ¼ 1.96 Âˇ2 Â SEM. [17] Inter-rater agreement was also measured using Bland-Altman plots. For such, dispersion diagrams were created to show individual differences (Y axis) in function of the means of the two evaluations (X axis). [22] For the analysis of the degree of linear correlation between the subscore of the 26 SS-QOL items that address participation and the total SS-QOL score, which evaluates HRQoL, Spearman's correlation coefficients (rho) were calculated and interpreted as follows: 0.1-0.3 ¼ weak correlation; 0.4-0.6 ¼ moderate correlation; and 0.7-0.9 ¼ strong correlation. [23] Simple linear regression was used as the measure of validity. For such, the scores referring to participation (26 items) were considered the independent variables and the total SS-QOL score was the dependent variable. Thus, it was possible to determine the degree to which the evaluation of participation predicted the evaluation of HRQoL.
The graphic representation of the Bland-Altman plots was created using the MedCalc Statistical Software (MedCalc. Inc., USA). All other analyses were performed using the SPSS para Windows (SPSS Inc., Chicago, IL). The alpha risk for all analyses was set to 0.05.

Results
A total of 123 individuals with hemiparesis stemming from a stroke were recruited, 23 of whom were excluded for cognitive deficit and 19 were excluded for aphasia. Thus, the sample was made up of 81 stroke survivors. Table 1 displays the clinical and demographic characteristics of the sample. Six individuals were excluded during the analysis of reproducibility (one for losing a loved one and five for not returning to the second evaluation between the initial test and the retest within seven to 15 d).
Regarding reproducibility, excellent degrees of reliability were found: intra-rater ICC 2.1 ¼ 0.96; inter-rater ICC 2.1 ¼ 0.95. In terms of agreement, the SEM was 3.50 and 4.17 and the MDC was 5.18 and 5.66 in the intrarater and inter-rater analyses, respectively ( Table 2). Figure 1 illustrates intra-rater and inter-rater agreement. Comparing the mean of the differences in the measurements obtained from different examiners, symmetrical distribution was found around the mean line, but with broad agreement limits and a high degree of bias, especially in the inter-rater analysis.
A strong positive correlation was found between the scores on the 26 items that address the participation component of the ICF and the total SS-QOL score. The same was found for the language, mobility, self-care, social roles, upper extremity function and work/productivity domains. Moderate correlations were found for the family roles and thinking domains ( Table 3).
The simple linear regression analysis revealed a strong predictive value (r 2 ¼ 0.73), which means that the evaluation of participation predicted 73% of the evaluation of HRQoL. The same was true for the separate SS-QOL domains, except family roles and thinking, which exhibited moderate predictive values (Table 3).

Discussion
The primary aim of the present study was to analyse the reproducibility of the 26 SS-QOL items that address participation following a stroke and the secondary aim was to analyse the correlation between the subscore of these 26 items and the total SS-QOL score. The analysis of the findings revealed adequate reproducibility and a strong correlation between the subscore of the 26 items and the total SS-QOL score.
Reproducibility (reliability and agreement) is defined as the capacity of an assessment tool to offer consistent results when used by different researchers or at different points in time. [24] Reproducibility studies are crucial to determining the variability in a method or assessment tools to avoid the misinterpretation of variables analysed before and after an intervention. Specifically with regard to reliability, the ICC values for intra-rater and inter-rater agreement were excellent in the present study, demonstrating that the 26 SS-QOL items are reliable for measuring the participation component of the ICF. Intra-rater and inter-rater agreement was evaluated using the SEM and MDC. [17] The small SEM encountered signifies a variation of 3.50 points (intra-rater) and 4.17 points (inter-rater) when the measure is administered to the same individual on two separate occasions, which is related to the error of the measure rather than a change in the clinical condition of the patient. The MDC encountered signifies that a change greater than 5.18 points (intra-rater) and 5.66 points (inter-rater) has less than a 5% probability of being due to chance or random error of the measure. Lin et al. [25] report a variation of 5.9, 4.0 and 5.3 points on the mobility, self-care and upper extremity function subscales, respectively.
Intra-rater and inter-rater agreement was evaluated visually using Bland-Altman plots, which demonstrated symmetrical dispersion around the mean line, but with broad bounds and a high degree of bias, especially in the inter-rater analysis. The Bland-Altman plot has been used in different reliability studies. [20] However, it is not possible to compare the results of the present study, as no previous articles were found in the literature using this method to analyse the reliability of the SS-QOL for the evaluation of social participation among stroke survivors.
Regarding the secondary objective of the present study, a strong, positive correlation was found between the subscore of the 26 items that address participation and the total SS-QOL score. Moreover, the 26 items that address participation can strongly predict HRQoL evaluated by the total SS-QOL score and the same was found for the individual domains of the questionnaire, except the family roles and thinking domains, which had moderate predictive values. Thus, both instruments evaluate similar measures.
However, it should be stressed that participation and HRQoL are distinct constructs. While the former regards the involvement of an individual in real-life situations, [7] the latter regards restrictions associated with a given adverse health condition. [11,25,26] The term HRQoL refers to the perception one has regarding one's disease and its effect on one's life, including personal satisfaction with one's physical, functional, emotional and social well-being. [27] Participation is related to one's set of skills and performance in daily living to participate in a social context, considering environmental and personal factors as either facilitators or barriers. [7] Thus, further studies are needed for a more in-depth understanding of the construct validity of these items.
Although it was not the aim of the present study to investigate possible determinant factors for  participation, the stratification of the sample based on the level of independence and motor impairment could affect the results. Despite this limitation, the 26 SS-QOL items that address participation proved to be reproducible for the analysis of social participation following a stroke in a sample with different functional levels.
The present findings are of extreme importance to the field of physical therapy and the rehabilitation of stroke survivors. These findings can contribute to the development of effective evaluation strategies and provide important data for the establishment of adequate treatment plans for this population. Thus, the rehabilitation process can be directed towards more specific goals focused on the patient expectations, thereby contributing to greater humanization and effectiveness of treatment to improve social participation following a stroke.

Conclusions
Based on the findings of the present study, the 26 items of the SS-QOL that address the participation component of the ICF demonstrated adequate reproducibility (reliability and agreement). Thus, participation, which represents the social aspects of functionality, can be adequately evaluated with these items. Moreover, the planning of treatment strategies and the follow-up of participation after a stroke can be monitored in a more realistic fashion.