Prior participation as a standardized patient improves OSCE scores of third-year medical students: A pilot comparative study at Université Paris Cité Medical School

Abstract Objective Objective structured clinical examinations (OSCE) are one of the main modalities of skills’ assessment of medical students. We aimed to evaluate the educational value of the participation of third-year medical students in OSCE as standardized patients. Methods We conducted a pilot OSCE session where third-year students participated in sixth-year students’ OSCE as standardized patients (cases). Their scores in their own subsequent OSCE exams were compared with third-year students who had not participated (controls). Students’ perceptions (stress, preparedness, ease) regarding their OSCE were compared with self-administered questionnaires. Results A total of 42 students were included (9 cases and 33 controls). Median [IQR] overall score (out of 20 points) obtained by the cases was 17 [16.3–18] versus 14.5 [12.7–16.3] for controls (p < 0.001). Students’ perception of their evaluation (difficulty, stress, communication) was not significantly different between cases and controls. Most cases agreed that their participation was beneficial in reducing their stress (67%), increasing their preparedness (78%) and improving their communication skills (100%). All cases agreed that this participation should be offered more widely. Conclusion Students’ participation in OSCE as standardized patients led to better performance on their own OSCE and were considered beneficial. This approach could be more broadly generalized to improve student performance.


Introduction
Objective Structured Clinical Examinations (OSCE) are one of the flagship tools to assess medical students' skills (Sloan et al. 1995).This concept, which has been used for many years in Canada and Switzerland and more recently in France (Nuzzo et al. 2020;Matet et al. 2021), is based on placing students in a clinical situation with a standardized patient or participant (SP) and an evaluating observer.This allows for the evaluation of multiple clinical, technical, relational, or ethical capacities in a standardized setting.OSCE is intended to be reproducible, fair, and effective in assessing student's organizational and technical competencies, but also in testing their medical soft skills (Casey et al. 2009;Brannick et al. 2011).
This evaluation method is currently being deployed mainly for French medical students in their last two years of medical school (fifth-and sixth-year students) with a final competitive national exam including OSCE, the first of which will take place in 2024.However, to date, OSCE has hardly been used to evaluate younger students, particularly

Practice points
Objective structured clinical examinations (OSCE) are one of the main modalities of skills assessment for medical students.Early participation in OSCE of third-year medical students as standardized patients resulted in better scores in their own subsequent OSCE.Most students claimed that their participation as standardized patients was beneficial in reducing their stress, increasing their preparedness and improving their communication skills those in the second or third year, even if several studies have supported the early introduction of practical training to improve communication skills in undergraduate medical students (Marcus et al. 1994;Nuzzo et al. 2020).
In this context, preparing students as soon as possible for this evaluation is a priority for French medical schools.Thus, the idea of asking younger medical students to participate in OSCE as SPs in OSCE of older students may come with a pedagogical interest for them (Heinke et al. 2013;Kim et al. 2020;Schwill et al. 2020).Indeed, we hypothesized that early participation in OSCE as SP could add pedagogical value for younger students, improving their performance on their own OSCE assessments, as well as their preparedness, confidence, and ease.The primary objective of this study was to compare the performance of third-year (3Y) medical students who had previously participated in OSCE assessments as SP to those of students who had never participated in OSCE as SP.The secondary objectives were to compare students' perceptions (stress, preparedness, ease) about their own OSCE assessments.

Study population
Our study was conducted among 3Y medical students of Universit e Paris Cit e Medical School (UPCMS), carrying out their elective clinical placements in three medical departments (internal medicine, geriatrics and emergency departments) of one of the three AP-HP.Nord University Hospitals (UPCMS network, including Bichat, Beaujon and Bretonneau hospitals).A collaboration has been initiated since 2021 between these departments for the organization of common OSCE for the sixth-year (6Y) medical students, with three OSCE stations (Supplementary Appendix 1) at each session.The present study was conducted for 6Ystudents' OSCE conducted in February and April 2021 (Figure 1).During each session, 6Y-students were divided into two groups on three identical stations assessing different skills, with a medical teacher as the evaluator.
3Y-students, who were doing their clinical elective at the same time in the same departments, were offered to participate on a voluntary basis in 6Y-students' OSCE evaluation as SP.These 3Y students were given previous training by the investigators including an explanation of the different stations, including instructions given to candidates and directives to play their role as SPs (Supplementary Appendix (1).For both OSCE sessions, 3Y-students had the opportunity to participate in all three stations and thus could play three different roles as SP.These 3Y-students were further referred to as cases.
Controls were selected among 3Y-students of the same cohort, who had not participated in 6Y-students' OSCE as SP and were doing their clinical electives in the same departments, or in one of the following: Gastroenterology department (Beaujon Hospital), Emergency Department (Bichat Hospital).3Y-students who had already been evaluated by OSCE were excluded from the study.

Outcomes
For all 3Y-students of the UPCMS, a 3Y-students' OSCE assessing semiology skills was organized in May 2021 (Figure 1, Supplementary Appendix (2).The primary outcome was the score obtained at the 3Y-students' OSCE, assessed by non-blind evaluators, with a standardized evaluation grid (Supplementary Appendix (3).All evaluators were university faculty, trained and familiar with OSCE.
The secondary endpoints were the overall perception of the 3Y-students during their OSCE (stress, preparedness, ease).After taking the 3Y-students' OSCE, cases and controls were asked to fill out a non-anonymized questionnaire, designed by the investigators to collect 3Y-students' perceptions, using Likert scales (Figure 3).The questionnaires received by cases also included specific questions about their perception of their participation in 6Y-students' OSCE as SP, and whether they felt it was beneficial from an educational point-of-view (Figure 4).
Each 3Y student gave written consent to participate in this study.De-identification before analysis was performed to ensure that no nominative data were disseminated.The data were then stored on a secured server.

Statistical analysis
OSCE scores were compared between cases and controls at each station and overall, using Mann-Whitney tests.Quantitative variables are expressed as a median and interquartile range in square brackets and qualitative variables as numbers and percentages in brackets.Secondary endpoints, assessed by Likert scale responses, were analyzed by analyses of variance.All analyses were performed with R software version 4.1.1(R Foundation for Statistical Computing, Vienna, Austria), using the Likert library.A p-value <0.05 was considered significant.

Study population
A total of nine 3Y-students participated as SPs during the 6Y-students' OSCE (Table 1).At the time of their participation, these 3Y-students were doing their elective placement in the Internal Medicine departments (N ¼ 9) of Beaujon Hospital; 6 (66.7%) of them were female and the median age was 21 years .These cases were compared to 33 3Y-students controls, without SP experience (Table 1).Overall, 42 students (9 cases and 33 controls) were included in our study.

Secondary outcomes
Perception of cases compared to controls regarding their 3Y-OSCE 3Y-students' perceptions about their own OSCE, assessed by a questionnaire, are presented in Figure 3.When comparing cases to controls, there was no significant difference for each of the questions, particularly regarding their appreciation of the difficulty of the stations (p ¼ 0.963) and their level of stress (p ¼ 0.534).No case felt uncomfortable communicating with the patient (Question 3; 0% for cases vs. 15% for controls) and very few felt difficulties concerning the duration of the stations (Question 5; 22% for cases vs. 48% for controls), although differences between the groups were not statistically significant.

Perception of cases regarding their participation in the 6Y-student's OSCE as SP
Cases' perception about their participation in 6Y-student's OSCE as SP is shown in Figure 4. Most had a positive opinion regarding their participation, with no difficulties in   playing the role of SPs.All of them found that it helped them identify difficulties encountered by evaluating 6Ystudents.
Most students somewhat or strongly agreed that this participation was beneficial in (i) reducing their level of stress (67%); (ii) increasing their preparedness for their own OSCE (78%) and (iii) improving their communication skills (100%).Finally, most of them believed it gave them an advantage over other students, who had not participated (89%).All cases strongly or somewhat agree that this participation should be offered more widely.

Discussion
This pilot study showed a potential educational benefit of the participation of 3Y-students as SP in OSCE to improve performance in their own OSCE.Additionally, most students who participated in OSCE as SP found that such participation was beneficial and should be systematically offered, although their perceptions in terms of stress and capacity to communicate were not different to that of students without SP experience.
To date, few studies in the literature assessed the benefit of students' participation in OSCE as SP.Some works have focused on the quality of peer's evaluation by students during OSCE and found no difference between senior evaluators, with overall student satisfaction (Kim and Kim 2020;Schwill et al. 2020) Similarly, training programs in the form of OSCE organized by older medical students were well perceived by younger medical students (Taylor et Quick 2020;Braier-Lorimer and Warren-Miell 2022), and participation in OSCE case developments was considered beneficial by students (Cosker et al. 2021).Our study was unique in that younger, undergraduate 3Y-students, under the responsibility of their teachers, participated in the OSCE of their older peers, as SP.
It is interesting to note that these students performed better in their own SCE evaluations than students who had not participated in the program.This highlights the benefit of training younger students in OSCE to perform better.However, the causality link between these higher scores and students' previous participation as SP should be interpreted with caution.Indeed, many biases could be involved.First, the selection of cases and controls was not random, and the cases were selected on a voluntary basis.This may introduce bias, as the cases were likely more motivated and involved in their studies, which may select better students who would have performed better on their OSCEs, regardless of their participation as SPs.In addition, as 3Y-students were evaluated for the first time with OSCE, this could have increased the advantage for cases, as they had a better idea of how OSCE's were conducted, teachers' expectations and the importance of time management.The differences found here in terms of scores may be mitigated once all students will have been regularly evaluated by OSCE.Also, evaluators were not blind to 3Y-students' condition.However, this bias should be mitigated by the use of a standardized scoring grid, limiting subjective evaluator bias.Finally, even if the topics of 6Y-and 3Y-OSCE were different, preventing cases from getting good marks solely because of the knowledge gained during their SPs experience, they could have had an advantage in generally expected skills (such as behaviour, hand washing, and general interrogation of the SPs).This could potentially contribute to the observed difference with controls.
Somewhat surprisingly, we found no differences in students' perception concerning their own OSCE, particularly in terms of their perception of the test (difficulties, stress level, ease).As it was their first OSCE, most third-year students acknowledged a high level of stress before the evaluation, regardless of whether they had already participated in OSCE as SP.Nevertheless, there was a tendency for cases to benefit in terms of communication and time management skills, although not significantly.
Finally, most of the cases were satisfied with their participation in OSCE as SP and student feedback very positive.For instance, they were able to see if 6Y-students were struggling or, on the contrary, were very comfortable during their OSCE.This gave them examples of models to emulate, or on the contrary to avoid.In addition, all cases thought that such participation as SP should be offered more widely.
We acknowledge some limitations to our study, including the small number of participating students, preventing multivariable analyses.Moreover, cases and controls were not doing their elective in the same departments, which may have impacted their formation and their own OSCE evaluations.In addition, feedback questionnaires were not anonymized, there may be a potential response bias.
Altogether, and despite these biases, our study shows that 3Y-students' participation as SP seems to have pedagogical benefits as well as is appreciation by the students.More data is needed to determine precisely the long-term educational benefits of the participants as SP for medical students.Consequently, in light of this pilot study, an OSCE training course will be created at UPCMS to train 2Y and 3Y-students to play the role of SP during OSCE and participate in the OSCE evaluations of 6Y-students more widely.We will then evaluate this new program by comparing the performances of the participating students with those of students not participating in peers' OSCE, at a much larger scale.We also plan to assess the sustainability of the potential benefit after several OSCE evaluations.
In conclusion, our pilot study demonstrated the feasibility of having younger students participate in OSCE as SPs, with both potential improvements in students' skills and satisfaction.

Figure 2 .
Figure 2. Comparison of third-year students' OSCE scores between cases and controls (N : 42; including 9 cases and 33 controls).OSCE: Objective structured clinical examinations.

Figure 3 .
Figure 3. Comparisons of appreciations and feelings of third-year medical students regarding their OSCE evaluations (N ¼ 42).OSCE: Objective structured clinical examinations; NS: non-statistically significant.

Table 1 .
Characteristics of third-year medical students included in the studies (N ¼ 42).Results are presented as number (percent) or median[IQR]or mean (SD).3Y-students:third-year medical students; SP: standardized patients, OSCE:Objective structured clinical examinations.