Validation and Evaluation of an OSCE in Undergraduate Doctor of Pharmacy Program

Abstract:

Introduction and purpose: In schools not performing OSCE method, faculty members’ lack of time, human resources and managing the complexity of such exam are the most common obstacles. In this study, we utilized different methods to evaluate the the validity and reliability of a simple diversified OSCE as part of final exam for ambulatory care clerkship along with the theoretical exam. Methods: We analyzed the correlation and difference between students’ achievements in the written exam and their achievements in OSCE. We also compared between students’ achievements according to each OSCE station they enrolled in. As students’ feedback is an important indicator of the success of the learning process, students’ feedback about OSCE was also measured. Results: A moderate correlation was observed between students’ overall marks in OSCE and that in written exam (r² = 0.36 p= 0.001). Prescription station showed a moderate correlation with the written exam (r² = 0.30, p < 0.001) and the counseling station showed a weak correlation (r²= 0.125, p < 0.001). Whereas no significant correlation (p= 0.3) was observed on comparison of the written exam with the online station nor with the communication station. Highest students scores were observed in prescription-based station 16.7/20 (± 3.2) whereas the lowest scores were detected in online resourcesbased station 13.4/20 (± 5.8). The majority of students were satisfied about the validity and reliability of OSCE. Discussion: A carefully designed development process based on a well-structured blueprint is needed to implement a sophisticated OSCE. The validation and evaluation of an OSCE should performed both quantitatively and qualitatively. A simple diversified OSCE, can serve as a reliable, valid and economically feasible assessment tool of student performance. Together, the traditional written exam and OSCE provide a broad assessment of pharmacy student’s skills and knowledge.

Key words: OSCE, Doctor of Pharmacy, ambulatory care clerkship, clinical skills, written exam.