Purpose Test equating studies in medical education have been conducted only for high-stake exams or to compare two tests given in a single course. Based on item response theory, we equated computer-based test (CBT) results from the basic medical education curriculum at the College of Medicine, the Catholic University of Korea and evaluated the validity of using fixed passing scores.
Methods We collected 232 CBTs (28,636 items) for 40 courses administered over a study period of 9 years. The final data used for test equating included 12 pairs of tests. After test equating, Wilcoxon rank-sum tests were utilized to identify changes in item difficulty between previous tests and subsequent tests. Then, we identified gaps between equated passing scores and actual passing scores in subsequent tests through an observed-score equating method.
Results The results of Wilcoxon rank-sum tests indicated that there were no significant differences in item difficulty distribution by year for seven pairs. In the other five pairs, however, the items were significantly more difficult in subsequent years than in previous years. Concerning the gaps between equated passing scores and actual passing scores, equated passing scores in 10 pairs were found to be lower than actual passing scores. In the other two pairs, equated passing scores were higher than actual passing scores.
Conclusion Our results suggest that the item difficulty distributions of tests taught in the same course during successive terms can differ significantly. It may therefore be problematic to use fixed passing scores without considering this possibility.
PURPOSE This study aims at to make intermediate-term evaluations of a curriculum by investigating its development, operation and outcomes 2 years after its revision.
METHODS A survey using 5-point-Lickert scale questionnaires was given to the group of directors who developed the curriculum, instructors who only used it in their classes, and a student group from the first and second grades. Focus group interviews were performed in the professor groups.
RESULTS Curricular reform was evaluated as being systematic, democratic, and positive in general. Both groups answered questions positively about the relevance of the integrated curriculum and introduction of clinical medicine (ICM), graded as 3.4 (professor) and 3.5 (student). As for problem-based learning (PBL) and the patient-doctor-society (PDS), the professor group responded more positively than students. The 'web-based learning center' was recognized positively by many more students (4.01) than professors (2.75). With regard to the education outcome, professors gave 3.3 points and students 3.5 to an item that asked 'whether students attained the learning goal or not?' Professors, through interviews, showed their satisfaction with the attempt to reform the curriculum, but they pointed out that long-term evaluations should be performed.
CONCLUSION The interim evaluation of the revised curriculum, from its planning to its effects, affirmed by several suggestions to be successful in the long run through 1) enhancement of systematic participation and communication, 2) further integration, 3) steady evaluation, 4) greater effort on professional development, and 5) active interaction between professors and students.
Citations
Citations to this article as recorded by
Validation of criteria for evaluating competency-based curriculum in medical schools using the Delphi hierarchy process method So-Young Lee, Seung-Hee Lee Korean Journal of Medical Education.2025; 37(3): 247. CrossRef
Evaluation of Medical Humanities Course in College of Medicine Using the Context, Input, Process, and Product Evaluation Model So Young Lee, Seung-Hee Lee, Jwa-Seop Shin Journal of Korean Medical Science.2019;[Epub] CrossRef
Basic research to guide alterations in an outcome-based curriculum Hyo Hyun Yoo Korean Journal of Medical Education.2014; 26(4): 265. CrossRef
A Survey of Student Satisfaction with a Portfolio Process and Assessment Dong-Mi Yoo, Jae-Jin Han, Eun-Kyung Eo Korean Medical Education Review.2014; 16(1): 42. CrossRef
The Satisfaction of the Freshman of Graduate Medical Students to the New Medical Curriculum: One Medical School Jong Park, Young Joon Ahn Korean Journal of Medical Education.2011; 23(2): 95. CrossRef
Happiness of Medical Students and Tasks in Medical Education Eun Bi Lim Korean Medical Education Review.2011; 13(2): 25. CrossRef