| Home | E-Submission | Sitemap | Contact us |  
top_img
Korean J Med Educ > Volume 28(1); 2016 > Article
Jaiprakash, Min, and Ghosh: Increased correlation coefficient between the written test score and tutors’ performance test scores after training of tutors for assessment of medical students during problem-based learning course in Malaysia

Abstract

This paper is aimed at finding if there was a change of correlation between the written test score and tutors’ performance test scores in the assessment of medical students during a problem-based learning (PBL) course in Malaysia. This is a cross-sectional observational study, conducted among 264 medical students in two groups from November 2010 to November 2012. The first group’s tutors did not receive tutor training; while the second group’s tutors were trained in the PBL process. Each group was divided into high, middle and low achievers based on their end-of-semester exam scores. PBL scores were taken which included written test scores and tutors’ performance test scores. Pearson correlation coefficient was calculated between the two kinds of scores in each group. The correlation coefficient between the written scores and tutors’ scores in group 1 was 0.099 (p<0.001) and for group 2 was 0.305 (p<0.001). The higher correlation coefficient in the group where tutors received the PBL training reinforces the importance of tutor training before their participation in the PBL course.

Introduction

The goals of problem-based learning (PBL) are to develop flexible knowledge, efficient problem solving skills, self-directed learning skills, efficient cooperation skills, and intrinsic motivation [1]. To ascertain that the goals of PBL have been achieved, we used two kinds of scores—the written test score and tutors’ performance test score—in a study at MAHSA University Faculty of Medicine, Malaysia. Those items were to assess the overall performance of the students in a PBL session. This study was aimed at finding if there was a change of correlation between the written test score and tutors’ performance test scores after the tutors were trained for the PBL course for medical students.

Case report

1. Subjects and methods

This was an observational study, conducted from November 2010 to November 2012, on two groups of year 2 medical students. Group 1 consisted of 127 students and group 2, 137 students. In group 1, tutors did not receive the training for PBL; while, in group 2, the tutors received the training. Tutors’ scores assess the students’ performance based on soft skills such as communication skills, analytical skills, and presentation skills. The written test score is comprised of the scores obtained from the mini test which is given in the last session of the PBL in the form of multiple true or false questions designed to test the knowledge gained by the students on the case discussed in the PBL session. We retrieved the PBL scores and the end-of-semester scores of students after obtaining permission from the dean of the medical school. Students of each group were also divided into low, middle, and high achievers based on the grades obtained in the end-of-semester exams. Low achievers were students who got marks less than 60%; middle achievers, between 60% and 74%; and high achievers, equal to or greater than 75%. Data were compared by analysis of variance. Pearson correlation coefficient was obtained between the two kinds of scores in each group using SPSS version 19.0 (IBM Corp., Armonk, USA).

2. Results

The written test score and the tutors’ score were the highest in the high achievers for both of the groups whereas the tutors’ score was high for the middle achievers in group 2. Scores between different achievers in both groups were found to be statistically significant except for the tutors’ score in group 1 (Table 1). The correlation coefficient between the written scores and tutors’ scores in group 1 was 0.099 (p<0.001) and for group 2 was 0.305 (p<0.001).

Conclusion

As expected, all the scores were higher in the high achievers group rather than in the middle or low achiever groups. It is anticipated that high achievers performed well in two of the aspects of the PBL, not only knowledge gained but also performance level. When we compared the written test score and tutors’ score among the different achievers, there was a statistically significant difference in both the groups, except the tutors’ score in group 1 (Table 1). We found that this result could be because the tutors in group 1 were not well informed about the scoring in the PBL session at our university.
In another study conducted on the PBL assessment, it was found that a structured assessment pattern for PBL improved student participation, because the students and the tutors knew what was expected from the session. This also helped to standardize the PBL assessment pattern [2]. A study conducted in Malaysia on the PBL assessment instrument showed not much variability in the scores between the tutors [3]. In this study, we tried to determine the correlation between two kinds of scores in both the groups of students. We found a higher correlation between two kinds of scores in group 2. This implies that assessment by the tutors in group 2 was more stable than in group 1. This could be attributed to the fact that tutors underwent a PBL facilitation training before they took the PBL session for group 2. In spite of the tutors being the same for group 1 and group 2, when they were facilitating for group 1, they had no formal training on how to facilitate and assess a PBL session. Assessments of medical students in PBL are examples of assessments that depend on consistency of the raters [4]. Therefore, just one form of assessment may be not a very reliable tool for assessing a PBL session. A need for an assessment which combines the written test score the tutors’ performance test scores is essential. The preparedness of the tutors on how to assess and facilitate a PBL session is one of the important aspects we need to look into for a PBL-based curriculum.
In conclusion, scores in group 2 are statistically significant among three types of achievers. Group 2 has a better correlation between the scores than group 1 so that assessment by the tutors in group 2 can be said to be more stable than for group 1. Since tutors for group 2 received PBL training, the importance of tutor training should be stressed in this university.

Acknowledgments

We would like to thank all the medical students who participated in the study. We would also like to thank the dean, deputy dean, faculty of medicine MAHSA University, Kuala Lumpur, Malaysia and the management for giving us the permission to conduct the study.

Notes

Funding
None.
Conflicts of interest
None.

Table 1.
Mean Scores of the Problem-Based Learning Course in Medical Students of MAHSA University Faculty of Medicine, Malaysia
Group 1
Group 2
No. of students Means scores±SD p-value among achievers No. of students Means scores±SD p-value among achievers
Written test score
 Low achiever 86 61.1±14.0 93 53.2±10.1
 Mid achiever 26 69.5±5.7 33 59.0±8.5
 High achiever 14 73.3±5.2 0.000 11 68.0±5.7 0.000
Tutors’ score
 Low achiever 86 72.2±14.0 93 70.1±22.8
 Mid achiever 26 75.7±9.8 33 83.6±6.7
 High achiever 14 78.9±7.8 0.128 11 80.9±14.8 0.002

SD: Standard deviation.

References

1. Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educ Psychol Rev 2004; 16: 235-266.
crossref
2. Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001; 357: 945-949.
crossref
3. Harden RM. Assess students: an overview. Med Teach 1979; 1: 65-70.
crossref pmid
4. Downing SM. Reliability: on the reproducibility of assessment data. Med Educ 2004; 38: 1006-1012.
crossref pmid
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-747-6206
E-mail : kjme_office@daum.net
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2021 by Korean Society of Medical Education.                 Developed in M2PI