| Home | E-Submission | Sitemap | Contact us |  
top_img
Korean J Med Educ > Volume 31(4); 2019 > Article
Swatan, Prihatanto, Rehatta, and Atika: The outcome evaluation between two different curriculum: students’ retention capacity based on comparative study in Indonesia

Abstract

Purpose

To compare the outcome of two different curriculum in clinical pathology using students’ retention capacity.

Methods

A cross-sectional study was conducted in December 2017. There were two groups of students enrolled in clinical pathology course. A retention test therefore was conducted 5 months after the course had finished using 25 multiple choice questions –single best answer. Comparisons of means are determined by Mann-Whitney U-test using SPSS version 23.0 (IBM Corp., Armonk, USA).

Results

A total of 280 students participated in the study of a 5-month post-course evaluation, and the average for the first group (n=138) is 10.93±3.57 while the second group (n=142) average is 8.56±3.19. The difference between the first and the second group are statistically significant (p<0.00001).

Conclusion

The group with longer duration and more number of parallel courses had a higher retention score. Further research using retention test in different courses should be conducted to analyze which curriculum that promotes students’ understanding of medical knowledge.

Introduction

Medicine is a vast and continuously evolving knowledge with a concise medical curriculum required to support the learning process of medical students. In Indonesia, an undergraduate medical student must undergo a minimum of 11-semester course in medical sciences, consisted of seven-semester preclinical and four-semester clinical study. During the preclinical learning, there are two different teaching methods, which are, the old and new curriculum. The old curriculum consisted of full semester courses in conjunction with several integrated module blocks. The new curriculum consisted of clustered courses (several inter-correlating courses delivered in the allocated time) and organ-based blocks [1].
Majority of medical schools in Indonesia also follow the new curriculum due to the change of regulations by Indonesian Medical Council with Association of Indonesian Medical School in 2012, which requires both horizontal and vertical integration in at least 50% of the its curriculum [1]. It is argued that the recent changes in the medical curriculum are made to enhance students’ understanding the courses. However, there are no evidence to support this claim. Moreover, there are no evaluation studies to proof that the new curriculum gives a better outcome than the old one.
Curriculum evaluation is a necessary and important aspect of any educational system because it provides the basis for its policy decisions, for feedback on continuous adjustments and processes of its implementation. Student assessment is an important aspect of curriculum evaluation because it measures the learning quality and fundamental success of any student [2]. One of the parameters that could be used to evaluate the students learning quality is retention capacity. It reflects on the long-term memory of the student, which results in the understanding of course materials. It differs from the short term memory achieved through short term memorization and easily lost when left unused or replaced by a piece of new information [3,4].
Considering that there was no evaluation study of the old and new curriculum. This study, therefore, aims to determine which curriculum type has better outcome in terms of clinical pathology retention capacity.

Methods

This was a cross-sectional study conducted in December 2017 in one of the public universities in Indonesia. Subjects were undergraduate students at the faculty of medicine who took part in the clinical pathology course in 2017. There were two different groups in this study. The first group consisted of all 3rd year medical students who was taught using the old curriculum and willing to participate in this study. The second group consisted of all 2nd year who were taught using the new curriculum and willing to participate in this study. Exclusion criteria for this study were: (1) students who retake the clinical pathology course, (2) students that did not attend the clinical pathology examination, (3) student that did not attend the retention test, (4) students that was found cheating during the retention test, and (5) students that did not complete the retention test.
The first group took the clinical pathology course for 14 weeks (February–June 2017), parallel with seven other courses (microbiology, parasitology, philosophy, integrated module of ethics and medical law, integrated module of tropical medicine, research, and medical skills 4). The amount of credits given for completing this course is four credits and course assessment is conducted 3 times, two written examinations consisting of multiple-choice questions–single best answers (MCQ-SBA) with five options and one practical examination.
The second group took the clinical pathology course for 6 weeks (May–June 2017), along with one other course (anatomical pathology). Three credits are given upon completion and course assessment is conducted 2 times, which are, one written examination of MCQ-SBA with five options in it and one practical examination.

1. Retention test

A retention test was conducted 5 months after the course had finished. Items for the retention test covered all information points throughout the clinical pathology course. It was pre-validated by the department of clinical pathology and post-validated through the means of item analysis using the formula from the book of medical education [5,6]. We used MCQ-SBA items with five options due to the students’ familiarity with the instrument and its superiority in determining one’s understanding and its convenience in item analysis with a high-reliability index [7]. Initially, there were 30 MCQ-SBA items with five options. After validation and primary pilot testing, only 25 items were considered valid and used in this study.
The retention test was carried out using computer-based test method consisted of 25 MCQ-SBA with five options. A correct answer worth one mark and there were no penalties for incorrect answer. The time allotted for this test is 25 minutes.

2. Statistical analysis

Acquired data was analyzed using IBM SPSS Statistics for Windows ver. 23.0 (IBM Corp., Armonk, USA). Data were expressed as mean±standard deviation. Data distribution was analyzed using one-sample Kolmogorov-Smirnov test. Comparison of scores between two groups was analyzed using Mann-Whitney U-test, p-value of <0.05 was considered as statistically significant. Reliability score was measured using Kuder-Richardson Formula 20 (KR-20).

3. Ethical clearance

This study follows the principles of the Declaration of Helsinki and had received ethical clearance from Faculty of Medicine Universitas Airlangga before it began (ethical clearance no., 297/EC/KEPK/FKUA/2017). All subjects gave their informed consent prior to their inclusion in the study. Details for informed consent was given before subjects signed the it. Details that might disclose the identity of the subjects under study were omitted.

Results

A total of 438 students (234 3rd and 204 2nd year medical students) were enrolled in the clinical pathology course in 2017. From all its students, only 288 (146 3rd and 142 2nd year medical students) were willing to participate in this study. Eight students from the first group were excluded because the students did not complete the retention test. Final samples for this study were 138 students from the first group (49 males and 89 females) and 142 students from the second group (53 males and 89 females).
A total of 280 responses were collected and validated. Mean difficulty index (P) was 39.39±16.91 while mean discrimination index (D) was 0.34±0.14. The data showed a wide distribution in both difficulty index (5.26 to 75.66) and discrimination index (0.08 to 0.63). None of the items were left unanswered by the students.
Based on difficulty index, items were categorized into three groups (difficult P <30%, acceptable P 30%–70%, and easy P >70%). In this test, there were seven difficult items, 16 acceptable items, and two easy items.
Based on discrimination index, items were categorized into four groups (excellent discrimination D >0.40, good discrimination D 0.30–0.39, acceptable discrimination D 0.20–0.29, and poor discrimination D 0.00–0.19). There were eight items with excellent discrimination index, seven items with good discrimination index, seven items with acceptable discrimination index, and three items with poor discrimination index. There were no items with negative discrimination index.
The average of the first group (n=138) was 10.93±3.57 with the score ranging from 2 to 20. Meanwhile, the average score of the second group (n=142) was 8.56±3.19 with the score ranging from 1 to 17 (Table 1). Based on the difficulty level, the students in the first group had a higher correct answer percentage in all level of difficulties compared to the second group (p<0.05 in a difficult and acceptable level, p>0.05 in easy level) (Table 2).
One sample Kolmogorov-Smirnov test for normal distribution showed that data was not normally distributed (p<0.0001). Comparison test using Mann-Whitney U-test showed a significant difference in score between the two groups (mean rank, 167.44 versus 114.32; Z-score, -5.509; p<0.0001). The reliability score using KR-20 formula was 0.62.
We also compare the percentage of decrease in students’ retention test result from the post-course written examination test result. The first group averaged at 34.02±19.74, while the second group averaged at 41.90±20.61 decrease in retention test result compared to the post-course written examination test result. Comparison test using Mann-Whitney U-test showed a significant difference between the two groups (mean rank, 124.50 versus 156.05; Z-score, -3.259; p=0.001).

Discussion

There is no study that had been conducted to compare the retention capacity between the old and new curriculum. This study found that retention capacity in the first group is higher than the second group (10.93±3.57 versus 8.56±3.19, p<0.0001). In addition, the percentage of decrease on retention test result in the first group is lower compared to the second group (34.02±19.74 versus 41.90±20.61, p=0.001). These differences might be due to: (1) amount of load and time given during courses, (2) number of parallel courses given, and (3) variations in factors that affects individual learning experience.
One might argue that the significant difference from both groups is because the discrepancy in the credit earning points. Even so, the outcome and the material given is the same. The difference in credit made the same amount of material learned in a shorter amount of time. However, when we compare the percentage of decrease on retention test result of both groups it is still significantly different. In this case, the same amount of material given in shorter time will lead to reduced retention. These findings supported the cognitive load theory which demonstrates that the brain only has a limited capacity to store information from working memory, called the cognitive load, to be further processed into long-term memory. After achieving the maximum load of information, the input received will be either forgotten or abandoned. This is where repetition plays a role. It prevents information to be either forgotten or abandoned by transforming the working memory into long-term memory [8].
Variation of courses that are given at the same time may also affect the retention of course materials. Ideally, courses material should be integrated both horizontally and vertically. This may involve lecturers from across disciplines to talk more on the same topic or organs but using different approaches based on their department or specialty. In group 1, several basic medical science courses are given during a full 14 weeks, in addition to the integrated module. This type of courses allows the student to integrate the material as shared commonalities among subjects and focused on finding the connection among topics and interest to see the whole picture of a subject [9].
Other factors that may affect a student academic performance is the student’s learning experience. It is known that learning is an individualized process which is dependent on a person’s experience. Several factors that may affect learning experience include (1) individual factors such as motivation, regularity on studying daily, and revising topics after class; (2) teacher factors such as teaching style, teacher’s guidance, and utilization of different learning instruments; (3) learning environment; and (4) other factors such as family issues and health issues [10].
Majority of the test items are categorized into acceptable difficulty level (64.00%) and high discrimination index (88.00%). In this study, we observed a wide distribution range in both difficulty index (5.26 to 75.66) and discrimination index (0.08 to 0.63). This condition may result from random guessing due to no penalties given in wrong answers. Same observations were also reported by Pande et al. [11] in the 240 physiology MCQ-SBA examination (difficulty index [P], 8.57 to 95.71; discrimination index [D], -0.54 to 0.8).
In conclusion, medical curriculum may have a significant impact on student’s retention capacity. In order to have better retention, we should consider the amount of time given to process a certain load of information. This study may be repeated for 1-year and 2-year post-course retention test with better preparation and higher reliability instrument. Further research using retention test in different courses may be conducted to analyze which curriculum that promotes students’ understanding of medical knowledge.

Acknowledgments

The authors would like to thank Yulia Nadar Indrasari, Endang Retnowati, and Yetti Hernaningsih from department of clinical pathology for their contribution in validating the MCQ-SBA items. The authors would like to thank Firas Farisi Alkaff from department of pharmacology who provided insight, expertise, and proofread the article which had greatly assisted this research.

Notes

Funding
No funding.
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
Conception and design of this research: JPS, FSIP, NMR; acquisition of data: JPS; analysis and interpretation of data: JPS, FSIP, NMR, Atika; drafting the article: JPS, FSIP; revising the article: JPS, FSIP, NMR, Atika; and final approval of the version to be published: JPS, FSIP, NMR, Atika.

Table 1.
Retention Test Score Distribution between Groups
Score range Group 1 Group 2
0–2 1 (0.72) 4 (2.82)
3–5 9 (6.52) 19 (13.38)
6–8 25 (18.12) 49 (34.51)
9–11 43 (31.16) 45 (31.69)
12–14 40 (28.99) 22 (15.49)
15–17 14 (10.14) 3 (2.11)
>17 6 (4.35) 0

Data are presented as number (%).

Table 2.
Correct Answer Distribution between Groups
Difficulty level Group 1 Group 2 p-value
Difficult 212 (21.95) 163 (16.40) 0.003
Acceptable 1,085 (49.14) 847 (37.28) 0.000
Easy 212 (76.81) 205 (72.18) 0.155

Data are presented as number (%).

References

1. Konsil Kedokteran Indonesia. Standar Pendidikan Profesi Dokter Indonesia. Jakarta, Indonesia: Konsil Kedokteran Indonesia; 2012.

2. UNESCO International Bureau of Education. Module 8: curriculum evaluation and student assessment. http://www.ibe.unesco.org/fileadmin/user_upload/COPs/Pages_documents/Resource_Packs/TTCD/sitemap/Module_8/Module_8.html. Published 2009. Accessed January 6. 2019.

3. Van Gessel E, Nendaz MR, Vermeulen B, Junod A, Vu NV. Development of clinical reasoning from the basic sciences to the clerkships: a longitudinal assessment of medical students’ needs and self-perception after a transitional learning unit. Med Educ 2003; 37(11):966-974.
crossref pmid
4. Kandel ER, Dudai Y, Mayford MR. The molecular and systems biology of memory. Cell 2014; 157(1):163-186.
crossref pmid
5. Ananthakrishnan N. The item analysis. In: Ananthakrishnan N, Sethuraman KR, Kumar S, eds. Medical Education Principles and Practice. Pondicherry, India: Alumni Association of National Teacher Training Centre; 2000. p. 131-173.

6. Singh T, Gupta P, Singh D. Principles of medical education. 4th ed. Gwalior, India: Jaypee Brothers Medical Pub; 2013.

7. El-Uri FI, Malas N. Analysis of use of a single best answer format in an undergraduate medical examination. Qatar Med J 2013; 2013(1):3-6.
crossref pmid pmc
8. Van Merriënboer JJ, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ 2010; 44(1):85-93.
crossref pmid
9. Abdulrahman KB, Mennin S, Harden R, Kennedy C. Routledge international handbook of medical education. New York, USA: Routledge; 2016.

10. Sharma P, Singh P, Kalhan S, Garg S. Analysis of factors affecting academic performance of MBBS students in pathology. Ann Int Med Dent Res 2017; 3(5):PT9-PT15.
crossref
11. Pande SS, Pande SR, Parate VR, Nikam AP, Agrekar SH. Correlation between difficulty and discrimination indices of MCQs in formative exam in physiology. South East Asian J Med Educ 2013; 7(1):45-50.
crossref
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-747-6206
E-mail : kjme_office@daum.net
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2019 by Korean Society of Medical Education. All rights reserved.                 powerd by m2community