Promoting medical student’s clinical reasoning during COVID-19 pandemic

Article information

Korean J Med Educ. 2023;35(2):187-198
Publication date (electronic) : 2023 June 1
doi :
Faculty of Medicine, Sebelas Maret University, Surakarta, Indonesia
Corresponding Author: Bulan Kakanita Hermasari ( Sebelas Maret University, Ir. Sutami Road No. 36A Kentingan, Surakarta 57126, Indonesia Tel: +62.271.664178 Fax: +62.271.637400 email:
Received 2022 October 4; Revised 2023 January 13; Accepted 2023 March 17.



The development of students’ clinical reasoning skills should be a consideration in the design of instruction and evaluation in medical education. In response to the coronavirus disease 2019 (COVID-19) pandemic, several changes in the medical curriculum have been implemented in promoting clinical reasoning. This study aims to explore medical students’ perceptions and experiences with the clinical reasoning curriculum during the COVID-19 pandemic and determine their skills development.


The study used a mixed-method design with a concurrent approach. A cross-sectional study was conducted to compare and examine the relationship between the outcomes of the structured oral examination (SOE) and the Diagnostic Thinking Inventory (DTI). Then, the qualitative method was used. A focus group discussion using a semi-structured interview guide with open-ended questions was conducted, then the verbatim transcript was subjected to thematic analysis.


There is an increase in SOE and DTI scores between second-year to fourth-year students. The diagnostic thinking domains and SOE are significantly correlated (r=0.302, 0.313, and 0.241 with p<0.05). The three primary themes from the qualitative analysis are perceptions regarding clinical reasoning, clinical reasoning activities, and the learning component.


Even if students are still studying throughout the COVID-19 pandemic, their clinical reasoning skills can improve. The clinical reasoning and diagnostic thinking skills of medical students increase as the length of the school year increases. Online case-based learning and assessment support the development of clinical reasoning skills. The skills are supported in their development by positive attitudes toward faculty, peers, case type, and prior knowledge.


Clinical reasoning is one of the most critical competencies a doctor should possess. It involves logical thinking processes leading to determining the diagnosis and management of the disease [1]. The ability is also considered very important and is associated with an essential skill in applying physician professionalism [2]. Educational methods such as problem-based learning (PBL) and team-based learning are recommended to develop the reasoning abilities of medical students [3]. Furthermore, insufficient attention to teaching clinical reasoning skills increases the incidence of misdiagnosis as a cause of medical errors [4]. Therefore, it is essential to educate medical students about this skill.

The development of students’ clinical reasoning abilities should be considered in teaching and assessment design in medical education [5,6]. Several innovations were developed to teach clinical reasoning. This teaching sometimes takes the form of a specific course [7], but this skill is usually integrated into the entire curriculum or clinical rotation [5]. The standard approach to clinical education applies to all stages of the process, including gathering information through history taking, physical and proposed laboratory examinations, identification of emerging patterns of signs and symptoms, determining the most likely diagnosis, and deciding on the appropriate treatment for a particular patient’s condition [8]. This process forms the basis for developing its assessment system. Furthermore, various methods were developed to test the component of the process. Some focus on specific sub-task in the clinical. However, other methods examine the entire process of clinical reasoning [9].

The introduction of coronavirus disease 2019 (COVID-19) has presented medical education with unprecedented challenges. Early clinical attachment delivery, which is designed to prepare undergraduate students for the clinical environment and foster communication, history-taking, and examination skills [10], has faced difficulties. Innovative approaches have been taken to solve this problem with the use of a virtual environment. After undergoing the medical curriculum during the COVID-19 pandemic for 2 years, it is necessary to conduct a study on how the clinical reasoning abilities of medical students develop, what are the opinions and experiences of students about learning clinical reasoning, as well as what factors support students to learn clinical reasoning. Therefore, this study aims to explore medical students’ perceptions and experiences with the clinical reasoning curriculum during the COVID-19 pandemic and determine their skills development.

This study was conducted in 2021 at an Indonesian Medical School. The school has accreditation level A, which is the highest level of accreditation for medical schools in Indonesia. The Indonesian Accreditation Body of Health Institutions in Indonesia was acknowledged by the World Federation of Medical Education, so the medical school meets good quality standards of medical education.

The curriculum was divided into three phases, namely year I of basic medical sciences (normal conditions), years II–III of clinical sciences based on the body system, and year IV of integrated clinical sciences. The theme of this stage underlies the learning activities of that year. PBL through block implementation and case-based clinical reasoning (CBCR) is taught during the process. Curriculum evaluation in 2019 mandated the need for learning modification to promote clinical reasoning skills. Consequently, a case-based clinical discussion was implemented in 2020 for the 2nd–4th-year students. This case-based activity was implemented in four online meetings of 100 minutes. Students are provided with information about the chief complaint, which was developed into a complete case scenario, starting with the data from the history, physical examination, proposal, and interpretation of supporting investigations to disease management. Each student developed a complete case scenario before the online class with instructor and uploaded the scenario in a Google Drive provided by the course coordinator. Students simulated an actual clinical encounter by pretending to be a doctor during the online class and were asked to seek information from friends that acted as patients. The data used in the simulation was a case scenario developed by friends acting as simulated patients. From the chief complaint, the information needed to develop a diagnosis and determine treatment was gradually explored (serial cue method) [11]. The steps of a medical investigation were simulated in stages, from history taking to requesting a diagnostic examination. Answers were provided in response to each question or suggestion requested. At each step, instructors should ask why students request specific information. They provided feedback on the questions and responses while collecting clinical information [11].

Students were tested with a structured oral examination (SOE) at the end of the semester. They completed an oral examination scenario consisting of several tasks: conduct history taking, propose a physical examination, interpret supporting examination results, determine a diagnosis, and arrange therapy. They performed the tasks on an online simulated patient. An analytic rubric was developed to assess the standardized oral tests according to tasks. The scenarios are selected based on the block and selected from the list of diseases in the Competency Standard of Indonesian Medical Doctors. Using online platforms such as Zoom; an examiner will test each student for 15 minutes using a blueprint consisting of three sets of SOE questions that have been randomly and systematically selected (Table 1).

Mapping of Teaching, Learning, and Assessment Activities


This study used a mixed methods design, with a concurrent approach, which is intended to provide an overview of students’ clinical reasoning skills and the reason why learning supports the development of this ability during the COVID-19 pandemic. Informed consent was submitted before filling out the quantitative survey, and each participant was provided with information on the research data’s purpose, method, and confidentiality. Participants were also informed that the results of this study will not affect their academic scores.

1. Quantitative studies

A cross-sectional study was conducted to evaluate the development of clinical reasoning ability during the COVID-19 pandemic using SOE and test the correlation between the results of the SOE with the Diagnostic Thinking Inventory (DTI). The DTI questionnaire was shared with students via Google Forms immediately after completing the SOE. A total of 213 responses were selected by stratified random sampling from participants in the SOE and those that filled out the DTI survey. The data were analyzed descriptively to compare each batch. Furthermore, the author conducted a correlation test on the mean scores and deviation standards of SOE and DTI.

2. Qualitative studies

To explore more about student perception and experience regarding clinical reasoning curriculum during the COVID-19 pandemic and factors affecting the learning of clinical reasoning, a qualitative approach was conducted. The data were collected using focus group discussion (FGD). The respondents were interviewed with a semi-structured guide with open-ended questions. FGD questions include their understanding of clinical reasoning, the learning activities that shape their clinical reasoning development, how they learn before and during the case-based schedule, how they prepare for SOE, and the factors that support the development of their abilities. Twenty students from the list of DTI survey respondents volunteered to participate after being invited to join the FGD. Interviews were conducted on three groups of students according to their generation; each contained 6–8 respondents. The participant filled in written consent before the interview. The FGD was facilitated by 1st, 3rd, and 6th authors, while the recordings were then converted into verbatim transcripts. Afterward, a thematic analysis was performed by 1st, 2nd, 4th, and 5th authors on the verbatim transcript using a pre-determined coding scheme developed based on research questions. The coding scheme consists of three preliminary themes: student perception of their learning, learning and assessment activities that promote clinical reasoning skills, and factors/component affecting clinical reasoning development. The coders separately re-read all verbatim transcripts for data familiarization, then identified codes according to the preliminary themes. After that, the coders discussed grouping the codes into sub-themes. If there is a difference of opinion, then the sub-theme chosen is based on agreement by at least three coders. These steps were repeated until all the code has been grouped into subthemes.


A total of 213 out of 468 individual responses to DTI were selected by stratified random sampling. All DTI items were found to be valid with a Cronbach α score of 0.814. Table 2 shows the respondent data by gender and class. Furthermore, it indicates that more than 70% of the respondents are women.

Differences in DTI and SOE Mean Scores Based on Student Classes

Table 2 presents the oral examination scores for each class. The highest score is 100 for 3rd- and 4th-year students, while the lowest score is 31.0 for 2nd-year students. The lowest and highest mean scores are in the 2nd and 4th years, respectively.

Fig. 1 shows the diagnostic thinking domain scores for each batch. The red dotted line serves as a comparison [12]. Furthermore, the score on the line represents the diagnostic thinking domain value in each academic and professional phase of the DTI developer. It shows an increasing trend, although there is a slight decrease of the scores from the 2019 class (year 3) to the 2018 class (year 4).

Fig. 1.

Flexibility in Thinking and Structure of Memory Scores in All Year

(A) Diagnostic Thinking Inventory (DTI)-flexibility in thinking scores in all year. (B) DTI-structure of memory scores in all year. (C) Overall DTI scores in all year. SD: Standard deviation.

Table 3 shows the results of the SOE score correlation test with all diagnostic thinking domains. SOE correlated significantly with the diagnostic thinking domains with moderate correlation strength.

Correlation Test of the Mean Scores of Structured Oral Examination Scores and Diagnostic Thinking Ability

The qualitative analysis of the FGD verbatim transcripts resulted in three themes: perceptions related to clinical reasoning, clinical reasoning activities, and learning component. Table 4 shows the results of the qualitative analysis.

Qualitative Analysis Result


This study provides additional evidence regarding the online CBCR method and SOE as part of the clinical reasoning curriculum during the COVID-19 pandemic. Furthermore, it describes the components that affect the learning of this skill.

Fig. 1 shows the development of medical students’ clinical reasoning abilities measured by DTI. There appears to be an increase in diagnostic thinking abilities from the class of 2020 to 2018. Despite the DTI scores of the 2018 class is slightly decreased compared to the 2019 class, the trend of increasing DTI is consistent with the score at levels equivalent to those of the study by Bordage et al. [12] when developing the questionnaire. The factor that caused the decline in the class of 2018 DTI score are that the classes of 2019 and 2020 received exposure to CBCR earlier than the class of 2020. The classes of 2019 and 2020 are introduced to clinical reasoning learning in the 2nd and 4th semesters, while the 2018 class are trained in CBCR in the 6th semester. Exposure to clinical reasoning learning through CBCR should be from the beginning along with the development of student knowledge (knowledge-oriented approach). Based on the study by Schmidt and Mamede [11], students of the 2018 class may be in stage 1 of clinical reasoning development, in that, when presented with clinical cases, students can only focus on isolated signs and symptoms, even though they are already in the integrated clinical medicine phase in their curriculum. Meanwhile, students of the 2019 class have entered the transition between stages 1 and 2 which is in stage 2 students’ knowledge structures alter as a result of the broad and repetitive application of the knowledge when exposed to patient cases. Their networks of in-depth, causal, pathophysiological understanding of the disease are condensed into diagnostic terms or high-level, simplified causal models that explain the sign and symptoms [11]. Hence, CBCR learning focuses on two concurrent strategies: (1) developing illness scripts early at the beginning of the curriculum, starting with simple cases and progressively developing more complex scripts to remember; and (2) assigning a systematic, analytical reasoning habit starting with patient problems vignettes and concluding with a decision about the diagnosis, the disease mechanism, and the patient management actions to be taken [13].

Table 3 shows that there is a relationship between SOE and DTI scores. The clinical reasoning capacity of medical students was assessed using a SOE and a DTI questionnaire. SOE can test clinical reasoning skills and knowledge in numerous categories that can be applied to different levels of medical students [14]. It is believed that DTI can evaluate flexibility in thinking and memory structure at various levels of clinical experience [12,15].

The qualitative analysis yielded three themes (Table 4): perceptions of the clinical reasoning curriculum, learning system, and learning components. Quotations were not compared between classes because the respondents’ perceptions regarding clinical reasoning were mostly similar and there were no differences in perceptions by grade. Due to the limited number of words, authors write the quotation that is closest to the related code or sub-theme.

First, in the students’ perceptions of clinical reasoning, students of 2020 class considered that clinical reasoning is an essential skill of a doctor. Meanwhile, the 2019 students believed that the ability is related to patient’s wellbeing. Furthermore, the 2019 student’s response to the teaching was excellent, which could be the basis of the curriculum’s success in improving clinical reasoning abilities. The 2018 and 2019 students also mentioned the need for clinical reasoning teaching since the beginning of their medical curriculum. It corresponds with the quantitative analysis result that conveys the declining score of the class 2018 (year 4). Amey et al. [16] state that clinical reasoning should be taught in the medical student as early as the first year of medical curriculum.

Second, students from all batches feel that the learning activities at the institution train them to master clinical reasoning skills, i.e., PBL, case-based clinical discussion, think-aloud and serial cue methods, and case-based simulation training. The chief complaint and task steps (serial cue-method) stimulated them to think about improving the skill by finding a diagnosis. In addition to the whole case method, the serial cue technique is one of the instructional designs for teaching clinical reasoning [17,18].

Students believe that it is simpler to think when they are asked to verbalize their thoughts while searching for information to finish the major complaint. This method is known as the think-aloud technique and some of the benefits are outlined by Bowen and Ilgen [19] and Pinnock et al. [20]. First, it explains and describes a complex and parallel information with clinical reasoning processes. Second, it causes students to observe how clinicians filter essential information as well as organize and construct relationships between these data. Finally, thinking aloud contains all the reasoning processes, prioritizing the process of determining the diagnosis. According to students, a learning activity that supports the development of clinical reasoning abilities is simulation-based learning. Hussein Mohamed and Giuliana [21] stated that simulation is exposure to controlled clinical situations that can provide students with opportunities to be more familiar with applying clinical knowledge. Furthermore, students can explore their decision-making abilities more autonomously, but facilitator support is still available. Simulation-based learning facilitates the seeking autonomy and further understanding of students’ cognitive functions, including critical thinking, clinical reasoning, and judgment [21].

Students explained that during medical education, the several assessment methods used to test clinical skills include multiple-choice questions, progress tests, objectivestructured clinical examination, and SOEs. Assessment of clinical reasoning ability should be conducted systematically throughout the medical education curriculum, in various settings, and using multiple methods [9,22]. Faculty can use a systematic assessment to capture an overview of clinical reasoning abilities, weaknesses, strengths, and developments [23].

Case type, teacher, prior knowledge, and peers were identified as components that assist clinical reasoning learning in the last theme. Ambiguous clinical cases are considered more exciting and stimulate clinical reasoning than straightforward ones. This is because these cases require students to think more critically. Students that presented ambiguous cases seemed slower and tried to remember more information than those that viewed clear circumstances. This shows that ambiguous clinical cases lead students to switch from automatic to reflective reasoning, indicated by the longer processing time and more literal propositions remembered [24,25]. Furthermore, students dominantly use the type 2 clinical reasoning process as novices or beginners [10]. Previous research has indicated that novices employ a pathophysiological analytic method for diagnosis more frequently than a pattern recognition technique, which develops after trainees establish substantial illness scripts through exposure or clinical experience [26].

Audétat et al. [27] stated that students encounter several potential difficulties in learning clinical reasoning, including problems in identifying important facts and digging up data to formulate hypotheses. The study also states that the instructor’s role is to detect and identify these difficulties during learning and provide specific feedback according to the identified problems. In addition to the instructor, peer role is cited as a factor that influences the acquisition of clinical reasoning. Yamauchi et al. [7] states that peer-to-peer learning is a feasible teaching format with high acceptance. Peers can assume the role of simulated patients, observing the practices and providing feedback.

Regarding prior knowledge, Beullens et al. [28] states that knowledge restructuring is initiated when students study extensively through reading papers and clinical case reports. In analytic-type clinical reasoning, the students diagnose by analyzing signs and symptoms data referring to biomedical knowledge [24]. This is gradually ‘encapsulated’ under the student’s previous clinical knowledge [24]. The representation of knowledge possessed directs students in gathering information and generating hypotheses [9].

To conclude, this study supports the evidence that clinical reasoning should be taught to medical students from the beginning of their education to develop the skill. This study also confirms that during the COVID-19 pandemic, the institution has implemented trustworthy educational strategies to support the development of students’ clinical reasoning abilities. The case-based activity, whether it is an online, or face-to-face method, strengthened by other techniques such as think-aloud, serial cue, and simulation-based education, can promote clinical reasoning skills. Furthermore, SOE is a clinical reasoning assessment method that is relatively easy to perform and can be combined with other assessment techniques. Students’ clinical reasoning abilities develop with the support of good perceptions of clinical reasoning, instructors, friends, case types, and prior knowledge. Therefore, understanding these factors helps the institution to create the curriculum and support system to support the development of students’ clinical reasoning abilities.


The authors acknowledge all students that participated in this study.



This study was supported by Sebelas Maret University.

Conflicts of interest

No potential conflict of interest relevant to this article was reported.

Author contributions

Conceptualization: BK, AM, DN, EPP; methodology: BK, AM, DN, VIB; formal analysis: BK, DN, AM, AAL; validation: EPP, VIB; writing–original draft preparation: BK, AM, DN; writing–review and editing: BK, DN, AAL; and approval of final manuscript: all authors.


1. Gruppen LD. Clinical reasoning: defining it, teaching it, assessing it, studying it. West J Emerg Med 2017;18(1):4–7.
2. Young M, Thomas A, Lubarsky S, et al. Drawing boundaries: the difficulty in defining clinical reasoning. Acad Med 2018;93(7):990–995.
3. Kassirer JP. Teaching clinical reasoning: case-based and coached. Acad Med 2010;85(7):1118–1124.
4. Patel VL, Arocha JF, Zhang J. Medical reasoning and thinking. In : Holyoak KJ, Morrison RG, eds. The Oxford Handbook of Thinking and Reasoning Oxford, UK: Oxford University Press; 2012. 736–754.
5. Vidyarthi AR, Kamei R, Chan K, Goh SH, Lek N. Factors associated with medical student clinical reasoning and evidence based medicine practice. Int J Med Educ 2015;6:142–148.
6. Levin M, Cennimo D, Chen S, Lamba S. Teaching clinical reasoning to medical students: a case-based illness script worksheet approach. MedEdPORTAL 2016;12:10445.
7. Yamauchi K, Hagiwara Y, Iwakura N, et al. Using peer role-playing to improve students’ clinical skills for musculoskeletal physical examinations. BMC Med Educ 2021;21(1):322.
8. Audétat MC, Laurin S, Dory V, Charlin B, Nendaz MR. Diagnosis and management of clinical reasoning difficulties: Part I. Clinical reasoning supervision and educational diagnosis. Med Teach 2017;39(8):792–796.
9. Daniel M, Rencic J, Durning SJ, et al. Clinical reasoning assessment methods: a scoping review and practical guidance. Acad Med 2019;94(6):902–912.
10. Gay S, Bartlett M, McKinley R. Teaching clinical reasoning to medical students. Clin Teach 2013;10(5):308–312.
11. Schmidt HG, Mamede S. How to improve the teaching of clinical reasoning: a narrative review and a proposal. Med Educ 2015;49(10):961–973.
12. Bordage G, Grant J, Marsden P. Quantitative assessment of diagnostic ability. Med Educ 1990;24(5):413–425.
13. ten Cate O, Custers EJ, Durning SJ. Principles and practice of case-based clinical reasoning education: a method for preclinical students Cham, Switzerland: Springer Nature; 2018.
14. Caldwell KE, Zarate Rodriguez JG, Hess A, Han BJ, Awad MM, Sacks BC. Standardized oral examinations allow for assessment of medical student clinical knowledge and decrease racial grading differences in a surgery clerkship. Surgery 2022;171(3):590–597.
15. Keshmiri F, Owlia F, Kazemipoor M, Meybodi FR. Assessment of clinical reasoning and diagnostic thinking among dental students. Res Sq [Preprint] 2021;Aug. 20.
16. Amey L, Donald KJ, Teodorczuk A. Teaching clinical reasoning to medical students. Br J Hosp Med (Lond) 2017;78(7):399–401.
17. Pinnock R, Anakin M, Lawrence J, Chignell H, Wilkinson T. Identifying developmental features in students’ clinical reasoning to inform teaching. Med Teach 2019;41(3):297–302.
18. Kiesewetter J, Sailer M, Jung VM, et al. Learning clinical reasoning: how virtual patient case format and prior knowledge interact. BMC Med Educ 2020;20(1):73.
19. Bowen JL, Ilgen JS. Now you see it, now you don’t: what thinking aloud tells us about clinical reasoning. J Grad Med Educ 2014;6(4):783–785.
20. Pinnock R, Young L, Spence F, Henning M, Hazell W. Can think aloud be used to teach and assess clinical reasoning in graduate medical education? J Grad Med Educ 2015;7(3):334–337.
21. Hussein Mohamed E, Giuliana H. Seeking autonomy: grounded theory of clinical reasoning processes during simulation based experiences. Nurse Educ Pract 2022;63:103408.
22. Sutherland RM, Reid KJ, Chiavaroli NG, Smallwood D, McColl GJ. Assessing diagnostic reasoning using a standardized case-based discussion. J Med Educ Curric Dev 2019;6:2382120519849411.
23. Schuwirth LW, van der Vleuten CP. How ‘testing’ has become ‘programmatic assessment for learning’. Health Prof Educ 2019;5(3):177–184.
24. Mamede S, Schmidt HG, Rikers RM, Penaforte JC, Coelho-Filho JM. Breaking down automaticity: case ambiguity and the shift to reflective approaches in clinical reasoning. Med Educ 2007;41(12):1185–1192.
25. Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S. The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking. Acad Med 2017;92(1):23–30.
26. Jackson JM, Skelton JA, Peters TR. Medical students’ clinical reasoning during a simulated viral pandemic: evidence of cognitive integration and insights on novices’ approach to diagnostic reasoning. Med Sci Educ 2020;30(2):767–774.
27. Audétat MC, Laurin S, Sanche G, et al. Clinical reasoning difficulties: a taxonomy for clinical teachers. Med Teach 2013;35(3):e984–e989.
28. Beullens J, Struyf E, Van Damme B. Diagnostic ability in relation to clinical seminars and extended-matching questions examinations. Med Educ 2006;40(12):1173–1179.

Article information Continued

Fig. 1.

Flexibility in Thinking and Structure of Memory Scores in All Year

(A) Diagnostic Thinking Inventory (DTI)-flexibility in thinking scores in all year. (B) DTI-structure of memory scores in all year. (C) Overall DTI scores in all year. SD: Standard deviation.

Table 1.

Mapping of Teaching, Learning, and Assessment Activities

Year Phase Block system Teaching learning activities Assessment activities Modification during COVID-19 pandemic (2020) Modification during COVID-19 pandemic (2021)
1 Basic medical sciences - Biocellular and hematology - Interactive lectures - Case-based MCQs All activities were implemented full online All activities were implemented full online
- Integument and musculoskeletal system - The 7-steps of PBL tutorial discussion - OSPE
- Digestive and metabolism system - Laboratory-based practical activities - Clinical skills examination Clinical skills examination → video of student performing clinical skills Clinical skills examination → video of student performing clinical skills
- Cardiorespiration system
- Neuroendocrine and special senses system - Clinical skill trainings
2 Body system-based clinical management - Basic of immunology and tropical disease - Interactive lectures - Case-based MCQs All activities were implemented full online All activities were implemented full online
- Integument and musculoskeletal system - The 7-steps of PBL tutorial discussion - OSPE
- Gastrohepatointestinal disease - Laboratory-based practical activities - OSCE OSCE → video of student performing clinical skills OSCE → video of student performing clinical skills and SOE
- Immunohematology disease - Clinical skill trainings - Progress test
- Metabolism and endocrine disease - Case-based clinical discussion (since 2020)
- Cardiovascular disease
3 - Respiration diseases - Interactive lectures - Case-based MCQs All activities were implemented full online All activities were implemented full online
- Neurology disease - The 7-steps of PBL tutorial discussion - OSPE
- Mental disease - Laboratory-based practical activities - OSCE OSCE → video of student performing clinical skills OSCE → video of student performing clinical skills and SOE
- Reproductive and urogenital disease - Clinical skill trainings
- Human life cycle - Case-based clinical discussion (since 2020)
4 Integrated disease management - Emergency and traumatology - Interactive lectures - Case-based MCQs All activities were implemented full online All activities were implemented full online
- Community medicine and national health system - The 7-steps of PBL tutorial discussion - OSPE
- Acupuncture - Laboratory-based practical activities - OSCE OSCE → video of student performing clinical skills OSCE → video of student performing clinical skills and SOE
- Herbal medicine - Clinical skill trainings - Progress test
- Elective blocks: clinical nutrition; sport medicine; occupational health; biotechnology - Case-based clinical discussion (since 2020)

COVID-19: Coronavirus disease 2019, PBL: Problem-based learning, MCQs: Multiple-choice questions, OSPE: Objective structured practical examination, OSCE: Objective-structured clinical examination, SOE: Structured oral examination.

Table 2.

Differences in DTI and SOE Mean Scores Based on Student Classes

Student class No. (%) Mean scores
Structure of memory Flexibility in thinking Overall DTI SOE
2020 77 (36.2) 74.48±11.75 72.13±8.83 146.61±17.82 72.87±13.93
2019 68 (31.9) 80.41±10.17 76.84±8.25 157.25±16.48 85.46±9.39
2018 68 (31.9) 78.84±9.34 76.53±9.58 155.37±17.51 80.66±12.96
Total 213 (100.0)
p-value 0.002* 0.002* <0.001* <0.001*

Data are presented as number (%) or mean±standard deviation, unless otherwise stated.

DTI: Diagnostic Thinking Inventory, SOE: Structured oral examination.


Statistically significant.

Table 3.

Correlation Test of the Mean Scores of Structured Oral Examination Scores and Diagnostic Thinking Ability

Structured oral examination
Diagnostic thinking Flexibility in thinking Structure of memory
r 0.302 0.313 0.241
p-value <0.001 <0.001 <0.001

Table 4.

Qualitative Analysis Result

Themes Sub-themes Quotations
Perception of clinical reasoning
Understanding of clinical reasoning Essential competency for physician “Clinical reasoning is something we must have as medical students because clinical reasoning will be very useful later in determining the patient’s diagnosis. Not only the diagnosis but also further treatment or therapy is given to the patient.” (FG-2018)
Related to patient safety “So, clinical reasoning is highly beneficial for the safety of the patient because it enables, we, doctors, to have a basis or guide for examining all problems and determining the management or solution, thereby avoiding being careless while diagnosing or providing treatment.” (FG-2019)
Responses to clinical reasoning learning Improvement of clinical reasoning ability “I feel that my clinical reasoning ability started to improve from the 3rd semester when I was initially taught. This is because I was still confused about the case. But now, in the third year, my skills have improved.” (FG-2019)
Can be taught early in the medical curriculum “It is very important to be taught from the beginning to medical students. Because in my opinion, reasoning, a way of thinking, cannot be instilled in only one night, it has to be instilled, it has to be nurtured from the start. Therefore, from the preclinical stage, medical students must form a mindset regarding the correct clinical reasoning, so that they can get the right diagnosis from their patients.” (FG-2020)
“At this preclinic stage, we learn about a lot of diseases, and there are lots of diseases that have the same clinical manifestations in the course of their disease. So if, for example, from the start, we don’t cultivate clinical reasoning, we can’t relate one symptom to another, it will definitely be difficult to diagnose in the future.” (FG-2018)
Learning method Case-based “From personal experience, I learned about clinical reasoning from PBL tutorials and the scenario of clinical skills training. These teaching methods can educate us how to conduct clinical reasoning from a case, despite the cases in the scenario being specific to a disease.” (FG-2018)
Think aloud “During each lesson, the instructor always prompted me to give the reasons. For instance, explain why you asked that question when taking a history. Then, during a physical examination, I was questioned about its purpose until the lesson was concluded. It (mentioning the cause) makes it easier to evaluate the situation critically.” (FG-2020)
Serial cue method “Initially, we were instructed to complete a case scenario-based history, physical examination, and supporting investigations. Then, they instructed us to provide one primary diagnosis and two differential diagnoses, and we were introduced to the patient’s pharmacological and non-pharmacological therapy options.” (FG-2019)
Simulation “Because, in my perspective, the simulation (in the clinical skills training) is more like a description of the genuine circumstance, so it is similar to a real exam or even a situation that may be encountered when we become doctors.” (FG-2018)
Assessment of clinical reasoning ability CBT MCQs “The MCQ CBT exam presented numerous case scenarios as multiple-choice questions to assess my clinical reasoning.” (FG-2018)
Progress test “From the progress test, we can learn more from our experiences in clinical skills training and PBL tutorials for clinical reasoning.” (FG-2019)
OSCE “Before we OSCE, we must prepare and read clinical cases one by one.” (FG-2020)
“Almost similar to SOE questions and assignments, but when our OSCE are asked to perform it actively, not just orally.” (FG-2019)
Structured oral examination “Case scenarios and duties such as obtaining anamnesis, suggesting a physical examination, proposing supporting examinations and interpreting them, determining and differentiating diagnoses, writing prescriptions, and delivering teaching are included in the SOE. This is comparable to the everyday instruction in the integration course.” (FG-2018)
Component of clinical reasoning learning
Facilitator Stimulus question “However, after working with multiple tutors, we discovered that the instructor who asked the most questions first was more effective…… the instructor would point out the error, similar to the doctor who said, why did you do that? If so, you can determine what it is, so we can evaluate where we went wrong.” (FG-2019)
Feedback “The facilitator will indicate if there is a problem, such as if the scenario is missing key details or if the history taking is not sufficiently in-depth for the examination.” (FG-2020)
“…. if we were incorrect, the instructor would point out the error, why did you do that? If so, you can determine what it is, so we can evaluate where we went wrong. Only after the tutor explains what is correct should clinical reasoning be in form.” (FG-2018)
Observed “The instructors asked all of their students to try the simulation one-on-one, and the instructors responded one by one; therefore, this could provide the best clinical reasoning education.” (FG-2019)
Case type Ambiguous clinical case “In the block exam, it is still determined per body part/system, so we can still make educated guesses like ‘this disease is just around the nervous system or the skin.’ But I believe we can be more elaborating to think and analytic, if we have to investigate from the symptom (in the exam item), what condition it will lead to, as in the progress test.” (FG-2018)
Peer Feedback “Students assist one another in their various cases within our tutorial group. In addition to developing scenarios, we are responsible for creating videos. Because each student's case is unique, the chat group conversation is also highly helpful; each participant answers the instructor’s questions with insightful comments.” (FG-2020)
Discussion partner “The scenarios in the guide were discussed, including diseases with the above characteristics. Consequently, we question each other, ‘basically what, how come?’ So, we’ll choose something similar when we are done with that kind of discussion, which is more of a casual conversation.” (FG-2018)
Basic knowledge Biomedical sciences “So, all the lessons at the Faculty of Medicine include clinical reasoning, even from the labs. I think that is also included because all of this starts from the physiological, the anatomical. After all, if we don’t understand the basics, we also can’t understand how to diagnose. The good one.” (FG-2018)
Accumulation of knowledge “Theories (from the block lectures) serve as the basis for clinical reasoning, which will be used in future practice. Then from that theory, a mind map or critical reasoning is formed to determine disease diagnoses.” (FG-2018)

PBL: Problem-based learning, MCQs: Multiple-choice questions, CBT: Computer-based test, OSCE: Objective-structured clinical examination, SOE: Structured oral examination.