Clinical reasoning is one of the most critical competencies a doctor should possess. It involves logical thinking processes leading to determining the diagnosis and management of the disease [1]. The ability is also considered very important and is associated with an essential skill in applying physician professionalism [2]. Educational methods such as problem-based learning (PBL) and team-based learning are recommended to develop the reasoning abilities of medical students [3]. Furthermore, insufficient attention to teaching clinical reasoning skills increases the incidence of misdiagnosis as a cause of medical errors [4]. Therefore, it is essential to educate medical students about this skill.

The development of students’ clinical reasoning abilities should be considered in teaching and assessment design in medical education [5,6]. Several innovations were developed to teach clinical reasoning. This teaching sometimes takes the form of a specific course [7], but this skill is usually integrated into the entire curriculum or clinical rotation [5]. The standard approach to clinical education applies to all stages of the process, including gathering information through history taking, physical and proposed laboratory examinations, identification of emerging patterns of signs and symptoms, determining the most likely diagnosis, and deciding on the appropriate treatment for a particular patient’s condition [8]. This process forms the basis for developing its assessment system. Furthermore, various methods were developed to test the component of the process. Some focus on specific sub-task in the clinical. However, other methods examine the entire process of clinical reasoning [9].

The introduction of coronavirus disease 2019 (COVID-19) has presented medical education with unprecedented challenges. Early clinical attachment delivery, which is designed to prepare undergraduate students for the clinical environment and foster communication, history-taking, and examination skills [10], has faced difficulties. Innovative approaches have been taken to solve this problem with the use of a virtual environment. After undergoing the medical curriculum during the COVID-19 pandemic for 2 years, it is necessary to conduct a study on how the clinical reasoning abilities of medical students develop, what are the opinions and experiences of students about learning clinical reasoning, as well as what factors support students to learn clinical reasoning. Therefore, this study aims to explore medical students’ perceptions and experiences with the clinical reasoning curriculum during the COVID-19 pandemic and determine their skills development.

This study was conducted in 2021 at an Indonesian Medical School. The school has accreditation level A, which is the highest level of accreditation for medical schools in Indonesia. The Indonesian Accreditation Body of Health Institutions in Indonesia was acknowledged by the World Federation of Medical Education, so the medical school meets good quality standards of medical education.

The curriculum was divided into three phases, namely year I of basic medical sciences (normal conditions), years II–III of clinical sciences based on the body system, and year IV of integrated clinical sciences. The theme of this stage underlies the learning activities of that year. PBL through block implementation and case-based clinical reasoning (CBCR) is taught during the process. Curriculum evaluation in 2019 mandated the need for learning modification to promote clinical reasoning skills. Consequently, a case-based clinical discussion was implemented in 2020 for the 2nd–4th-year students. This case-based activity was implemented in four online meetings of 100 minutes. Students are provided with information about the chief complaint, which was developed into a complete case scenario, starting with the data from the history, physical examination, proposal, and interpretation of supporting investigations to disease management. Each student developed a complete case scenario before the online class with instructor and uploaded the scenario in a Google Drive provided by the course coordinator. Students simulated an actual clinical encounter by pretending to be a doctor during the online class and were asked to seek information from friends that acted as patients. The data used in the simulation was a case scenario developed by friends acting as simulated patients. From the chief complaint, the information needed to develop a diagnosis and determine treatment was gradually explored (serial cue method) [11]. The steps of a medical investigation were simulated in stages, from history taking to requesting a diagnostic examination. Answers were provided in response to each question or suggestion requested. At each step, instructors should ask why students request specific information. They provided feedback on the questions and responses while collecting clinical information [11].

Students were tested with a structured oral examination (SOE) at the end of the semester. They completed an oral examination scenario consisting of several tasks: conduct history taking, propose a physical examination, interpret supporting examination results, determine a diagnosis, and arrange therapy. They performed the tasks on an online simulated patient. An analytic rubric was developed to assess the standardized oral tests according to tasks. The scenarios are selected based on the block and selected from the list of diseases in the Competency Standard of Indonesian Medical Doctors. Using online platforms such as Zoom; an examiner will test each student for 15 minutes using a blueprint consisting of three sets of SOE questions that have been randomly and systematically selected (Table 1).

This study used a mixed methods design, with a concurrent approach, which is intended to provide an overview of students’ clinical reasoning skills and the reason why learning supports the development of this ability during the COVID-19 pandemic. Informed consent was submitted before filling out the quantitative survey, and each participant was provided with information on the research data’s purpose, method, and confidentiality. Participants were also informed that the results of this study will not affect their academic scores.

A total of 213 out of 468 individual responses to DTI were selected by stratified random sampling. All DTI items were found to be valid with a Cronbach α score of 0.814. Table 2 shows the respondent data by gender and class. Furthermore, it indicates that more than 70% of the respondents are women.

Table 2 presents the oral examination scores for each class. The highest score is 100 for 3rd- and 4th-year students, while the lowest score is 31.0 for 2nd-year students. The lowest and highest mean scores are in the 2nd and 4th years, respectively.

Fig. 1 shows the diagnostic thinking domain scores for each batch. The red dotted line serves as a comparison [12]. Furthermore, the score on the line represents the diagnostic thinking domain value in each academic and professional phase of the DTI developer. It shows an increasing trend, although there is a slight decrease of the scores from the 2019 class (year 3) to the 2018 class (year 4).

Table 3 shows the results of the SOE score correlation test with all diagnostic thinking domains. SOE correlated significantly with the diagnostic thinking domains with moderate correlation strength.

The qualitative analysis of the FGD verbatim transcripts resulted in three themes: perceptions related to clinical reasoning, clinical reasoning activities, and learning component. Table 4 shows the results of the qualitative analysis.

This study provides additional evidence regarding the online CBCR method and SOE as part of the clinical reasoning curriculum during the COVID-19 pandemic. Furthermore, it describes the components that affect the learning of this skill.

Fig. 1 shows the development of medical students’ clinical reasoning abilities measured by DTI. There appears to be an increase in diagnostic thinking abilities from the class of 2020 to 2018. Despite the DTI scores of the 2018 class is slightly decreased compared to the 2019 class, the trend of increasing DTI is consistent with the score at levels equivalent to those of the study by Bordage et al. [12] when developing the questionnaire. The factor that caused the decline in the class of 2018 DTI score are that the classes of 2019 and 2020 received exposure to CBCR earlier than the class of 2020. The classes of 2019 and 2020 are introduced to clinical reasoning learning in the 2nd and 4th semesters, while the 2018 class are trained in CBCR in the 6th semester. Exposure to clinical reasoning learning through CBCR should be from the beginning along with the development of student knowledge (knowledge-oriented approach). Based on the study by Schmidt and Mamede [11], students of the 2018 class may be in stage 1 of clinical reasoning development, in that, when presented with clinical cases, students can only focus on isolated signs and symptoms, even though they are already in the integrated clinical medicine phase in their curriculum. Meanwhile, students of the 2019 class have entered the transition between stages 1 and 2 which is in stage 2 students’ knowledge structures alter as a result of the broad and repetitive application of the knowledge when exposed to patient cases. Their networks of in-depth, causal, pathophysiological understanding of the disease are condensed into diagnostic terms or high-level, simplified causal models that explain the sign and symptoms [11]. Hence, CBCR learning focuses on two concurrent strategies: (1) developing illness scripts early at the beginning of the curriculum, starting with simple cases and progressively developing more complex scripts to remember; and (2) assigning a systematic, analytical reasoning habit starting with patient problems vignettes and concluding with a decision about the diagnosis, the disease mechanism, and the patient management actions to be taken [13].

Table 3 shows that there is a relationship between SOE and DTI scores. The clinical reasoning capacity of medical students was assessed using a SOE and a DTI questionnaire. SOE can test clinical reasoning skills and knowledge in numerous categories that can be applied to different levels of medical students [14]. It is believed that DTI can evaluate flexibility in thinking and memory structure at various levels of clinical experience [12,15].

The qualitative analysis yielded three themes (Table 4): perceptions of the clinical reasoning curriculum, learning system, and learning components. Quotations were not compared between classes because the respondents’ perceptions regarding clinical reasoning were mostly similar and there were no differences in perceptions by grade. Due to the limited number of words, authors write the quotation that is closest to the related code or sub-theme.

First, in the students’ perceptions of clinical reasoning, students of 2020 class considered that clinical reasoning is an essential skill of a doctor. Meanwhile, the 2019 students believed that the ability is related to patient’s wellbeing. Furthermore, the 2019 student’s response to the teaching was excellent, which could be the basis of the curriculum’s success in improving clinical reasoning abilities. The 2018 and 2019 students also mentioned the need for clinical reasoning teaching since the beginning of their medical curriculum. It corresponds with the quantitative analysis result that conveys the declining score of the class 2018 (year 4). Amey et al. [16] state that clinical reasoning should be taught in the medical student as early as the first year of medical curriculum.

Second, students from all batches feel that the learning activities at the institution train them to master clinical reasoning skills, i.e., PBL, case-based clinical discussion, think-aloud and serial cue methods, and case-based simulation training. The chief complaint and task steps (serial cue-method) stimulated them to think about improving the skill by finding a diagnosis. In addition to the whole case method, the serial cue technique is one of the instructional designs for teaching clinical reasoning [17,18].

Students believe that it is simpler to think when they are asked to verbalize their thoughts while searching for information to finish the major complaint. This method is known as the think-aloud technique and some of the benefits are outlined by Bowen and Ilgen [19] and Pinnock et al. [20]. First, it explains and describes a complex and parallel information with clinical reasoning processes. Second, it causes students to observe how clinicians filter essential information as well as organize and construct relationships between these data. Finally, thinking aloud contains all the reasoning processes, prioritizing the process of determining the diagnosis. According to students, a learning activity that supports the development of clinical reasoning abilities is simulation-based learning. Hussein Mohamed and Giuliana [21] stated that simulation is exposure to controlled clinical situations that can provide students with opportunities to be more familiar with applying clinical knowledge. Furthermore, students can explore their decision-making abilities more autonomously, but facilitator support is still available. Simulation-based learning facilitates the seeking autonomy and further understanding of students’ cognitive functions, including critical thinking, clinical reasoning, and judgment [21].

Students explained that during medical education, the several assessment methods used to test clinical skills include multiple-choice questions, progress tests, objectivestructured clinical examination, and SOEs. Assessment of clinical reasoning ability should be conducted systematically throughout the medical education curriculum, in various settings, and using multiple methods [9,22]. Faculty can use a systematic assessment to capture an overview of clinical reasoning abilities, weaknesses, strengths, and developments [23].

Case type, teacher, prior knowledge, and peers were identified as components that assist clinical reasoning learning in the last theme. Ambiguous clinical cases are considered more exciting and stimulate clinical reasoning than straightforward ones. This is because these cases require students to think more critically. Students that presented ambiguous cases seemed slower and tried to remember more information than those that viewed clear circumstances. This shows that ambiguous clinical cases lead students to switch from automatic to reflective reasoning, indicated by the longer processing time and more literal propositions remembered [24,25]. Furthermore, students dominantly use the type 2 clinical reasoning process as novices or beginners [10]. Previous research has indicated that novices employ a pathophysiological analytic method for diagnosis more frequently than a pattern recognition technique, which develops after trainees establish substantial illness scripts through exposure or clinical experience [26].

Audétat et al. [27] stated that students encounter several potential difficulties in learning clinical reasoning, including problems in identifying important facts and digging up data to formulate hypotheses. The study also states that the instructor’s role is to detect and identify these difficulties during learning and provide specific feedback according to the identified problems. In addition to the instructor, peer role is cited as a factor that influences the acquisition of clinical reasoning. Yamauchi et al. [7] states that peer-to-peer learning is a feasible teaching format with high acceptance. Peers can assume the role of simulated patients, observing the practices and providing feedback.

Regarding prior knowledge, Beullens et al. [28] states that knowledge restructuring is initiated when students study extensively through reading papers and clinical case reports. In analytic-type clinical reasoning, the students diagnose by analyzing signs and symptoms data referring to biomedical knowledge [24]. This is gradually ‘encapsulated’ under the student’s previous clinical knowledge [24]. The representation of knowledge possessed directs students in gathering information and generating hypotheses [9].

To conclude, this study supports the evidence that clinical reasoning should be taught to medical students from the beginning of their education to develop the skill. This study also confirms that during the COVID-19 pandemic, the institution has implemented trustworthy educational strategies to support the development of students’ clinical reasoning abilities. The case-based activity, whether it is an online, or face-to-face method, strengthened by other techniques such as think-aloud, serial cue, and simulation-based education, can promote clinical reasoning skills. Furthermore, SOE is a clinical reasoning assessment method that is relatively easy to perform and can be combined with other assessment techniques. Students’ clinical reasoning abilities develop with the support of good perceptions of clinical reasoning, instructors, friends, case types, and prior knowledge. Therefore, understanding these factors helps the institution to create the curriculum and support system to support the development of students’ clinical reasoning abilities.