Educational implications of assessing learning outcomes with multiple choice questions and short essay questions

Article information

Korean J Med Educ. 2023;35(3):285-290
Publication date (electronic) : 2023 August 31
doi :
Department of Medical Education, University of Ulsan College of Medicine, Seoul, Korea
Corresponding Author: Su Jin Chae ( Department of Medical Education, University of Ulsan College of Medicine, 88 Olympic-ro 43-gil, Songpa-gu, Seoul 05505, Korea Tel: +82.2.3010.4242 Fax: +82.2.3010.4242 email:
Received 2023 June 20; Revised 2023 July 9; Accepted 2023 July 28.



This study investigates the characteristics of different item types to assess learning outcomes and explore the educational implications that can be obtained from the results of learning outcome assessments.


Forty-five second-year premedical students participated in this study. Multiple choice question (MCQ) and short essay question (SEQ) scores and pass rates for 10 learning outcomes were analyzed. Descriptive statistics and correlation analysis were used to analyze the data.


The correlation analysis indicated that there was a significant correlation between SEQs and pass rate but there was no significant correlation between MCQs and pass rate. Some students with identical scores on the MCQs had different scores on the SEQs or on the learning outcomes.


This study showed that students’ achievement of learning outcomes can be assessed using various types of questions in outcome-based education.


Traditionally, student assessments in medical schools have primarily relied on written exams, which focus on knowledge-based learning objectives that professors deem important [1]. These exams are mainly used as summative assessments to determine student progress and graduation rather than to provide feedback on the current state of their learning [2]. Thus, as an appropriate assessment method for this purpose, medical schools have been using multiple choice questions (MCQs) [3], which have become a powerful assessment tool because they ensure assessment reliability and fairness [4,5]. However, as MCQs are an assessment method that emphasizes recall and comprehension, essay-type questions were introduced as an alternative assessment approach for assessing higher-order cognitive skills [6,7].

In outcome-based education, which is a major issue in contemporary medical education, assessment can provide information about a student’s current performance level and feedback on students’ weaknesses [8]. However, few studies on outcome-based education have evaluated whether students achieved the intended learning outcomes and whether the assessment results were used to provide feedback and learning guidance.

The University of Ulsan College of Medicine implemented an outcome-based assessment for its new curriculum introduced in the 2022 academic year. The system’s primary objective is to evaluate whether students have successfully achieved each course’s desired learning outcomes and determine their pass or fail status accordingly. To be recognized as having completed the course, all learning outcomes must be successfully achieved, and those who have not must repeat the course. Consequently, there was a need for a comprehensive assessment of each course’s set learning outcomes, and the university opted to use both MCQs and short essay questions (SEQs) for assessment.

This study aims to explore the educational significance found in the assessment system of University of Ulsan College of Medicine, which implements an outcome-based assessment, and to discuss the education implications obtained from the results of the assessments using various question types.


1. Participants and educational conditions

Forty-five participants in their second-year premedical program at University of Ulsan College of Medicine who had enrolled in the course “Cells and Metabolism” volunteered for this study. University of Ulsan College of Medicine implemented a pass/fail outcome-based assessment system for the second-year premedical program in 2022. “Cells and Metabolism” is a course that integrates several basic medical fields, such as biochemistry and genetics, and includes 10 learning outcomes. The main contents of these outcomes are related to cell structure, cell function, gene expression, genome, properties and metabolism of carbohydrates, breakdown and synthesis of monosaccharides, electron transport system, properties and metabolism of lipids, metabolism of amino acids and nucleic acids, and integrative human metabolism. To assess the achievement of each learning outcome, there were 6–18 MCQs and 1 SEQ per outcome. The assessment was conducted using a computer-based test consisting of 107 questions, with 97 MCQs and 10 SEQs (Table 1). Based on the assessment results, students’ achievement of the 10 learning outcomes was graded as either pass or fail, following predefined criteria.

Content and Assessment Results for Each Learning Outcome

2. Data collection and analysis

The scores of the MCQs and SEQs were converted to t-scores and standardized to a mean of 50 and standard deviation of 10. We also calculated the achievement rate of each student’s learning outcome, which is the percentage of the 10 predetermined learning outcomes passed by the students. Descriptive statistics were used to obtain basic information on academic achievement, and correlation analyses were conducted to explore the relationship between the assessment methods. The data were analyzed using IBM SPSS Statistics ver. 27.0 for Windows (IBM Corp., Armonk, USA).

3. Ethics statement

This study was exempted by the Institutional Review Board at Asan Medical Center (exemption no., 2022-1527).


1. Achievement results by learning outcomes

The results of the number of questions, MCQ scores, SEQ scores, and pass rates for the 10 learning outcomes are shown in Table 1. The MCQ scores ranged from 3.56 to 12.18, while the SEQ scores ranged from 1.87 to 4.11. On average, the pass rate for the learning outcomes was 0.74, indicating that students passed approximately seven out of 10 learning outcomes. The learning outcomes for “genome” (pass rate=0.87) and “properties and metabolism of carbohydrates” (pass rate=0.84) were achieved by a majority of students, but “electron transport system” (pass rate=0.47) and “cell function” (pass rate=0.64) were less successful than the other learning outcomes.

Table 2 presents the correlation analysis results of MCQ scores, SEQ scores, and pass rate. Correlation analysis revealed a slightly stronger correlation between MCQs and SEQs (r=0.631, p<0.01). Further, SEQs and the pass rate had a positive correlation (r=0.474, p<0.01), while the correlation between MCQs and the pass rate was not significant [9].

Correlations for MCQ Scores, SEQ Scores, and Pass Rate

2. Comparison of pass rate and SEQ scores based on MCQ scores

The top and bottom 25% (11 students each), selected based on the MCQ score quartiles, were analyzed for their MCQ and SEQ scores and pass rates; the results are presented in Table 3. The top and bottom 25% groups had average pass rates of 86% and 50%, respectively. Among students with low MCQ scores, both L3 and L4 had a score of 34.7 points. However, the content and number of the learning outcomes they passed and their SEQ scores varied. It is noteworthy that L10 had lower SEQ scores compared to other similarly situated students.

Learning Outcomes of Top and Bottom 25% of the Group Based on MCQ Scores

The top 25% of the group included students with identical MCQ scores but different SEQ scores. For example, H1 and H2 obtained identical MCQ scores, while their SEQ scores differed by more than seven points. Further, the content and number of the learning outcomes they passed varied. While H7 had high MCQ scores, their SEQ scores were extremely low, resulting in a low pass rate of 10%.


This study sought the educational implications of using MCQs and SEQs to assess learning outcomes in outcome-based education. The findings and their implications are discussed below.

First, we found that SEQs had a significant correlation with MCQs and pass rates in learning outcomes. These results are consistent with research indicating that SEQs assess higher-order cognitive domains compared with MCQs [6,7,10]. Thus, SEQs should be leveraged more extensively in assessing student achievement of learning outcomes. While MCQs have been commonly used for their objectivity, additional methods that can accurately assess student learning should be explored. Moreover, continuous training and evaluation of SEQ raters should be implemented to ensure assessment reliability and objectivity [11].

Second, certain students (e.g., L10, H7) tended to score lower on the SEQs compared to others. SEQs assess higher-order cognitive skills, such as problem-solving and abstract thinking [6,10]. Therefore, if two students have the same score on the MCQs, the student with the lower score on the SEQs may not have a lower level of higher-order cognitive skills compared with their counterparts. To address this, these students should be trained on how to approach SEQs or provided with feedback to support the development of their higher-order cognitive skills.

Third, the content and number of learning outcomes passed by students varied, even when they obtained the same SEQ scores (e.g., L3 and L4, and H1 and H2). Learning outcomes for a course represent the essential elements that students must fulfill in their learning process. By assessing which learning outcomes have been achieved and which have not, you can provide appropriate feedback on students’ learning progress. And we enable targeted interventions to improve their learning and ensure full achievement of the course learning outcomes [1,12]. Furthermore, it is crucial to acknowledge that students achieving the same score may exhibit disparate learning statuses. Therefore, it is imperative to employ assessment methods that comprehensively evaluate students across multiple dimensions.

This study’s limitation is that it focused only on a single course at University of Ulsan College of Medicine. However, the findings are valuable for demonstrating the feasibility of evaluating students’ achievement of learning outcomes using MCQs and SEQs within the context of outcome-based education. Further, the specific information about student achievement from these assessment methods can be useful for remediation. Future research should explore and develop diverse assessment methods capable of evaluating learning outcomes and positively impacting educational activities.





This study was supported by the University of Ulsan (no., 2022-0696).

Conflicts of interest

No potential conflict of interest relevant to this article was reported.

Author contributions

Conception or design of the work: SJC, HJK, JHP; data collection, data analysis: HJK; Interpretation, drafting the article: HJK, SJC; critical revision of the article: SJC, JHP; and final approval of the version to be published: SJC.


1. Im SJ. Assessment in outcome-based education. Korean Med Educ Rev 2013;15(1):25–30.
2. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med 2002;77(5):361–367.
3. Epstein RM. Assessment in medical education. N Engl J Med 2007;356(4):387–396.
4. Hift RJ. Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ 2014;14:249.
5. Herrero JI, Lucena F, Quiroga J. Randomized study showing the benefit of medical study writing multiple choice questions on their learning. BMC Med Educ 2019;19(1):42.
6. Palmer EJ, Devitt PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions?: research paper. BMC Med Educ 2007;7:49.
7. Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ 2004;38(9):974–979.
8. Yu T. Analysis of the US competency-based education policies and implications for competency-based education in Korea. J Educ Technol 2019;35(2):289–311.
9. Gogtay NJ, Thatte UM. Principles of correlation analysis. J Assoc Physicians India 2017;65(3):78–81.
10. Khan MU, Aljarallah BM. Evaluation of Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) as a tool for assessing the cognitive skills of undergraduate medical students. Int J Health Sci (Qassim) 2011;5(1):39–43.
11. Shumway JM, Harden RM; Association for Medical Education in Europe. AMEE guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach 2003;25(6):569–584.
12. Seong TJ, Lee B, Ahn S, Gwak YL. Effects of computerized formative assessment with the cognitive diagnosis model on achievement and affective attitude for math. J Educ Eval 2019;32(1):1–25.

Article information Continued

Table 1.

Content and Assessment Results for Each Learning Outcome

Content of learning outcomes No. of MCQs No. of SEQs MCQ scores mean±SD SEQ scores mean±SD Pass rate
1. Cell structure 8 1 6.29±1.24 3.47±1.34 0.76
2. Cell function 12 1 6.09±2.67 2.09±1.64 0.64
3 Gene expression 18 1 12.18±2.47 3.04±1.55 0.78
4. Genome 6 1 3.56±1.54 4.11±1.16 0.87
5. Properties and metabolism of carbohydrates 6 1 3.84±1.08 4.02±1.47 0.84
6. Breakdown and synthesis of monosaccharides 12 1 7.92±2.02 2.27±1.51 0.71
7. Electron transport system 6 1 3.96±1.48 2.27±2.35 0.47
8. Properties and metabolism of lipids 12 1 8.67±2.03 3.09±1.79 0.78
9. Metabolism of amino acids and nucleic acids 9 1 5.12±2.01 2.58±1.79 0.73
10. Integrative human metabolism 8 1 4.36±1.45 1.87±1.69 0.78
Total 97 10 61.98±11.45 28.80±9.63 0.74

Data are presented as number, mean±standard deviation, or %.

MCQs: Multiple choice questions, SEQs: Short essay questions.

Table 2.

Correlations for MCQ Scores, SEQ Scores, and Pass Rate

MCQ scores SEQ scores Pass rate
MCQ scores 1 0.631** 0.112
SEQ scores 1 0.474**
Pass rate 1

MCQ: Multiple choice question, SEQ: Short essay question.



Table 3.

Learning Outcomes of Top and Bottom 25% of the Group Based on MCQ Scores

Achievement Students MCQ scores SEQ scores Pass (○)/fail (X) for the 10 learning outcomes
Pass rate (%)
1 2 3 4 5 6 7 8 9 10
Low L1 42 11 X X X X X 50
L2 42.5 20 X X X X X 50
L3 43.5 20 X X X X X 50
L4 43.5 11 X X X X X X 40
L5 44 26 X X X X X X 40
L6 45 18 X X X X X 50
L7 48 17 X X X X X 50
L8 48.5 25 X X X X 60
L9 50.5 27 X X X X 60
L10 51 16 X X X X X 50
L11 52 21 X X X X X 50
High H1 71 32 X X X 70
H2 71 25 X 90
H3 72.5 42 100
H4 73 35 X 90
H5 74.5 28 X 90
H6 75.5 3 100
H7 75.5 42 X X X X X X X X X 10
H8 78 41 100
H9 82 47 100
H10 84 43 100
H11 86 43 100
Mean 61.98 28.8 86.36

MCQ: Multiple choice question, SEQ: Short essay question.