The flipped classroom has been suggested as a method for efficient teaching in medical education. However, its feasibility and effectiveness in the educational environment are often overlooked. The authors redesigned an epidemiology course applying the flipped classroom method under a traditional curriculum consisting of heavily scheduled classroom hours and explored its feasibility and effectiveness.
In the fall semester of 2017, we flipped the ‘practice of epidemiology’ course for third-year medical students at Korea University College of Medicine. We provided online lectures and assigned readings as pre-class materials, and substituted group discussions and communicative activities for traditional lectures. We conducted pre- and post-course surveys to review students’ perceptions. We also analyzed the pre-test results and final exam scores for quantitative comparison.
Ninety-seven students out of 120 completed the course. Most students made use of the online lectures, but not the reading materials. Lack of time was the most frequently cited reason for under-preparedness. We observed improvements in preparedness, participation, and effectiveness at the end of the course, while changes in satisfaction were unclear. Students’ perceptions of course relevance and difficulty were predictive of pre-test outcomes, but the effects of preparedness and length of materials were insignificant. The authors found no evidence of differing test scores before and after the course.
This study supports the feasibility of the flipped classroom model even under a traditional, heavily scheduled medical curriculum. To encourage self-directed learning and achieve better learning outcomes, restructuring pre-existing curricular components should also be considered in parallel with new instructional methods.
Recent research in medical education underscores the importance of early clinical exposure supported by competency-based learning, facilitating students’ development of practical skills and providing more opportunities for medical reasoning [
The idea of the flipped classroom is attractive in that it effectively utilizes limited time as well as facilitating interaction to improve learning outcomes [
However, it is unclear whether this approach can also be conducted effectively in contexts of relatively conservative and traditional educational culture, including: (1) heavily scheduled class hours with little time for students’ self-directed learning; (2) situations where most teachers still prefer the traditional lecture format; (3) low trust in students’ self-directed learning ability; and (4) limited educational resources, such as education-dedicated faculties, compared to western and some advanced Asian medical schools.
Therefore, the purpose of this study was to examine the feasibility of a flipped classroom model in the context of a conventional lecture-based curriculum.
Practice of epidemiology is a required course for the third-year medical students of the 6-year curriculum at Korea University College of Medicine. It was scheduled as a 2-hour, once- weekly class that lasted for 5 weeks in the 2017 fall semester.
Prior to the first session, students were informed that the flipped learning method would be implemented in the course. They were also informed that there would be a pre- and post-course survey to evaluate their perceptions of and satisfaction with the course, but participation in the survey was entirely voluntary. For in-class activities such as group discussions, students were randomly allocated to one of 10 groups comprising 12 students each.
Among the 120 students who initially registered for the course, two students withdrew to take a gap year and 21 students refused to turn in the course survey. The final data collected from 97 students (65 males, 32 females; mean age, 22.4 years) were included in the analysis.
Four qualified personnel from the Department of Preventive Medicine assisted with the course as tutors: a professor, M.D., Ph.D., who oversaw the course; a research associate professor with a Ph.D. in epidemiology; and a preventive medicine specialist and a junior resident, who were both Ph.D. candidates. Each tutor was asked to organize one or two sessions and prepare the corresponding course materials, such as online lectures. They were also assigned as supervisors to two or three groups to facilitate group discussion or help the students stay on track.
Pre-class materials, including online lectures and assigned reading materials, were provided to students in advance for independent self-study. Online lectures were produced in the ‘Diamond Studio’ at Korea University College of Medicine, a facility fully equipped with a professional-class audio/video recording system and content-editing software. To ensure student focus for the duration of the online lectures, the tutors were encouraged to adhere to a maximum of 30 minutes of recording time [
The tutors motivated the students for independent study with persistent reminders to complete the preclass learning activities prior to class. For this purpose, we instructed the tutors to create pre-tests consisting of 10 multiple-choice questions from their designated sessions. We urged the tutors to create simple questions focused on the content covered in the pre-class materials, since the primary purpose of the test was to assist students’ preparedness rather than to grade them. At the beginning of each session, the students took a 10-minute pre-test. Group discussion or questions were not allowed during this time.
We provided case studies for each session as in-class materials. Three cases were originally developed by the Center for Disease Control and Prevention, and one each by the Association of Teachers of Preventive Medicine and the Korean Society for Preventive Medicine. Each case began with a brief description of actual epidemiological events, followed by 10 to 20 step-by-step questions logically designed to explain how learners can approach certain concepts of epidemiology. The students were encouraged to manage the questions through active group discussion or online searches based on what they had learned from the pre-class materials. During this period, the tutors also actively rotated among the assigned groups to ascertain whether the students had any difficulties, answer their questions, and provide detailed explanations when necessary. At the end of each session, the students summarized the results of the case study based on the group discussions. They gave a short presentation of their work, and the tutors provided feedback. Finally, the students submitted a brief group report on the case study.
The group reports were evaluated by a tutor on the same day they were submitted. The focus of this evaluation was to check for student misunderstandings of key concepts and common errors. Supplementary comments along with a final summary of the key issues from each session were added to the reports and sent back to the students on the same day. The report feedback was managed by a single tutor throughout the course to ensure timely responses, consistency, and uniform explanations of the key concepts. At the beginning of each session, the students were provided with grade reports from the previous session’s pre-test so as to keep them on track. The report was also given after the mid-term exam of the affiliated ‘preventive medicine’ course. The students who scored in the bottom percentile were given an alert advising them to pay more attention to the class materials. In addition, for the students who sought additional help or had difficulties with the independent study, the tutor provided a personalized counseling session at their request.
On the day of each session, the tutors held a preview conference led by the organizing professor. The tutors reviewed the results from the previous session, such as the pre-test scores or group reports, and discussed additional explanations to be provided to the students when necessary. They also previewed the course materials for each day and identified the key concepts to be delivered to ensure that sessions were facilitated in a uniform manner.
The structure of the ‘practice of epidemiology’ course is summarized in
We planned to evaluate students’ performance change (i.e., quantitative evaluation) and adaptation to the new setting. For the former, we first analyzed the results of five pre-tests to measure the effects of the pre-class materials. Furthermore, we also assessed the relevance of the pre-class study and pre-test results through multiple regression analysis. We compared the results of the pre-tests and final exams to assess the effectiveness of the flipped learning method. For the latter, we conducted pre- and post-course surveys using a questionnaire composed of questions about various aspects of the flipped classroom design [
Since the medical students were defined as vulnerable study participants, we strictly observed the relevant requirements and regulations. The students were informed about the study’s purpose and associated data collection along with the course outline. Those who agreed with the study purpose and data collection submitted a written consent form voluntarily. The overall study design, including student protection measures, was reviewed and approved by the Korea University Institutional Review Board (approval no., 1040548-KU-IRB-17-205-A-1).
We evaluated students’ utilization and perception of the pre-class materials in four aspects: preparedness, relevance, difficulty, and length (
Students’ opinions of their states of preparedness are suggested in
We considered the four aspects of the utilization and perception of pre-class materials described in
The pre- and post-course survey of students’ attitudes and perspectives on the ‘practice of epidemiology’ class are described in
The authors’ new trial of flipping the class in the undergraduate epidemiology course showed promising results in terms of students’ preparedness, class participation, and effectiveness in understanding the course. The students were able to understand the working logic of a flipped classroom after finishing the course and were prepared to meet the learning objectives of problem-solving competency against actual public health issues. They evaluated the effect of interactions with peers and tutors for better understating of the course highly. Their attitudes toward active participation in class discussions and presentations also improved significantly throughout the course, as in a previous study [
However, it was not apparent whether the students gained more interest in this subject or were satisfied after the course. After reviewing students’ written comments on our course (
Meanwhile, our findings showed that the online lectures were instrumental in the students’ learning. Our finding that a significant proportion of the students watched the online lectures prior to class was consistent with previous studies [
The students’ indifference toward and low utilization of the reading materials can be understood likewise. Even in studies showing the positive effects of online lectures, the role of reading materials was inconclusive [
In short, significant attention should be paid to producing more organized and relevant pre-class materials based on established instructional rules and principles [
We compared the summary of pre-test and final exam scores to evaluate the quantitative effectiveness of the flipped learning approach. To minimize irrelevant effects from a small-group study room where the students could discuss their answers with others, we prepared individualized test sheets in last two sessions. We also repeated several questions from the pretests on the final exam to evaluate the learning outcome. Nevertheless, neither a significant improvement nor a decline in student scores was observed at the end of the course, in line with studies in similar settings [
To the best of our knowledge, few existing studies have described the specific composition of tutors. We successfully organized the flipped class with four qualified instructors who also participated as tutors, and two graduate school students who helped run the sessions. Medical school staffs in Korea, as well as other countries with under-resourced educational environments, have heavy workloads that involve patient care, research, and student education at the same time. Moreover, since medical education has such a small window for trial and error because of its special purposes, there are not many opportunities for the implementation of new teaching formats. Under these circumstances, institutions willing to implement the flipped learning model should also take the available workforce into consideration as a significant aspect of the requirements.
We would like to share the lessons from this study with educators who are considering a flipped classroom model in limited or similar educational contexts as ours. The provision of redundant lectures is one issue that might have affected our study. Although there are no stated rules, it is recommended that the flipped classroom entirely flip the class and thus involve no in-class lectures [
Some limitations on our study should be considered. Since our main purpose was the implementation of a new system, we adopted survey items and domains from previous studies [
In conclusion, our study supports the feasibility of the flipped classroom model even under a traditional, heavily scheduled undergraduate medical curriculum, especially in terms of students’ preparedness and engagement in the course. To encourage self-directed learning and achieve better learning outcomes, however, we also suggest considering restructuring other preexisting curricular components in parallel with implementing a novel instructional method.
Supplementary files are available from
Reasons for Incomplete Preparedness or Repeated Study of Course Materials.
Students’ Post-course Feedback on the Flipped Learning Design.
Students’ Post-course Feedback on Course Components of the Flipped Class Design.
The authors would like to express their gratitude to the 2017 first-year medical students of Korea University College of Medicine for their active participation in this study. The authors would also like to thank Korea University Graduate School of Public Health students Wonju Cho and Jin A Kim for their support in running the class and facilitating the administrative work. Finally, the authors would like to express their deep appreciation to Medical Education Center research assistant Hyelim Kim and Academic Affairs Department staff member Yongjin Lee, who assisted the study with their expert management of the students and course evaluations, and who willingly shared their expertise and evaluation tools.
Not applicable.
No potential conflict of interest relevant to this article was reported.
Conception and design of the study: SS, YML, BCC; acquisition, analysis, and interpretation of the data: SS, BCC; acquisition of study data: JJ, ESC; critical revision: YML, JJ, ESC; drafting the article: SS; and final approval of the version to be published and agreement of its publication: SS, YML, JJ, ESC, BCC.
Set A: Scores on the fourth and fifth session pre-test questions and final exam questions corresponding to those sessions. Set B: Scores on the subsets of questions from the fourth and fifth pre-tests that were repeated on the final exam, and final exam questions that were identical to those questions.
Overview of Pre-class Materials for the Practice of Epidemiology Course
Session title | 1. Descriptive epidemiology: epidemic disease in a community | 2. Validity and reliability: screening for human immunodeficiency virus | 3. Surveillance system: injury surveillance | 4. Outbreak investigation: students have diarrhea |
5. Study design: oral contraceptives and ovarian cancer |
---|---|---|---|---|---|
Video length (min) | 21 | 32 | 24 | One: 15 | 16 |
Two: 29 | |||||
Views (%) | 27 (22.5) | 95 (79.2) | 128 (101) | One 106 (88.3) | 85 (70.8) |
Two: 73 (60.8) | |||||
Assigned reading(s) | 18 Pages | 21 Pages | 5 Pages | Not assigned | 27 Pages |
Pre-test questions | 6 MCQ+2 short-answer | 10 MCQ | 8 MCQ | 10 MCQ | 10 MCQ |
MCQ: Multiple-choice question.
Two online lectures were prepared for the fourth session.
Students’ Utilization and Perception of Online Lectures and Assigned Readings
Response items | Session 1 | Session 2 | Session 3 | Session 4 | Session 5 |
---|---|---|---|---|---|
Online lectures | |||||
Watched before class | |||||
More than once | 4 (4.2) | 4 (4.1) | 5 (5.3) | 3 (3.1) | 5 (5.2) |
Once | 63 (65.6) | 68 (70.1) | 71 (74.7) | 70 (72.9) | 75 (77.3) |
Partly | 18 (18.8) | 10 (10.3) | 7 (7.4) | 14 (14.6) | 7 (7.2) |
Did not watch | 11 (11.5) | 15 (15.5) | 12 (12.6) | 9 (9.4) | 10 (10.3) |
Relevance in pretest |
3.68±1.15 | 3.04±1.01 | 2.90±1.07 | 2.72±1.02 | 1.90±1.01 |
Difficulty |
2.60±0.89 | 3.10±0.95 | 3.30±0.92 | 3.51±0.97 | 3.70±1.14 |
Length |
3.06±0.71 | 3.22±0.72 | 3.19±0.76 | 4.40±0.85 | 2.57±0.91 |
Assigned readings | |||||
Read before class | |||||
More than once | 2 (2.1) | 1 (1.0) | 2 (2.1) | NAd) | 2 (2.1) |
Once | 21 (21.9) | 20 (20.6) | 20 (20.8) | 18 (18.6) | |
Partly | 20 (20.8) | 11 (11.3) | 13 (13.5) | 13 (13.4) | |
Did not read | 53 (55.2) | 65 (67.0) | 61 (63.5) | 64 (66.0) | |
Relevance in pretest |
3.46±0.71 | 3.19±0.79 | 3.09±0.75 | NA | 2.90±1.06 |
Difficulty |
3.07±0.69 | 3.48±0.77 | 3.38±0.78 | NA | 3.47±1.08 |
Length |
3.35±0.74 | 3.43±0.68 | 3.26±0.75 | NA | 3.56±0.88 |
Data are presented as number (%) or mean±standard deviation.
NA: Not assessed.
Measured on a 5-point Likert scale (1, not helpful with the pretest; 5, helpful with the pretest).
Measured on a 5-point Likert scale (1, easy; 5, difficult).
Measured on a 5-point Likert scale (1, short; 5, long). d)Session 4 did not have assigned reading material.
Multiple Regression on Students’ Preparedness and Perception of Course Materials Influencing Pre-test Score
Factor | Course material |
|||||
---|---|---|---|---|---|---|
Online lectures |
Assigned readings |
|||||
Coefficient | SE | p-value | Coefficient | SE | p-value | |
Preparedness |
1.364 | 2.738 | 0.618 | -0.947 | 3.54 | 0.79 |
Relevance |
7.971 | 0.952 | <0.001 | 7.638 | 2.61 | 0.004 |
Difficulty |
-6.333 | 1.102 | <0.001 | -6.01 | 2.953 | 0.044 |
Length |
2.92 | 1.09 | 0.008 | 1.46 | 3.528 | 0.68 |
SE: Standard error.
Higher preparedness indicates more watching/reading of course materials in advance.
Higher relevance indicates that the respective course material was helpful for the pre-test. Measured on a 5-point Likert scale (1, not helpful with the pre-test; 5, helpful with the pre-test).
Measured on a 5-point Likert scale (1, easy; 5, difficult).
Measured on a 5-point Likert scale (1, short; 5, long).
Students’ Attitudes and Perspectives on the Practice of Epidemiology Course before and after Completion
Survey question |
Pre-course | Post-course | p-value |
---|---|---|---|
Preparedness | |||
I understand the purpose and study objective of the course. | 3.26±0.94 | 3.57±0.99 | 0.03 |
Participation | |||
I actively participate(d) in in-class activities such as group discussions or presentations. | 3.38±1.15 | 3.80±1.01 | <0.001 |
I try (tried) to make up for what I don’t (didn’t) understand clearly by asking for help (ask instructor, ask/discuss with friends/refer to supplementary materials, etc.). | 2.78±1.05 | 3.58±1.02 | 0.08 |
Effectiveness (in understanding the course) | |||
I have (had) no problem understanding the course. | 3.35±0.99 | 2.90±1.07 | 0.39 |
Group discussions or presentations are (were) helpful in understanding the course. | 2.99±0.97 | 3.42±1.01 | <0.001 |
I understand the key points of the course. | 2.97±0.95 | 3.32±0.91 | <0.001 |
I am confident about how to study for the course. | 2.99±0.90 | 3.04±1.08 | 0.21 |
Satisfaction | |||
I had/gained interest in preventive medicine. | 2.85±0.95 | 3.07±1.17 | 0.82 |
I am generally satisfied with the course. | 3.03±0.97 | 3.32±0.98 | 0.93 |
Data are presented as mean±standard deviation.
Measured on a 5-point Likert scale (1, strongly disagree; 5, strongly agree).