| Home | E-Submission | Sitemap | Contact us |  
top_img
Korean J Med Educ > Volume 33(2); 2021 > Article
Yo, Witjaksono, Fitriani, Werdhani, and Parikesit: Assessing webinar outcomes for health professionals: a perspective from Indonesia during coronavirus disease 2019 pandemic

Abstract

Purpose

With the ongoing coronavirus disease 2019 (COVID-19) pandemic, health institutions and hospitals are increasingly relying on e-learning for continuing education. However, in many countries there is still limited data on the effectiveness of online learning particularly in the healthcare field. This study aims to evaluate whether webinar as a form of online educational intervention is satisfactory and effective for the continuing education of health professionals in Indonesia.

Methods

We collected participants’ demographic information including health profession, place of work, work unit, and year of graduation. There were six independent webinars included in this study. Webinar outcomes included satisfaction and learning scores. Regarding satisfaction, participants were told to complete a satisfaction survey and asked whether they would recommend the webinar to their colleagues. Regarding learning, information on their mean pre-test and post-test scores was collected.

Results

A total of 3,607 health professionals were enrolled, with the highest participation in webinars about emergency cases and COVID-19 management. The response towards satisfaction was overwhelmingly positive. In all six webinars, post-test scores were statistically significantly higher than pre-test scores. Recently graduated physicians scored higher in learning than senior physicians, while place of work and work unit did not significantly affect the scores.

Conclusion

The use of webinar for health professionals training in Indonesia was well-received amid the ongoing pandemic. In the future, health institutions and teaching hospitals should optimize the implementation of webinar training as it is associated with low cost, high flexibility, and less time commuting.

Introduction

In the era of the Fourth Industrial Revolution, educators from all sectors are required to keep up with the latest information and communication technology. Today’s health professions education, for example, cannot escape the influence of technology on its content and delivery. Medical schools, teaching hospitals, and other health institutions are seeing an increase in the use of e-learning in educating the next generation of health professionals. E-learning, also known as online learning, is referred to as the use of the Internet or other information technologies to acquire knowledge and relevant skills [1]. It can be either asynchronous or synchronous, but the main advantage that e-learning offers is flexibility for participants to adjust their own pace of learning [1]. Although it was historically viewed as a replacement for traditional face-to-face teaching, e-learning is now seen as an adjuvant to transcend the learning process and make it more effective [2]. With the ongoing coronavirus disease (COVID-19) pandemic, online learning has attracted an unprecedented level of fame and has been incorporated by many medical schools around the world in their latest curriculum.
By utilizing technology such as videoconferencing platforms, medical educators spend less time commuting to classrooms and more time tending to their work in hospitals and clinics [3]. One of the forms of e-learning commonly used in today’s continuing education for health professionals is webinar. Webinar is a form of online educational intervention supported by audiovisual software that connects participants and tutors [4]. Its synchronous nature allows people in different locations to easily exchange questions, ideas, and feedback. Previous studies by Gegenfurtner et al. [4,5] concluded that webinars are more effective than both asynchronous learning and no intervention, and they are equivalent to or slightly more effective than traditional face-to-face teaching.
Although the learning efficacy and satisfaction of webinars in other academic fields are well-documented, these measures are still understudied for webinars related to continuing education for health professionals [4-6]. Moreover, healthcare and medical education vary between countries. They are affected by country-specific factors, namely economy, technological advances, political situation, social values, and public view of the educational system [7]. To our knowledge, there has not been any study that specifically addresses webinar outcomes for health professions in Indonesia. Hence, the aim of this study is to evaluate the experience of health professionals in Indonesia after participating in webinar, in terms of their satisfaction and learning outcomes. We hope the result of this study will be able to help health institutions, especially in Indonesia, create more effective webinars for health professions education.

Methods

This is a cross-sectional study that used three set of multi-item questionnaire to evaluate the satisfaction and learning output of participants following six separate webinars conducted by Universitas Indonesia Hospital via Zoom videoconferencing platform and Youtube Live from the hospital’s channel. The webinar series, known as i-CORPS (Interactive Course for General Practitioners), was part of Universitas Indonesia Hospital’s program for continuing education for health professionals. The hospital is part of a higher education institute that also manages a leading medical school in Indonesia. The reason why the hospital conducted this webinar series was due to pandemic constraints on medical education, where there is a need to re-invent continuing medical education (CME) in a way that does not involve face-to-face teaching. Moreover, we acknowledged that many hospitals in Indonesia are facing staff shortage due to the pandemic, and this has led to a significant increase in workload for health professionals. However, this crisis must not discourage health professionals from participating in CME activity as it is part of their professional obligation. Ethical clearance was given by an Institutional Review Board which is the Ethical Committee of Universitas Indonesia Hospital (005/SKPE/KKO/2020/00). Data from participants were safeguarded from potential privacy breaches and reserved only for this research.

1. Webinar background

There were six webinars included in the i-CORPS webinar series. Each webinar was delivered in 2 consecutive days with four to five subtopics and lasted for 1 hour each. Arranged in order from the first to sixth webinar, the themes were as follow: emergency cases in daily practice, disorders of the special senses and nervous system, clinical approach towards COVID-19, medicolegal aspect of medicine and non-clinical medicine, metabolic disorders and nutrition, and emergency cases of the vital organs. Albeit having different themes, all six webinars essentially had the same intended goal which was to provide high-quality and comprehensive CME for health workers in Indonesia in spite of the ongoing pandemic. Ultimately, it was hoped that health workers in Indonesia could still gain positive experience and significant learning improvements after participating in the webinar, since CME in Indonesia before the pandemic was heavily reliant on face-to-face teaching only. Registration was done separately for each webinar through the Universitas Indonesia Hospital official website (https://rs.ui.ac.id/webinar/ ). Therefore, the webinars were independent of one another in terms of scoring and data collection. Each subject could register as participant in as many webinars as they liked.

2. Sample

Subjects enrolled were physicians, nurses, pharmacists, other healthcare workers, and medical students and had to be based in Indonesia. Only those who gave consent to the webinar terms and conditions, followed all the procedures, and completed all the questionnaires were included as the study subjects. We collected relevant personal information about the subjects’ place of work or institution, health profession, year of graduation, and work unit. Following the end of each webinar, subjects were required to complete a satisfaction questionnaire and a post-test. Post-test could be submitted multiple times (up to 30 attempts) to achieve a score of 60 or more, in order to pass the test. The passing score (≥60 out of 100) was set the same across different groups as all participants were given exactly the same material by the same speaker in a particular webinar. Moreover, it was also adjusted with the passing score for the Indonesia National Board Medical Examination (>65) as well as the number of questions in the pre-test and post-test (10 items).

3. Survey data collection

In this study, a satisfaction survey was used to assess satisfaction outcome and the combination of pre-test and post-test scores were used to evaluate learning outcomes following webinar intervention. The scores were also assessed and compared with regards to the participants’ demographic profile such as their health profession, place of work, year of graduation, and work unit. For the satisfaction survey, a 10-point Likert scale (1, very unsatisfied; 10, very satisfied) was used. The satisfaction survey evaluated participants’ satisfaction towards four aspects of the webinar: registration procedure (one item), speaker (4–5 items; one for each webinar subtopic), educational material (4–5 items; one for each webinar subtopic), and videoconferencing platform (one item). An overall mean satisfaction score for each webinar was then calculated by adding up satisfaction scores from all four aspects and dividing the total by the number of survey items. Validity and reliability of the satisfaction survey were confirmed by calculating Cronbach’s α (0.953) and item validation correlation value (0.69–0.82). Also, a single yes/no question asked whether participants would recommend this type of webinar to their colleagues in the future. For pre-test and post-test, their questions were identical in the form of four-options multiple-choice questions. Each correct answer was given a score of 10 while wrong answer was given a score of 0. Questions varied between the six webinars and were developed properly and independently by all speakers using existing literature as reference depending on the assigned topic. Each speaker is a licensed physician and certified teacher in the Universitas Indonesia teaching hospital. The test questions covered various aspects of particular diseases according to each webinar theme, including etiology, pathophysiology, clinical signs and symptoms, diagnosis, treatment, and prevention.

4. Data analysis

Data were analyzed using IBM SPSS ver. 20.0 software (IBM Corp., Armonk, USA). In the descriptive analysis, categorical variables that describe the participants and webinar characteristics were displayed in frequency (N) and proportion (%). For statistical analysis purposes, the profession variable was regrouped into physician and non-physician (medical student, nurse, pharmacist, other healthcare professional) while the work unit variable was regrouped into patient service (emergency, outpatient, inpatient) and non-patient service (managerial and others). Reaction and learning outputs were computed as means and standard deviations, unless otherwise stated. Descriptive statistics were performed to describe the distribution of satisfaction scores reported by participants across all six webinars. Paired-samples t-test and independent-samples t-test were used to test the differences in webinar average pre-test and post-test scores and participants’ post-test scores according to demographic variables, respectively. The cut-off threshold selected for testing significance was p<0.05.

Results

A total of 3,607 health professionals from all around Indonesia participated in the webinar series (Table 1). Most of the subjects were physicians (93.6%) and worked mainly in Java Island (58.1%). Almost half of them (41.6%) were working in a managerial and related position or other unspecified work units. The majority of the subjects (78.9%) were recent graduates, completing their formal education within 2015–2020.
Table 2 shows the frequency and proportion of participants in each webinar. Of all subjects, 45.6% participated in only one webinar, 18% participated in two, 11.3% participated in three, 9% participated in four, 8.2% participated in five, and 7.9% participated in all six.
Table 3 shows the distribution of overall satisfaction mean score and response about webinar recommendation among physicians and non-physicians across all six webinars. Moreover, participants were also asked whether they would recommend this kind of webinar to their colleagues in the future. The responses were overwhelmingly positive, with more than 98% of all physicians and non-physicians saying “yes”.
Table 4 shows the average pre-test and post-test score for each webinar as well as the post-test score in different demographic subgroups according to the profession. In general, the scores statistically significantly increased from pre-test to post-test in all six webinars for both profession groups. Place of work and work unit did not have any statistically significant effect on any post-test score, with few exceptions. In contrast, the year of graduation played a much bigger role in determining the learning outcome for physicians. In all six webinars, the average post-test scores of recently graduated physicians were higher than those who graduated in 2014 or prior. The differences were all statistically significant except in webinar 3.

Discussion

To our knowledge, this is the first and largest study that explores how webinar affects the satisfaction and learning experience of Indonesian health professionals. In this study, satisfaction is defined as the extent to which participants feel satisfied and find the webinar engaging, whereas learning is referred to as the degree to which participants obtain the intended knowledge from the webinar. Since all methods and data collection were performed fully online, we were able to obtain an overview of webinar outcomes from thousands of health professionals spread across Indonesia as well as make meaningful comparisons between them.
Previously, several systematic reviews already evaluated the evidence regarding the benefits of online learning for continuing education for health professionals. Some studies found that e-learning intervention was able to provide a satisfactory learning experience for health professionals and attained comparable learning outcomes to face-to-face teaching [8-10]. Its strengths lie in its ability to offer a convenient mode of learning for busy professionals, enhance collaboration by bringing together participants from different geographical areas, and provide educational material at a low cost [11]. However, some other studies adopted a more ambivalent or even disapproving attitude towards e-learning, in which they reported that e-learning had negligible effect on health professionals’ knowledge and that it was difficult to directly compare e-learning with traditional face-to-face teaching [12-14]. Since conclusive evidence is still lacking, further investigation is warranted.
As the webinar series was mainly marketed to physicians particularly general practitioners, it is not surprising that they constituted 93.6% of all subjects and most of them (78.9%) were recent graduates. The fact that Universitas Indonesia teaching hospital was located in Depok, which is a part of satellite Jakarta metropolitan area, may explain why more subjects were from Java Island than from outside Java. We initially expected that most of them worked in the patient service units as the themes of the webinar were intended to supplement daily clinical practice, but surprisingly almost half of all subjects (41.6%) worked in a managerial and related position. This position includes, but is not limited to, hospital board of directors, academician, researcher, and member of health organizations (e.g., Ministry of Health). This proportion, however, may be overestimated as we did not account for other unspecified work units.
Bodolica and Spraggon [15] mentioned the persistent gap between the macro and micro level of healthcare governance as the main cause of healthcare inefficiencies. Macro governance refers to the role of managers and health organization leaders in managing resources to boost performance and efficiency, whereas micro governance relates to patient-physician relationship. Hospital managers often overlook the quality of patient care, but this pandemic situation might have potentially urged them to seek more active involvement and refresh their clinical knowledge. Hence, high participation of macro-level stakeholders in online educational interventions such as webinar suggests that this method may help bridge the gap within healthcare governance [16,17]. Webinars hold great appeal to all levels of stakeholders as they require low cost, minimum effort, and less time commuting. Currently, India is striving towards electronic health governance (eHealth Governance) which includes rapid development of e-learning programs and interventions to facilitate health professionals training across the country. To do this, the Indian Ministries of Health, Communication and Information Technology, and Family Welfare collaborate to connect medical colleges to hospitals and one another through a telemedicine network [18]. The framework for eHealth Governance in Indonesia is also worth exploring considering the unequal health service distribution across its thousands of islands. However, this needs strong political commitment and willingness to invest in e-learning resources.
Webinars about emergency cases (webinar 1 and 6) and COVID-19 management (webinar 3) had the highest attendance (60.2%), and most participants were from patient service work units worked in the emergency department. This trend may be associated with the current pandemic and state of emergency medicine (EM) in Indonesia. A recent review by Yusvirazi et al. [19] mentioned that in 2018 there were less than 100 EM-trained physicians in Indonesia, so most emergency units are handled by general practitioners who are mostly fresh graduates from medical school. Meanwhile, the number of COVID-19 cases also remains high in the country, so all health professionals are obliged to keep up with the most recent COVID-19-related knowledge. In addition, since the pandemic has worsened staff shortage in health facilities [20], senior workers may feel the need to re-educate themselves outside their area of expertise.
The mean overall satisfaction score reported by all physicians and non-physicians ranged from 8.70 (lowest) to 9.00 (highest). These numbers concluded that participants found the webinar series highly satisfying (Table 3). Our result supports the findings from researchers in Korea University College of Medicine. Due to the pandemic, its medical education department has changed the format for all seminars to be online. In addition to a surge of increase in participation, they found that participants were able to engage better with webinars through the “chat” feature. Although certain issues such as Internet connectivity and unfamiliarity with the webinar platform may cause negative experience, it is evident from the participants’ responses that the benefits outweigh the limitations [21]. A meta-analysis on three different learning methods found that webinar offers higher enjoyment than asynchronous online teaching but lower enjoyment than face-to-face teaching [22]. Nevertheless, the differences are unlikely to be statistically significant. It is too early to tell, however, whether this observed effect is likely to be maintained when face-to-face teaching becomes available again in the future [21].
Regarding knowledge, the significant increase from pre-test to post-test score in all six webinars reflects the intervention’s success. Physicians and non-physicians responded differently in terms of which webinar helped them learn the most. For example, the largest increase in learning output for physicians was shown in webinar 6, where the difference between average post-test score and pre-test score reached 14.1 points. Meanwhile, the biggest increase for non-physicians was shown in webinar 1, where the difference reached 19.2 points. Nevertheless, the findings above suggest that both physicians and non-physicians across all six webinars were able to significantly improve their clinical knowledge following webinar intervention. This may indicate that the webinar speakers were able to successfully adjust the difficulty level of the educational materials, allowing participants to grasp the knowledge easily despite coming from different backgrounds.
The pre-test and post-test questions were designed to be at general practitioner level knowledge, so it may be obvious that recent graduates achieved high post-test scores as they could still easily recall the knowledge from their time in medical schools. The main concern is why senior physicians were not able to score as highly as their junior counterparts. Like in other countries, physicians in Indonesia are obliged to participate in CME for re-registration purposes and lifelong learning [23]. Very limited study has been conducted to evaluate the current condition of CME in Indonesia, but studies in Pakistan and Australia found that physicians in Western countries are much more likely to attend CME than physicians in developing countries. Among those who attend, physicians prefer face-to-face activities to online-based activities. A high number of senior physicians have positive attitude towards CME but do not participate. Reasons vary, but the most common ones are lack of information about the benefits of CME, lack of time, and clash with duty hours [24,25].
The success of i-CORPS webinar series can hopefully serve as a reference for the implementation of future webinars or other online educational interventions. We believe that four main aspects contributed to the webinar success: (1) the webinar series covered a diverse range of medical topics and diseases commonly encountered by health professionals in daily practice; (2) rather than attempting to discuss all topics in one exhausting session, we separated the topics into six webinars that were conducted consecutively every week for 2 days each. This allowed the participants enough time to absorb as much knowledge as possible in a consistent manner; (3) all webinars were recorded, and the recordings were uploaded to a central server where participants can access it again any time they want. This combination of synchronous and asynchronous methods offers flexibility and convenience; and (4) using Zoom videoconferencing platform, the webinar was designed to be highly interactive by utilizing the “chat” and “polling” features. This enabled a two-way communication between participants and speakers that almost resembled face-to-face interaction.
This study is not without limitations. Most of the webinar participants were physicians, so the observed effect of webinar on satisfaction and learning outcomes may not be as pronounced in other health professionals as in physicians. Lastly, since the webinar required good Internet connection, potential participants from rural areas might not be included in this study. Further investigation is also needed to evaluate more accurately the satisfaction and effectiveness of webinar as an online educational intervention for other health professionals. Future research is also needed to study how this webinar series can ultimately impact the participants’ daily practice.
In conclusion, webinar as a form of online educational intervention is a reliable alternative for health professionals training during this difficult time. Considering its positive association with satisfaction and learning, webinar should be integrated with traditional, face-to-face teaching rather than being discarded in the future. However, several limitations of webinar must be addressed, including connection problems, lack of interest in online CME, server overload, and other technical problems. Hospitals and other health facilities are encouraged to organize more webinars and other online learning activities to boost their staff performance without interfering too much with their duty hours.

Acknowledgments

We would like to extend our appreciation and gratitude to all speakers of our webinar as well as all supporting IT staff.

Notes

Funding
No financial support was received for this study.
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
ECY analyzed the data collected, performed literature search, and drafted the manuscript. ANW, DYF, RAW, and DP helped coordinate with the webinar IT staff, assisted in data analysis, reviewed the manuscript, and suggested changes and additions. All authors provided final approval of this version of manuscript to be published.

Table 1.
Frequency and Proportion of Subjects according to Demographic Variables (N=3,607)
Variable No. (%)
Profession
 Physician 3,376 (93.6)
 Medical student 161 (4.5)
 Pharmacist 5 (0.1)
 Nurse 42 (1.2)
 Other health professionals 23 (0.6)
Place of work
 Java 2,094 (58.1)
 Outside Java 1,513 (41.9)
Work unit
 Managerial and others 1,500 (41.6)
 Emergency 1,117 (31.0)
 Outpatient 805 (22.3)
 Inpatient 185 (5.1)
Year of graduation
 Graduated within 2015–2020 2,845 (78.9)
 Graduated in 2014 or prior 762 (21.1)
Table 2.
Number of Subjects in Each Webinar (N=8,692)
Webinar Topic No. (%)
1 Emergency in daily practice 2,330 (26.8)
2 Nerve and special senses disorder 1,358 (15.6)
3 COVID-19 clinical management 1,569 (18.1)
4 Medicolegal and medical support services 853 (9.8)
5 Metabolic disorder and nutrition 1,251 (14.4)
6 Emergency cases of vital organ 1,331 (15.3)

COVID-19: Coronavirus disease 2019.

Table 3.
Overall Satisfaction Score and Response about Webinar Recommendation for Each Webinar
Variable Webinar 1 Webinar 2 Webinar 3 Webinar 4 Webinar 5 Webinar 6
Overall satisfaction score
 Physician 8.92±0.91 8.88±0.99 8.84±1.00 8.85±1.08 8.85±1.07 8.87±1.03
 Non-physician 8.92±0.94 8.75±1.14 8.89±1.04 8.70±1.29 9.00±0.98 8.91±1.08
Participants that would recommend this webinar to their colleagues
 Physician 99.6 99.4 99.5 99.1 99.6 99.8
 Non-physician 99.3 98.7 100.0 100.0 100.0 100.0

Data are presented as mean±standard deviation or proportion (%).

Table 4.
Mean Pre-test and Post-test Scores for Different Demographic Subgroups
Variable Wibinar 1
Wibinar 2
Wibinar 3
Wibinar 4
Wibinar 5
Wibinar 6
Mean±SD No. p-value Mean±SD No. p-value Mean±SD No. p-value Mean±SD No. p-value Mean±SD No. p-value Mean±SD No. p-value
1. Physician
Pre-test scorea) 61.77±15.19 2,188 <0.001* 45.43 ±14.79 1,279 <0.001* 57.23±15.14 1,487 <0.001* 57.13±16.62 813 <0.001* 52.23±15.16 1,202 <0.001* 41.77±14.97 1,272 <0.001*
Average post-test scorea) 73.76±14.05 2,188 54.60 ±14.09 1,279 67.92±13.11 1,487 70.77±14.73 813 63.29±13.64 1,202 55.87±13.73 1,272
Place of work
Javab) 73.70±14.13 1,240 0.808 54.61±13.93 779 0.979 68.38±13.26 901 0.099 70.71±14.85 489 0.888 63.31±13.56 736 0.955 50.34±13.24 778 0.131
Outside Javab) 73.84± 13.95 948 54.59±14.36 500 67.23±12.85 586 70.86±14.58 324 63.26±13.79 466 55.12±14.44 494
Year of garduation
Graduated within 2015-2020b) 75.48±13.15 1,767 <0.001* 55.10±13.51 961 0.041* 68.02±13.14 1,111 0.615 71.97±14.41 547 0.001* 63.88±12.99 809 0.016* 56.68 ±13.44 960 <0.001*
Graduated in 2014 or priorb) 66.54±15.38 421 53.10±15.66 318 67.63±13.05 376 68.30±15.12 266 01.52±15.32 303 53.36±14.32 312
Work unit
Patient serviceb) 72.97±14.26 1,321 0.001* 54.54±14.34 774 0.850 68.10±13.13 897 0.528 70.51±14.82 492 0.540 63.39±13.56 755 0.734 55.64±13.75 824 0.412
Non-patient serviceb) 74.96± 13.65 867 54.70± 13.72 505 67.66±13.10 590 71.16±14.61 321 63.11 ±13.79 447 56.30±13.69 448
II. Not physician
Pre-test scorea) 55.63±16.91 142 <0.001* 45.06±17.75 79 0.002* 51.34±15.05 82 <0.001* 55.25±19.61 40 0.015* 52.04±19.04 49 0.003* 36.10±13.77 59 <0.001*
Average post-test scorea) 74.81 ±16.01 142 52.46 ±15.49 79 60.76±16.28 82 61.85±22.15 40 63.28±17.38 49 50.84±17.19 59
Place of work
Javab) 74.01 ±16.32 91 0.431 50.89±14.30 49 0.250 59.58±15.83 60 0.280 54.46±21.73 19 0.043* 59.96±16.56 22 0.231 50.71 ±19.76 20 0.954
Outside Javab) 76.23 ±15.52 51 55.04±17.21 30 63.98 ±17.41 22 68.54±20.80 21 65.98±17.87 27 50.97±14.63 30

Data are presented as mean±standard deviation.

* p<0.05.

a) Score shown is the mean pre-test score and average post-test score for each webinar; paired-samples t-test was used.

b) Score shown is the average post-test score for various demographic subgroups; independent-samples t-test was used.

References

1. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med 2006; 81(3):207-212.
crossref pmid
2. Frehywot S, Vovides Y, Talib Z, et al. E-learning in medical education in resource constrained low- and middle-income countries. Hum Resour Health 2013; 11: 4.
crossref pmid pmc
3. Barteit S, Guzek D, Jahn A, Bärnighausen T, Jorge MM, Neuhann F. Evaluation of e-learning for medical education in low- and middle-income countries: a systematic review. Comput Educ 2020; 145: 103726.
crossref pmid pmc
4. Gegenfurtner A, Zitt A, Ebner C. Evaluating webinar‐ based training: a mixed methods study of trainee reactions toward digital web conferencing. Int J Train Dev 2020; 24(1):5-21.
crossref
5. Gegenfurtner A, Ebner C. Webinars in higher education and professional training: a meta-analysis and systematic review of randomized controlled trials. Educ Res Rev 2019; 28: 100293.
crossref
6. McKinney WP. Assessing the evidence for the educational efficacy of webinars and related internet-based instruction. Pedagogy Health Promot 2017; 3(1_suppl):47S-51S.
crossref
7. Faghihi A, Hoseini Moghadam M, Yamani N. Analysis of the key factors affecting the future of medical education discipline in 2025 based on STEPV model: a qualitative study. Adv Med Educ Pract 2020; 11: 191-201.
pmid pmc
8. Tomlinson J, Shaw T, Munro A, et al. How does tele-learning compare with other forms of education delivery?: a systematic review of tele-learning educational outcomes for health professionals. N S W Public Health Bull 2013; 24(2):70-75.
crossref pmid
9. Chipps J, Brysiewicz P, Mars M. A systematic review of the effectiveness of videoconference-based tele-education for medical and nursing education. Worldviews Evid Based Nurs 2012; 9(2):78-87.
crossref pmid
10. Rouleau G, Gagnon MP, Côté J, et al. Effects of e-learning in a continuing education context on nursing care: systematic review of systematic qualitative, quantitative, and mixed-studies reviews. J Med Internet Res 2019; 21(10):e15118.
crossref pmid pmc
11. Reeves S, Fletcher S, McLoughlin C, Yim A, Patel KD. Interprofessional online learning for primary healthcare: findings from a scoping review. BMJ Open 2017; 7(8):e016872.
crossref pmid pmc
12. Vaona A, Banzi R, Kwag KH, et al. E-learning for health professionals. Cochrane Database Syst Rev 2018; 1(1):CD011736.
crossref pmid
13. Lahti M, Hätönen H, Välimäki M. Impact of e-learning on nurses’ and student nurses knowledge, skills, and satisfaction: a systematic review and meta-analysis. Int J Nurs Stud 2014; 51(1):136-149.
crossref pmid
14. Regmi K, Jones L. A systematic review of the factors - enablers and barriers - affecting e-learning in health sciences education. BMC Med Educ 2020; 20(1):91.
crossref pmid pmc
15. Bodolica V, Spraggon M. Clinical governance infrastructures and relational mechanisms of control in healthcare organizations. J Health Manag 2014; 16(2):183-198.
crossref
16. Bodolica V, Spraggon M, Tofan G. A structuration framework for bridging the macro-micro divide in health-care governance. Health Expect 2016; 19(4):790-804.
crossref pmid
17. Brinkerhoff DW, Bossert TJ. Health governance: principal-agent linkages and health system strengthening. Health Policy Plan 2014; 29(6):685-693.
crossref pmid
18. Nair P. ICT based health governance practices: the Indian experience. J Health Manag 2014; 16(1):25-40.
crossref
19. Yusvirazi L, Ramlan AA, Hou PC. State of emergency medicine in Indonesia. Emerg Med Australas 2018; 30(6):820-826.
crossref
20. Bong CL, Brasher C, Chikumba E, McDougall R, Mellin-Olsen J, Enright A. The COVID-19 pandemic: effects on low- and middle-income countries. Anesth Analg 2020; 131(1):86-92.
crossref pmid
21. Lee YM, Park H, Pyun SB, Yoon YW. Enforced format change to medical education webinar during the coronavirus disease 2019 pandemic. Korean J Med Educ 2020; 32(2):101-102.
crossref pmid pmc
22. Ebner C, Gegenfurtner A. Learning and satisfaction in webinar, online, and face-to-face instruction: a metaanalysis. Front Educ 2019; 4: 92.
crossref
23. Mustika R, Nishigori H, Ronokusmono S, Scherpbier A. The odyssey of medical education in Indonesia. TAPS 2019; 4(1):4-8.
crossref
24. Ali SA, Hamiz Ul Fawwad S, et al. Continuing medical education: a cross sectional study on a developing country’s perspective. Sci Eng Ethics 2018; 24(1):251-260.
crossref pmid
25. Stewart GD, Khadra MH. The continuing medical education activities and attitudes of Australian doctors working in different clinical specialties and practice locations. Aust Health Rev 2009; 33(1):47-56.
crossref pmid
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-747-6206
E-mail : kjme_office@daum.net
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2021 by Korean Society of Medical Education.                 Developed in M2PI