The profession of a physician is inherently demanding as it deals with the quality of life of another human being. Physicians everywhere are morally obliged to keep up with the latest information and knowledge on treatments and diseases. Indeed, the four principles of health care ethics, including beneficence and non-maleficence, have been widely adopted to guide physicians in their clinical decision-making. Various academic institutions and health experts agree that continuing medical education (CME) is one effective method that enables physicians to update their knowledge and skills throughout their careers continuously. This term describes educational activities aimed at physicians to maintain and improve their professional knowledge and skills, ultimately leading to better patient care. A review by Cervero and Gaines [1] proved that CME is strongly associated with better physician performance and patient health outcomes.

A study by Cullen et al. [2] strongly suggested that CME must evolve to suit the needs of modern physicians in today’s digital age. Nowadays, physicians are busier than ever as they are increasingly overwhelmed with long work hours, bureaucratic tasks, financial issues, stressful on-call duties, and other work-life demands [2,3]. The ongoing coronavirus disease 2019 (COVID-19) pandemic has further worsened the situation, as it has led to staff shortage in many hospitals and eventually higher workload and risk of burnout for the remaining health care providers [4]. The problems as mentioned earlier are particularly evident in low-to-middle-income countries, where a physician’s workload is more physically and mentally draining due to a shortage of physicians. In a 2014 report by World Health Organization, most provinces in Indonesia (87.8%) had a lower ratio of physicians per population than the recommended 1:1,000 ratio [5]. Therefore, as physicians struggle to meet patients’ rising needs, there is an urgent need for a more accessible CME delivery method that requires less effort, cost, and commute time. Modern CME must shift from implementing solely traditional, face-to-face courses to integrating online and interactive educational activities. An example of online CME activity is a webinar, an online training program that utilizes the videoconferencing platform to offer synchronous learning.

Online CME offers excellent flexibility for physicians to set their schedule and pace of learning. Moreover, as many online courses are often recorded, saved, and uploaded, physicians can conveniently access them anytime and review the materials. Data from Mayo Clinic Cardiovascular Review for Cardiology Boards and Recertification showed that while physicians spent up to 50% less time engaging with online courses, their knowledge significantly increased by three-fold from pre-test to post-test [2]. Previous studies also supported the effectiveness of online CME in terms of physicians’ satisfaction, knowledge gain, and behavioral changes [6,7]. Interestingly, a recent study concluded that physicians were open to the idea of online CME programs as a substitute for face-to-face courses as they encouraged more participation in CME, promoted more engagement, and reduced the need and time to travel [8].

Although the effectiveness of online CME can be assessed based on participants’ knowledge gain and satisfaction, what ultimately matters to patient care is how much new knowledge is retained for long-term memory and how this knowledge translates into changes in daily practice and personal development. Physicians’ newly acquired knowledge following online training can be significantly lost in as little as several days to 2 months [9]. In addition, teaching hospitals and medical educators should utilize CME to improve physicians’ knowledge and assist them in enhancing their clinical performance and career development [10]. This study aims to analyze knowledge retention of physicians 3 months after participating in a webinar training and how they perceive the webinar series’ benefits on their daily practice and career development.

This research is a follow-up study from our previous research that evaluated participants’ satisfaction and learning outcomes following the i-CORPS (Interactive Course for General Practitioners) webinar series held by Universitas Indonesia Hospital [11]. The hospital is a teaching hospital that is part of a prominent medical school in Indonesia, the Faculty of Medicine, Universitas Indonesia. Considering the problems that CME faces amid the pandemic, the purpose of the i-CORPS webinar series was to explore contact-free teaching through online webinars and evaluate its effectiveness. Its primary target audience was physicians but was also open to other health professionals around Indonesia. The webinar series consisted of six webinars with different themes conducted separately, live-streamed on the Zoom videoconferencing platform, and uploaded to YouTube afterward for future access. The webinar series was conducted between September and October 2020, with each webinar being held on 2 consecutive days for 1 hour each. Each webinar covered four or five subtopics, and the main themes included COVID-19 clinical approach, diseases affecting the special senses and nervous system, emergency cases management, nutrition, and metabolic disorders, medico-legal medicine, and other related fields. The Ethical Committee of Universitas Indonesia Hospital granted ethical clearance for this study (005/SKPE/KKO/2020/00). This study evaluates the knowledge retention of physicians who participated in the webinar series, how their daily practice has been impacted over a 3-month timeframe, and how their career development for the future has been affected.

Despite concerns that any new knowledge can be quickly forgotten, especially after an extended period, there has been limited research into knowledge retention following online learning in the medical field. To our knowledge, this is the first study in Indonesia that investigated knowledge retention of physicians after online CME and its impact on their daily practice and career development. The study found a significant loss of knowledge among physicians 3 months after acquiring it from the webinar series. This result was consistent with Bell et al. [9], who observed significant retention loss after 3–8 days and no significant retention 55 days after the physicians completed an online tutorial on diabetes care guidelines. In one of their older studies, the physicians only retained 40% of their initial knowledge gains 4–6 months after completing an online tutorial on postmyocardial infarction care [14]. The retention loss observed in our study participants was not as extreme, but it remained an essential factor to address in order to improve online CME in the future. However, it is essential to note that this study’s findings might not be directly comparable to other studies’ findings on knowledge retention as the type of online CME might differ regarding the method of delivery and materials covered.

The reason for retention loss in our study participants might be the nature of the i-CORPS webinar series, which was delivered as a one-time event without a follow-up refresher course. A systematic review by Cook et al. [15] found four aspects associated with better learning outcomes in online learning for health professions education: interactivity, practice exercises, repetition, and feedback. The webinar series had incorporated three of these elements in its delivery, but the repetition of the materials was not well-addressed. All participants were given access to revisit the webinar materials online anytime on Youtube, but this was not reinforced and remained optional for them. Meanwhile, a randomized controlled trial involving 44 physicians assigned to an online CME course showed that the participants benefited from the repetition of the course, which was conducted 1 month after. The repetition was in the form of a 45-minute instructor-led web conference, and it was joined by 85% of the participants. A small but significant increase from participants’ scores at the post-test immediately after the course to participants’ scores at 12 weeks after the course [16]. Additionally, ensuring interactivity in learning is arguably one of the biggest challenges in conducting an online CME course. Although the i-CORPS webinar series has anticipated this problem by encouraging the instructors to be engaging, for example, using the “chat” and “polling” features in Zoom [11], this remains a limitation worth considering for future online CME courses. An interactive learning experience was associated with better knowledge retention, and failure to establish a two-way communication involving feedback may negatively affect participants’ attention span and retention level [17]. In a face-to-face educational course, the instructor can better engage with the participants since they are in the same room. Participants are also able to provide immediate feedback or ask questions directly to the instructor. However, in an online course, such two-way communication between instructor and participants can be greatly restricted due to multiple reasons such as lack of physical presence and poor Internet connectivity. A previous study found that students mainly struggled to pay attention and be motivated in an online course as they viewed the instructor on screen as “invisible”, “absent”, “passive”, and a “hybrid between a human and a computer”. This physical separation made it difficult for students to stay focused, get to know their instructor personally, and share the instructor’s enthusiasm for the subject [18]. Regarding connectivity, poor Internet connection can lead the videoconferencing platform to lag or freeze, therefore causing the audio or image to become out of sync. This eventually causes misunderstandings and communication delays between the instructor and participants.

Physicians’ age, education level, type of work, and place of work did not appear to influence their knowledge test scores at the follow-up survey. On the other hand, knowledge test scores at follow-up significantly differed by physicians’ age, year of graduation, and length of work. It was not surprising that physicians tended to absorb new knowledge much slower as they grew older. This phenomenon has significant implications on physicians’ retirement age, which has sparked debate over recent years about whether it must be regulated and its impact on the medical workforce. Some worry that aging physicians are more prone to making errors that disadvantage the patients, but others believe that allowing them to remain in practice can help mitigate the growing physician shortage in many countries [19-21].

Nevertheless, since age is just one of the many factors that affect older physicians’ performance, it must not be regarded as the sole predictor of competence [19-21]. One way to ensure aging physicians maintain their competence is by reinforcing CME and making it more accessible and convenient, which can be achieved by promoting online learning with the i-CORPS webinar series as an example. All webinars in the series were live-streamed on Zoom and recorded for future access, reducing the need for participants to spend excessive time, money, and travel on CME. This enabled not only younger but also older physicians to participate with great convenience. However, it must be noted that older physicians may feel uncomfortable or unfamiliar when using digital technologies, including video conferencing platforms. If this limitation is not correctly addressed, online CME can be counterproductive and hinder older physicians from participating.

Interestingly, this study also found that recently graduated physicians with less than 3 years working period scored significantly higher on the knowledge test than the more senior physicians. While it could be argued that fresh medical graduates might score higher because they could still remember a sheer amount of knowledge from their medical school years, we initially expected the senior physicians who had worked longer to score higher as they had more clinical exposure to patients in daily practice. The questions in the knowledge test covered clinical aspects of various diseases encountered in daily practice, which included their etiology, pathophysiology and pathogenesis, signs and symptoms, diagnosis, management, and prevention. This result further supported the findings in our previous study, in which we also discovered that physicians who graduated more recently (2015–2020) had higher average post-test scores than physicians who graduated in 2014 or prior in all six webinars [11]. We hypothesized that most recently graduated physicians could engage better with the webinar series and hence absorbed the material better than senior physicians. Hence, the importance of regularly updating clinical knowledge by participating in structured CME must be re-emphasized to all physicians regardless of their rank, seniority, or work history.

Although physicians with more extended work history scored lower on the knowledge test at follow-up, they perceived the webinar series’ benefits on daily practice more positively than those who had just begun medical practice. Those with more extended work history presumably had more experience in managing patients and could better apply the newly acquired knowledge from the webinar series to cases encountered in daily practice. Knowledge gain must not be used as the only parameter for determining the success of online CME since how relevant and applicable; it is to physicians’ work is as important, if not more so. Fortunately, there may be a mutualistic relationship between perception of online learning experience and knowledge gain. Cook and Thompson [22] implied that having a favorable perception of online learning was positively associated with improved physicians’ knowledge. Thus, future online CME must evaluate how well participants improve their knowledge, how satisfied they are with the course, and how they perceive its benefits in daily practice.

In this study, male physicians perceived the webinar series’ benefits on career development more positively than their female counterparts. In other words, male physicians could be putting more thought into how the webinar series might have influenced their careers more than female physicians. Although this trend could be multifactorial, it might reflect gender stereotypes among physicians to some extent. Previous studies already discussed this difference, in which they found that female physicians were less likely or slower to take on higher positions in medicine, pursue more advanced qualifications, and work in larger hospitals [23,24]. However, the difference in the rate of career development was more likely caused by female physicians’ tendency to bring up family and switch to part-time work rather than by gender discrimination [23-25]. Parenthood appeared to be negatively associated with career development, particularly for women [23]. This is especially true in Indonesia, where the traditional roles of men as breadwinners and women as family caregivers are still deeply rooted in its culture. Thus, utilizing the online form of CME may be a feasible strategy to give physicians from both genders equal opportunities as it offers a more flexible schedule that can be self-paced accordingly.

There are some limitations to this study. First, there was an unequal distribution of study participants based on specific demographic subgroups, resulting in a less accurate representation. In addition, since a physician could participate in more than one webinar and therefore would complete more than one questionnaire, such possibility of duplicate participation can lead to a less accurate representation. The cross-sectional nature of this study also limited its ability to investigate the long-term impact of the webinar series on physicians’ behavioral changes and quality of patient care. Since this study only evaluated one platform of online CME, which was webinar series, further investigation is needed to evaluate other platforms of online CME and eventually compare their efficacy. More research is also needed to determine the ideal time for conducting a follow-up refresher course after an online CME program to optimize knowledge retention. Refresher courses can be delivered in the form of case-based discussions to be more relevant to the physicians’ daily practice.

In conclusion, the decline of knowledge retention over time remains an important issue to address for future online CME programs. Properly structured refresher courses should be offered to physicians to help them retain the newly acquired knowledge, although the ideal interval from the initial course needs further study. The importance of participating in CME must be reemphasized to physicians of all ages, experiences, positions, and genders because CME is perceived to be beneficial for their clinical knowledge and competence in daily practice as well as career development. Utilizing the online platform of CME may allow for broader participation as it offers flexible learning and requires less time and cost.