Identifying teaching competencies for medical residents using a modified Delphi method
Article information
Abstract
Purpose
This study aimed to identify the teaching competencies of medical residents.
Methods
A modified Delphi study was conducted from January to March 2017. Twenty-four panelists (six medical educators, program directors, chief residents, and residents each) from various facilities in Japan participated in the study. The consensus criterion for this study was that more than 80% of the panelists gave a rating of 6 or higher on the 7-point Likert scale (“not at all important” to “extremely important”) without any comments. The modified Delphi approach resulted in a list of 27 resident teaching competencies after three rounds. These competencies were categorized based on Harden and Crosby’s 12 roles of medical teachers.
Results
Our study revealed that, of the 12 roles, residents were primarily viewed as “clinical or practical teachers,” “teaching role models,” “on-the-job role models,” “learning facilitators,” and “student assessors.”
Conclusion
The 27 resident teaching competencies indicate the importance of educational proximity for residents as teachers. It is expected that this finding will contribute to competency-based resident-as-teacher education.
Introduction
Clinical teachers play an important role in student training [1]. Effective teachers can motivate students, promote their learning, and maximize the learning potential of students in workplaces [1,2]. The teaching role of medical residents has been studied for decades, and the importance of the role continues to grow. Residents are considered to play a pivotal role in clinical education because they can act as near peer teachers to medical students and younger residents [3].
There are no separate residents in charge of students or younger residents and residents does not play teaching role officially in Japan. As in other countries, the main purpose of residency training is to gain clinical training. However, in fact, institutions of undergraduate and postgraduate training recommend that residents strengthen their teaching skills. The Liaison Committee on Medical Education [4] and the Accreditation Council for Graduate Medical Education [5] require that residents develop skills to educate patients, families, medical students, other residents, and other health professionals. In Japan, the new clinical training teaching guidelines issued by the Japanese Ministry of Health, Labour and Welfare [6] approve of residents teaching other residents officially for the first time.
Workshops have long been held to improve residents’ teaching skills. These workshops are called residents-asteachers (RaT) programs [3]. To our knowledge, the first research article to suggest the effectiveness of RaT programs was published in 1982 by Jewett et al. [7]. Since then, the number of such programs has increased. By 2016 more than 80% of American resident programs, including internal medicine, pediatrics, psychiatry, family medicine, and obstetrics/gynecology, seemed to include RaT programs [8].
However, residents were and still are very busy and at high risk for burnout [9]. It may thus be too much to place the burden of learning teaching skills on them any longer. If they are to be trained in teaching methods, they need to be selective about what they are exposed to in the wide range of teaching methods.
In fact, a survey of RaT programs pointed out that the contents and methodologies of such programs varied [8]. In terms of quality, program directors might not be satisfied with their RaT programs; thus, American program directors have demanded a national curriculum [10]. Even the recently published guideline on RaT [11], including other recent study [12], and one of the most-cited studies on this topic [13] have not developed a competency-based education method. None of these studies used formal consensus group methods, such as the Delphi method and nominal group technique [14]. To the best of our knowledge, no competency-based curriculum for RaT proposed by the scientific method. has been developed. Therefore, we believe that this might be an issue outside of Japan as well.
Amid growing calls for competency-based medical education, the inclusion of medical educator competencies has been advocated [15]. Srinivasan et al. [15] provided a conceptual framework of teaching competencies by teaching roles, such as clinical teachers, large group teachers, program directors, or educational researchers. However, there was no mention of residents in their study. Therefore, we identify a problem: the teaching competencies of residents are not widely recognized.
A few studies have attempted to define the qualities of, and techniques used by good resident teachers. However, these studies have focused on learners, omitting many other stakeholders involved in the development of consensus resident teaching competencies [16,17]. It is important that workshops or programs for residents focus on consensus-based resident teaching competencies. Despite residents’ critical role as teachers, no standardized teaching competencies have been established for them. Because teaching techniques are diverse, residents need to select appropriate teaching methods for their situations. Therefore, this study aimed to identify teaching competencies for residents as well as what is unique for them as near peer teachers.
Methods
Considering the aforementioned background, we conducted a modified Delphi study from January to March 2017 to identify teaching competencies for residents and to develop consensus on the most important competencies. We defined residents as trainee clinicians who have not yet worked as attending physicians, regardless of the clinical department.
The Delphi method is a group facilitation technique designed to transform opinions into group consensus. Consensus is obtained from the opinions of selected experts in certain fields through a series of structured questionnaires in an interactive process. It is a consensusbuilding method that has been widely used in social sciences and health-related research, especially in medical education [14].
We used the modified Delphi method for this study. Each panelist completed the survey online via Survey-Monkey (www.surveymonkey.com). All communication with panelists was conducted via anonymized emails and questionnaires to avoid bias. To increase validity, we conducted a literature review and made an initial list in the first round.
1. Panelists
The sampling method used in the Delphi method is not random sampling, but purposive sampling. In accordance with this method, we carefully sampled the “experts” who could answer our research questions. We sought consensus on resident teaching competencies from panelists engaged in training junior residents in Japan, including medical education specialists (researcher and degree holders), program directors, chief residents, and residents who had experience in clinical teaching. The reason for including the resident as a panelist is that related parties are often regarded as stakeholders in the Delphi process [14].
The number of panelists in the Delphi method is not clearly defined, but around 20 is recommended for consensus building in healthcare [18]. We emailed 26 candidates who matched these conditions and requested their participation. The candidates who opted to participate were asked for any further adaptations, and further participants were recruited. Our candidates were almost evenly distributed across five medical schools, four large-scale teaching hospitals, four small-scale teaching hospitals, and two teaching clinics. To gain a worldwide perspective, we invited candidates with varied experience in clinical teaching and learning. Although all of them were Japanese, seven of them had international experience in clinical teaching or learning.
This study was approved by the Ethics Committee of Hashimoto Municipal Hospital, Hashimoto, Japan (IRB approval no., H27-10-3). Written consent was obtained from all panelists.
2. Initial draft development
We started by generating a list of resident teaching competencies from a literature search and from previous studies, in which we explored the competencies of resident teachers. The two authors (T.H. and M.K.) independently searched previous studies or statements but found few articles related to resident teaching competencies. Therefore, we included articles on clinical teachers’ competencies that we assumed to be similar.
A literature search by the two authors (T.H. and M.K.) identified four articles and reviews [16,19-21], one book [22], and one guideline issued by the Japanese Ministry of Health, Labour and Welfare [23] regarding the competencies of clinical teachers. Ultimately, five works from Western countries and one from Japan were included. We discussed and agreed on prospective items to be included in the 45 items from our searches (Appendix 1).
Each item consisted of a description of the competency followed by a 7-point Likert scale (1=not at all important, 2=low importance, 3=slightly important, 4=neutral, 5=moderately important, 6=very important, 7=extremely important) indicating the strength of agreement with whether all residents should have the teaching competency.
3. Criteria for inclusion of items
Because there are no standard rules to determine when consensus is reached using the Delphi method, we had to decide on consensus criteria. Typically, an item that 70% of respondents agreed or strongly agreed on should be regarded as gaining consensus [14]. We referred to the criteria used in two previous studies that used a Delphi process to establish consensus on emergency medicine clerkship competencies and core teaching competencies for community preceptors [24,25]. The criteria were as follows: (1) Consensus for inclusion: more than 80% of panelists rated the items 7 (extremely important) or 6 (very important) without any comments that authors need to consider. (2) Reworded items and added to the next round: any items with such comments. (3) Exclude from the next round: items that the two authors (T.H. and M.K.) unanimously deemed inappropriate regardless of the consensus rate. After the first round, many items remained. Since we wanted to make the list more manageable for prospective participants, we added the following criterion to reduce the number of items. (4) Exclude from the next round: fewer than 80% of panelists rated an item 6 or 7 without such comments.
All answers were checked and compiled by the two authors (T.H. and M.K.).
4. Data collection and analysis
In the first round, we sent the web-based list to the panelists, asking them to rate each item on a 7-point scale, detect redundancies, propose additional items, or suggest changes in wording. For the first round only, panelists were provided with the opportunity to use their discretion to include additional competencies, so that panelists’ opinions could be incorporated into the initial list.
In the second and the third rounds, we generated new questionnaires based on responses to the first-round questionnaire. As with the first round, we sent emails to each member of the panel with a link of web-based list to rate items and add their opinions. In addition to the list, we attached a summary of the results from the previous round.
The researcher and the panelists repeatedly engaged in the iterative process of revising the list until no more items were excluded from the developed competency list. The survey was concluded at this point, and it was determined that the researchers had reached a consensus. This study was conducted from January to March 2017.
Results
1. Panelist members
Of the 26 invited candidates, a total of 24 participated as panelists (Table 1).
Distribution of the Expert Panel (n=24)
2. First round
Of the 24 panelists, 22 (92% response rate) returned fully completed questionnaires. The authors analyzed the data in accordance with the inclusion criteria (1) to (3). After this round, nine new items proposed by the panelists were added, and 10 of the 45 items were reworded. Six items with similar meanings were combined into three items. Sixteen items were remained as they were. Three items were removed at the authors’ discretion as the language was inappropriate. Nine items met the standard of consensus. In the end, 39 items remained in the second round. Descriptive statistics are presented in Appendix 1.
3. Second round
Of the 24 panelists, 20 (83% response rate) returned fully completed questionnaires. The authors analyzed the data in accordance with the inclusion criteria (1) to (4). After this round, five of the 39 items were reworded in response to suggestions from panelists. Panelists mentioned that one item had two meanings; therefore, it was divided into two items, and two other items that were similar in meaning were combined into one. Although one item was commented on by a panelist, but we left it as is because we did not think it needed to be reworded. As seventeen items fit the criterion for exclusion, we omitted them from the third round. Eleven items met the standard of consensus. Apart from that, the authors considered the items “can find learner’s strength” and “can understand the strengths and weaknesses of students and relate these to their learning” to have similar meanings. Therefore, only “can understand the strengths and weaknesses of students and relate these to their learning” was included. Therefore, in total, 12 items gained consensus. In the end, 10 items remained in the third round. Descriptive statistics are presented in Appendix 2.
4. Third round
Of the 24 panelists, 20 (response rate=83%) returned fully completed questionnaires. After this round, among the 10 items, five more items met the standard of consensus. A panelist suggested rewording one item. We decided to include this item with minor revisions. Therefore, in total, six items gained consensus. The remaining four items all fit the criterion for exclusion. Since all items met the inclusion or exclusion criteria, the authors determined that a consensus had been reached. Descriptive statistics are presented in Appendix 3.
5. Final list
Finally, after the three rounds, 27 items were generated as competencies of resident teachers (Fig. 1).
Summary of the Modified Delphi Process
6. Categorization
Medical educators play various roles, and the 12 roles of Harden and Crosby [1] are a common example. Our data suggest that resident teachers are also expected to play various roles. To visualize these roles and their corresponding competencies, we categorized them based on the 12 roles of Harden and Crosby [1] of medical teachers (Table 2). “Clinical or practical teachers,” “teaching role models,” “on-the-job role models,” “learning facilitators,” and “student assessors” were regarded as important competencies for resident teachers. The authors discussed and further divided them into classical teaching competencies and resident-specific teaching competencies.
Summary of Teaching Competencies for Resident Teachers according to Harden and Crosby’s 12 Roles
Discussion
This study aimed to identify teaching competencies for medical residents using a modified Delphi method. As a result, 27 teaching competencies were extracted; some of these were specific to resident teachers. Srinivasan et al. [15] presented teaching competencies according to position, revealing which kinds of teaching competencies are needed for medical educators at eight levels, from clinical teacher to education policy maker. The results illustrate that the contents and abilities for teaching could change with an individual’s position and that those in lower and higher positions are different rather than superior or inferior. Some activities are only possible for specific educators in specific positions at specific times. Our classification also provides similar evidence about residents. We speculate that these resident-specific tasks have to do with their proximity to learners, a characteristic that is unique to residents as near peer teachers.
Previous researchers have mentioned that educational proximity is a key characteristic of near peer teachers [26,27]. Ten Cate and Durning [26] identified 12 reasons that near peer teaching is beneficial. They demonstrated its importance using the term journeyman, which refers to “a tradesman or craftsman who has completed an apprenticeship. A typical guild structure shows three levels of expertise: apprentice, journeyman, and master [all of which are] phases that reflect growth toward independence” (p. 597). Journeymen are valuable because they are intermediaries “between ‘apprentice’ and ‘master,’ with both learning and teaching tasks” [26, p. 591]. Intermediate journeymen are closer to apprentices than masters and are hence closer, in terms of educational distance, to the apprentices they teach. Because of their proximity, they can see why apprentices cannot do a task and can, therefore, be approachable role models.
Additionally, we suggest that educational proximity can be divided into cognitive, social, and space-time proximity. It has been mentioned that many students may prefer to turn to peers rather than experts for assistance with conceptual problems because experts cannot visualize students’ reasons for experiencing difficulties with the subject matter [26]. In fact, Bulte et al. [28] found that learners consider near peer teachers to be cognitively closer to their own level than faculty teachers.
Residents are closer in cognitive educational distance to their learners. For example, among the 12 efficacies of Ten Cate and Durning [26] near peer teaching are the following: “to offer education to students on their own cognitive level” [26, p. 592], and “to socialize students in medical school and provide role models” [26, p. 595]. This is possible because of their close educational cognitive distance. Similarly, our findings suggest that teaching competencies, such as understanding the needs of students and providing guidance that is appropriate to student levels may be especially related to cognitive proximity. These competencies can enable residents to take on various information provider roles, such as conducting an educational lecture, providing effective feedback, guiding clinical reasoning, offering guidance for manual procedures, and offering one-to-one guidance more effectively.
Similarly, Lockspeiser et al. [27] mentioned social proximity in peer teaching. They explain that the proximity of social positions between the educator and learner, as well as cognition, leads to better relationships and interactions in learning. Among the 12 teacher roles of Ten Cate and Durning [26], “to offer students an alternative motivation as well as another method for studying” and “to enhance intrinsic motivation in students” [26, p. 595] correspond with this social proximity. Because of this proximity, residents can create a positive learning environment that is in line with some of our findings, by being able to “stimulate students’ motivation to learn” and “encourage questions to promote learning.”
Our study suggests that residents are also expected to act as role models. This can also be explained in terms of cognitive and social proximity. Rather than being a role model in its literal meaning of an individual whose attributes are emulated by someone in the interest of achieving an end goal [1], we believe that resident teachers can function as role models for students because they embody the next career goal for students, at close proximity. In a similar study that examined the role of student teachers, the importance of resident teachers was similarly emphasized in terms of educational distance [28].
Moreover, we regard space-time proximity as an important factor in near peer teaching. Interestingly, the panelists expected residents to be assessors. Space-time educational proximity is important for understanding this role. According to the pyramid by Miller [29], work assessment should be based on direct observation. The greater the chances of direct observation, the greater the ability to provide formative feedback. Direct observers may also notice physical or psychological changes in learners. Such familiarity may enable teachers to create safe learning environments. While also being sufficiently distant to maintain some objectivity, its proximity may contribute to some of the competencies such as “the time spent on and timing of guidance and coaching is appropriate,” “can recognize students who are at a high risk of burnout,” “can understand the strengths and weaknesses of students and relate these to their learning,” and “can notice when students are experiencing trouble.”
Nevertheless, based on the recent survey about RaT, it appears that the instruction on the evaluation has been infrequent [8]. Another researcher insisted that clinical teachers would not necessarily need competency in the specialized skill of evaluation [15].
In contrast with prior research, our study clearly revealed that even residents need the competency of assessing or evaluating. Competencies such as “can recognize students who are at a high risk of burnout” and “can notice when students are experiencing trouble” are especially important for residents who are closer to learners in space-time distance than busy attending doctors who cannot share time with medical students and younger residents. Burnout develops gradually in response to chronic stressors at work, and early warning signs are often missed [9]. Therefore, we strongly argue that near peers, who are closer to residents at high risk, can notice the signs of burnout in order not to miss them.
1. Limitations
This study was conducted by Japanese doctors. Because local culture influences local contexts, the Japanese context may influence the educational environment and teaching competencies [2]. To minimize this issue, the authors tried to invite panelists who have worked clinically or studied medical education outside of Japan. Seven panelists with international experience, including the United States and Europe, joined this study. We believe that our findings may reflect international perspectives. For example, the targeted skills and content areas reported by Al Achkar et al. [8] in their 2016 RaT survey of program directors across the United States are quite similar to our results. The only difference is in lecturing skills, which may be influenced by the Japanese cultural context, where lecturing is still one of the main teaching methods [30]. Further studies in other countries may be needed to determine whether teaching competencies are universal.
There is a growing momentum even in Japan to expect residents to teach younger residents and medical students in Japan [31]. Our findings would extremely contribute to the development of competency-based RaT programs in Japan. Furthermore, we expect that program developers of RaT programs will refer to our comprehensive competencies for residents all over the world, as this is the first report in the field of medical education. As one of the trials, our research team is currently running a 1-year RaT fellowship starting in 2018 based on the competencies. Although there are 27 teaching competencies, they do not need to be learned one by one. We designed our fellowship so that residents would acquire all 27 competencies upon completion of it. This way, we do not have to teach busy residents what they do not need. In addition to this, interviews with those participants and instructors are planned to refine the teaching competencies that residents should have.
Residents who developed those teaching competency will find joy in teaching, and while there is certainly the potential for increased workload, job satisfaction can be enhanced by successful teaching [3].
2. Conclusion
In conclusion, we identified 27 teaching competencies for residents using a modified Delphi method. The findings demonstrate the importance of proximity in residents as teachers. Moreover, we revealed that resident teaching competencies may need to include assessment. Based on this competency framework, we expect that RaT programs will be developed to improve resident teacher performance.
Acknowledgements
We thank the residents, the clinical teachers, and the educational experts who participated in this study as panelists. We also wish to acknowledge the support of Dr. Hiroshi Nishigori, Dr. Damon Sakai, and Dr. Richard Kasuya, who gave us much advice about this research.
Notes
Funding
This study was funded by grants from JSPS KAKENHI (grant no., 23H03129) and (grant no., 22K 21097).
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
Conceptualization: TH, MK; methodology: TH, MK; formal analysis: TH, MK; data curation: TH; writing–original draft: TH; writing–review & editing: MK; and project administration: TH.
