Abstract
-
Purpose
Continuing professional development is essential for improving health care services, especially in developing countries. Most of the training programs in Mongolia were not based on a targeted needs assessment. Therefore, we aimed to apply a mixed methods design to assess the training needs of Mongolian health professions.
-
Methods
We conducted a needs assessment using a convergent parallel mixed methods design in two steps. The survey and interview questions were developed to identify priority areas, targeted trainees, and effective training methods. A survey on 60 respondents, 15 individual interviews, and a focus group interview with 14 participants were conducted in the first step. In the second step, 12 representatives of key stakeholders were invited to a second focus group interview.
-
Results
Current health policy areas, areas related to future national plans, and areas not currently receiving governmental or international support were suggested as the main priorities. The stakeholder suggested that trainees should be selected based on their professional experience and language level, as well as each hospital’s needs. Building teams including various professions, such as nurses, technicians, and biomedical engineers, was recommended as a way to exchange ideas with each other and to build teamwork for future collaboration.
-
Conclusion
Medical training needs are dynamic and complex; therefore, a deep understanding of the context and setting is necessary. In this study, we assessed the targeted training needs of Mongolian health professions through a mixed methods design, which could be an effective way to conduct needs assessments for training programs.
-
Key Words: A mixed methods design, Needs assessment, Training program, Health professions
Introduction
Continuing professional development is essential for improving health outcomes and health care services. The linkage between the competency of health professions and the improvement of health outcomes is a major challenge in developing countries [
1]. Without active learning, health professions could not maintain their competencies in a sustained manner after graduation [
2]. Developing an effective program for continuing professional development is a four-step process that includes targeting the audience, identifying training needs, developing educational strategies, and determining the expected training outcomes. In order to identify training needs, it is important for health professions to reflect on their own knowledge and skills [
3]. Their thoughts about what situations they find challenging and feedback from colleagues could be useful for identifying training needs [
4].
Training programs that are effective at changing health professions’ behavior are based on well-prepared needs assessments [
5]. Needs assessment is a process that helps program developers to understand trainees and training environments [
6]. Training needs for targeted audiences can be investigated by surveys, focus groups, regional analyses, and evaluations of current programs. The results of a targeted needs assessment could be used by regulatory bodies to set standards and to establish requirements for training programs. It is also crucial for justifying the agenda and grant requests for the program [
7].
New strategies for needs assessment are needed [
2]. In an increasingly dynamic and complex world, quantitative and qualitative approaches should be combined in the form of a mixed methods design [
8]. The major advantage of using a mixed methods design is that doing so combines the strengths and non-overlapping weaknesses of both approaches [
9]. Even though a mixed methods design is more costly and time-consuming than using a single approach [
9], other researchers highlighted that it enables the assessment of multiple factors which are crucial for successful training implementation [
10]. Various ways exist to combine quantitative and qualitative approaches [
11,
12], including the use of a convergent parallel design where both methods are performed independently and the results are merged to extend the scope, breadth, and range of the components [
13].
In developing countries, building the capacity of health professions to meet local needs is challenging [
1]. Mongolia is a developing country located in East Asia, the population of which reached 3 million by the end of 2016 [
14]. Approximately 70% of the population live in the cities, and two-thirds of all medical doctors work in Ulaanbaatar, the capital city [
15]. Various international training programs have been implemented to improve health professions’ capacity in Mongolia. However, there are not any continuing professional development programs which are formally delivered by the academic societies or the Ministry of Health in Mongolia. A few international training programs provided by Korea and Japan used a survey for assessing the training needs of primary and secondary level hospitals. Both program developers organized the training of trainer workshops and several training programs for pediatric diseases, emergency care, ultrasonography, laboratory, X-ray, electrocardiography, and obstetric and gynecologic diseases areas. The program from Korea was implemented in the capital city and one province, while the program from Japan covered the two provinces in Mongolia [
16,
17]. There is lack of literature of the targeted needs assessment of tertiary level hospitals. Therefore, we aimed to apply a convergent parallel mixed methods design for targeting training needs and to develop a comprehensive training strategy for higher level training in Mongolia.
Methods
1. Setting and participants
Our research scope was to assess training needs of higher level health professions in Mongolia. Important stakeholders including doctors, professors, and high level officers from all the central hospitals and two government organizations which are responsible for post-graduate training were selected as targets for the study. We conducted the needs assessment in two steps. A convergent parallel mixed methods design was used for the first step of the needs assessment. A survey, individual interviews, and a focus group interview were conducted in parallel and analyzed separately. In the second step of the needs assessment, representatives of key stakeholders were invited to the second focus group interview. The aim was to develop a consensus framework based on the results of the first step. Some participants in the survey also participated in the interviews in both steps (
Fig. 1).
We administered the survey mainly to the medical doctors at tertiary level hospitals. However, we also included some hospital directors and government officers in the survey. Based on the health report of the Ministry of Health in Mongolia [
15] and studies in Mongolia and other developing countries [
16,
18], the survey focused on four points: the major needs of health professions, the major obstacles to health professions’ development, priority areas, and ways to target trainees for further training. Respondents chose two options out of five for major needs, three out of seven for major obstacles, five out of 14 for priority areas, and three out of eight for trainees to be targeted. At the end of each item, an answer option for “other” was added. Thus, the respondents were able to add their own responses (
Appendix 1).
The qualitative assessment of the first step included individual interviews and the first focus group interview. Individual interviews were conducted with medical doctors who had participated in previous fellowship training and stakeholders from three central hospitals and four specialized centers (cancer, maternal and child care, trauma and orthopedics, and infectious diseases). The first focus group interview was conducted with members of the two main organizations that are responsible for postgraduate training: The Center for Health Development under the Ministry of Health and the Mongolian National University of Medical Sciences under the Ministry of Education. The interview was focused on current health policies and international projects, the major needs and obstacles of health professions, the targeting of trainees, and the selection process of the fellowship training program (
Appendix 2). As the second step of the needs assessment, a second focus group interview was conducted with the 12 representative key stakeholders from the individual interviews and first focus group interview. The aim of the second focus group interview was to share the results of the first step and to develop a consensus about the final needs.
All quantitative data were analyzed by Windows Excel 2016.Ink (Microsoft Corp., Redmond, USA). To ensure reliability, second researcher analyzed the qualitative data as an inter-rater. In order to test the validity, research team constantly compared the data of both methods.
5. Ethics approval
This study was funded by the Korean Foundation for International Health. The Institutional Review Board of Seoul National University College of Medicine approved this study (C-1803-024-927). All stakeholders participating in this study provided informed consent and contributed of their free will.
Results
1. Survey results
A total of 60% of the respondents were women, and the plurality of respondents were 40–49 years old. In average seven specialized doctors from each hospital responded to the survey. Over one-third of the respondents (36.6%) had more than 20-years of work experience (
Table 1). Improving skills (24%) and the incentive system (23%) were identified as major needs, followed by improving the attitudes, human resources policy, and knowledge of health professions (
Table 2). Poor undergraduate medical education (21.5%), postgraduate education (19.4%), and continuing professional development training (15.3%) were identified as major obstacles to health professions’ development, followed by poor medical facilities and equipment (
Table 3). Pediatric diseases, cancer, cardiovascular diseases, surgery, and respiratory diseases were priority areas for future training (
Table 4). The question about trainees to be targeted was used to identify the primary candidates for future training. We divided doctors into three categories based on their work experience: those with less than 10 years of experience were classified as junior, those with 10–20 years of experience as mid-career, and those with more than 20 years of experience as senior. Junior doctors (30.7%), nurses (21.9%), and mid-career doctors (17.5%) were most commonly suggested to be targeted as trainees (
Table 5).
The interviewees emphasized cardiovascular diseases, cancer, research, emergency care, and pediatric as priority areas. They stated that team training for cardiovascular diseases is the current main direction of national health policy. Newly established centers, such as the center for orthopedics, the diagnostic center, and the university hospital were also mentioned. Participants recommended building and training new teams for the new centers and for areas that are lacking support from the government and international organizations. The other highlighted area of the interview was research. Improving research capacity is an important factor in health professions’ development. The interviewees mentioned that university faculty members and health professions at the hospitals could form research teams for further research development.
The interviewees suggested three main directions for selecting the trainees targeted for international training. First, team training was highly recommended. Building a team including members of various professions, such as nurses, technicians, and biomedical engineers, is important to promote collaborative practice. Second, the selection criteria should be based on trainees’ professional experience and language level. Some interviewees recommended including some members with a high level of English competency to help other members with communication. Third, the trainees’ selection process should be based on the hospital’s needs. A government organization should not recommend trainees whose background is not related to the hospital’s goals and needs.
3. Results of the focus group interview in the second step
During the second step of the needs assessment, we shared the results of the survey and the first focus group interview with the representative key stakeholders. The key stakeholders agreed that cardiovascular and pediatric diseases should be considered the most important areas. The stakeholders also mentioned that changes in infrastructure and national health policy should be considered as important priority areas. Some other important areas such as obstetrics and gynecology and infectious diseases were found to be already covered by governmental and international support. Therefore, the key stakeholders suggested basing future initiatives on unmet needs and avoiding overlap with other programs. The key stakeholders suggested targeting mid-career doctors as trainees. They pointed out that junior doctors would be advantageous as trainees because of their English level and active learning style. However, they could have some difficulties in recognizing crucial information in various learning situations. In contrast, mid-career doctors have more professional experience, meaning that they could more effectively learn in any situation. Thus, the key stakeholders highlighted that professional experience is the most important factor in effective training. Lastly, they agreed that training for members of other professions, such as nurses and technicians, is crucial for collaborative practice in health care.
Team training that would include members of various professions, such as nurses and technicians, was highly recommended. There was disagreement about whether teams should be formed within a hospital or organized across hospitals. The key stakeholders stated that it might be difficult to build a team across hospitals for some specialized areas, such as cancer and trauma. Nonetheless, they agreed that training teams at the same time would be more appropriate than dividing teams to be trained in different periods. In this way, the team members will be able to exchange ideas with each other and naturally build teamwork to support future collaboration. Six months was suggested as an appropriate period for both basic sciences and clinical training.
Discussion
Most of the studies which used a mixed methods design were conducted in the developed countries. There are lack of existing literature of training needs assessment in the developing countries. One study applied a mixed methods design for assessing the training needs of pediatric oncology nurses in some African and Asian countries [
19]. In our study, we applied a convergent parallel mixed methods design to assess the training needs of Mongolian health professions in the tertiary level hospitals. Our team developed survey questionnaires based on a review of the local literature. We found that one of the major needs of Mongolian health professions was to improve their skills. It could be supported by the fact that most of the previous international training programs were mainly based on lectures and discussions in order to improve the knowledge [
16,
17]. The results of the survey provided us a general overview of the priority areas and the trainees to be targeted. However, other researchers have stated that developing a survey based on a literature review [
6] and applying the categories created by researchers could be too general for specific contexts [
8]. According to the WHO, Mongolia has the highest burden of cardiovascular diseases and liver cancer which were also shown in our survey result. However, the survey respondents also chose pediatric diseases as one of the priority of topics which were declining in the past years [
14].
Qualitative approaches incorporate the categories identified by participants themselves, and are responsive to local situations [
8]. Thus, we conducted interviews in order to obtain a deeper and broader understanding of the situation. We found three main directions for priority areas and targeting trainees. Current health policy areas, areas relating to future national plans, and areas not covered by the government were suggested as the main priorities. For example, non-communicable diseases were already covered by the various training programs supported by the government and international aids. However, there was a lack of support to the areas which are covered by the newly established centers in the Ulaanbaatar city such as orthopedics center and diagnostic center [
16,
17]. The stakeholders suggested selecting trainees for international fellowship training based on their professional experience and language level, as well as each hospital’s needs. A high level of English language competency was also mentioned as a crucial factor for effective international training in a previous study [
20].
Other important issues were the team training and research areas which were highly recommended in the interview and not included in the literature review and survey results. Other researchers have reported that improving teamwork in medical teams is challenging, meaning that a systematic approach is needed for course development. They suggested including not only medical doctors, but also all types of health professions in team training [
21]. Inter-professional education improves patient care services and enhances the skills and experience of health professions [
22]. Several factors influence team training performance, such as the combination of team members and the scope of the organizations [
21]. During our focus group interview with stakeholders, there was disagreement on whether teams should be formed within a hospital or organized across hospitals. However, the participants agreed that the decision should depend on the training topic.
Even though, there were some different results from each method, most of the results from both methods were similar in our study (
Table 6). Various researchers stated that if the results from two approaches have similar conclusions, the results have greater credibility. Also, there are other studies emphasize the importance of utilizing the local stakeholders for the construct validity of the mixed methods [
23]. Our study included the representatives of the main government organizations, the main central and specialized hospitals, and the single national medical university in Mongolia. Therefore, our results could be used as the basic information on training needs for future programs and research.
Limitation of our study is that it was conducted to tertiary level hospitals in the capital city of Mongolia and the number of participants was relatively small. Thus, the results of this study cannot be interpreted as representing the training needs of all levels of the health delivery system in Mongolia. Further research should include health professions at both the regional and national levels in order to obtain wider information about training needs in Mongolia.
In conclusion, medical training needs are dynamic and complex, requiring a deep understanding of the context and setting. In this study, we assessed the targeted training needs of Mongolian health professions through a mixed methods design. The results could be used as a basis for implementing training programs in central and specialized hospitals. This study showed that a mixed methods design could be an effective way to conduct needs assessments for training programs.
ACKNOWLEDGMENTS
We would like to appreciate all the medical doctors and stakeholders who actively participated in the survey and the interviews.
ACKNOWLEDGMENTS
We would like to appreciate all the medical doctors and stakeholders who actively participated in the survey and the interviews.
Funding
This study was funded by the Korean Foundation for International Health. There is no role of the funding organization in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Author contributions
YHB and SJS developed the study design. YHB and NA collected and analyzed the data. All the authors had significant contribution in writing the paper. All the authors approved the final manuscript.
Fig. 1. Flow Chart of the Study Design
Table 1.Descriptive Statistics for the Respondents
Table 1.
|
Characteristic |
No (%) |
|
Gender |
|
|
Female |
40 (66.7) |
|
Male |
20 (33.3) |
|
Age (yr) |
|
|
≤29 |
9 (15.0) |
|
30–39 |
14 (23.3) |
|
40–49 |
26 (43.3) |
|
50–29 |
10 (16.7) |
|
≥60 |
1 (1.7) |
|
Work experience (yr) |
|
|
≤5 |
9 (15.0) |
|
6–10 |
10 (16.7) |
|
11–20 |
19 (31.7) |
|
≥21 |
22 (36.6) |
Table 2.Major Needs of Health Professions
Table 2.
|
Rank |
Major needs |
% of responses |
|
1 |
Improving skills |
24.0 |
|
2 |
Improving the incentive system |
23.0 |
|
3 |
Improving attitudes |
21.2 |
|
4 |
Improving human resources policies |
21.2 |
|
5 |
Improving knowledge |
9.7 |
|
6 |
Others |
0.9 |
Table 3.Major Obstacles to Health Professions’ Development
Table 3.
|
Rank |
Major needs |
% of responses |
|
1 |
Poor undergraduate education |
21.5 |
|
2 |
Poor postgraduate education |
19.4 |
|
3 |
Poor continuing professional development training |
15.3 |
|
4 |
Poor medical facilities |
15.3 |
|
5 |
Poor medical equipment |
15.3 |
|
6 |
Lack of incentives |
11.8 |
|
7 |
Lack of government support |
1.4 |
Table 4.Priority Areas for Future Training
Table 4.
|
Rank |
Priority areas |
% of responses |
|
1 |
Pediatric diseases |
15.1 |
|
2 |
Cancer |
11.9 |
|
3 |
Cardiovascular diseases |
11.5 |
|
4 |
Surgery |
7.9 |
|
5 |
Respiratory diseases |
7.3 |
|
6 |
Hospital management |
7.3 |
|
7 |
Nursing |
6.0 |
|
8 |
Emergency care and trauma |
5.5 |
|
9 |
Obstetrics and gynecology |
5.5 |
|
10 |
Infectious diseases |
5.0 |
|
11 |
Public health |
5.0 |
|
12 |
Diagnostic laboratory |
3.2 |
|
13 |
Medical education |
3.2 |
|
14 |
Medical research |
2.8 |
|
15 |
Others |
2.8 |
Table 5.Targeted Trainees for Future Training
Table 5.
|
Rank |
Targeted trainees |
% of responses |
|
1 |
Junior doctors |
30.7 |
|
2 |
Nurses |
21.9 |
|
3 |
Mid-career doctors |
17.5 |
|
4 |
Health administrators |
9.5 |
|
5 |
Technicians |
6.6 |
|
6 |
Senior doctors |
5.8 |
|
7 |
Professors |
4.4 |
|
8 |
Other |
2.2 |
|
9 |
Directors and heads |
1.4 |
Table 6.Comparison of the Results of Survey and Interviews
Table 6.
|
Survey |
Individual and focus group interviews |
|
Priority areas |
Pediatric diseases, cancer, and cardiovascular diseases |
- Emergency care and medical research |
|
- Focus on current health policies and not to overlap with other government and international training programs |
|
Targeted trainees |
Junior doctors, nurses, and mid-career doctors |
Mid-career doctors due to their professional experiences |
|
Effective training methods |
Not applicable |
Inter-professional team training |
References
- 1. Burdick WP, Morahan PS, Norcini JJ. Capacity building in medical education and health outcomes in developing countries: the missing link. Educ Health (Abingdon). 2007;20(3):65.
- 2. Norman GR, Shannon SI, Marrin ML. The need for needs assessment in continuing medical education. BMJ. 2004;328(7446):999-1001.
- 3. Davis D, Goldman J, Perrier L. Effective continuing professional development. In: Dent JA, Harden RM, eds. A Practical Guide for Medical Teachers. 2nd ed. Edinburgh, UK: Churchill Livingstone Elsevier; 2005: 38-48.
- 4. Sandars J. The use of reflection in medical education: AMEE guide no. 44. Med Teach. 2009;31(8):685-695.
- 5. Fox RD, Bennett NL. Learning and change: implications for continuing medical education. BMJ. 1998;316(7129):466-468.
- 6. Bass EB, Chen BY. Problem identification and general needs assessment. In: Thomas PA, Kern DE, Hughes MT, Chen BY, eds. Curriculum Development for Medical Education: A Six-Step Approach. 3rd ed. Baltimore, USA: John Hopkins University Press; 2016: 11-28.
- 7. Parry NM. The needs assessment in continuing medical education. Med Writ. 2014;23(2):125-128.
- 8. Cleland JA. Exploring versus measuring: considering the fundamental differences between qualitative and quantitative research. In: Cleland JA, Durning SJ, eds. Researching Medical Educatio. Oxford, UK: John Wiley & Sons; 2015: 3-15.
- 9. Johnson RB, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educ Res. 2004;33(7):14-26.
- 10. Chmiel AS, Shaha M, Schneider DK. Introduction of blended learning in a master program: developing an integrative mixed method evaluation framework. Nurse Educ Today. 2017;48:172-179.
- 11. Greene JC, Caracelli VJ, Graham WF. Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Policy Anal. 1989;11(3):255-274.
- 12. Bryman A. Integrating quantitative and qualitative research: how is it done? Qual Res. 2006;6(1):97-113.
- 13. Schoonenboom J, Johnson RB. How to construct a mixed methods research design. Kolner Z Soz Sozpsychol. 2017;69(Suppl 2):107-131.
- 14. World Health Organization. WHO country cooperation strategy 2017-2021: Mongolia. https://iris.wpro.who.int/bitstream/handle/10665.1/13684/WPRO-2017-DPM-005-eng.pdf. Published 2017. Accessed May 25, 2019
- 15. Centre for Health Development in Mongolia. Health indicators-2016. http://hdc.gov.mn/2018/sariin%20medee/2016-06.23-Health%20indicator.pdf. Published 2016. Accessed May 25, 2019
- 16. Yoon HB, Shin JS, Lee SH, et al. The effect of a clinical training program for capacity building of medical doctors in a developing country. Korean Med Educ Rev. 2015;17(2):78-87.
- 17. Centre for Health Development in Mongolia. Report on implementation of postgraduate training for primary and secondary health professionals. https://www.chd.mohs.mn/. Accessed May 25, 20192019.
- 18. Yoon HB, Shin JS, Lee SH, et al. Transnational collaboration for faculty development in health professions education in Mongolia. Korean J Med Educ. 2016;28(4):381-390.
- 19. Challinor JM, Hollis R, Freidank C, Verhoeven C. Educational needs and strategies of pediatric oncology nurses in low- and middle-income countries: an International Society of Pediatric Oncology Pediatric Oncology in Developing Countries Nursing Working Group Initiative. Cancer Nurs. 2014;37(4):E36-E47.
- 20. Kim DH, Yoon HB, Sung M, et al. Evaluation of an international faculty development program for developing countries in Asia: the Seoul Intensive Course for Medical Educators. BMC Med Educ. 2015;15:224.
- 21. Østergaard HT, Østergaard D, Lippert A. Implementation of team training in medical education in Denmark. Postgrad Med J. 2008;84(996):507-511.
- 22. Calman KC. Learning and teaching in medicine. In: Calman KC, ed. Medical Education: Past, Present, and Future. Edinburgh, UK: Churchill Livingstone Elsevier; 2006: 425-468.
- 23. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134-2156.
Appendices
Appendix 1.
- Survey Questionnaire
A. Survey questions
1. What are the major needs of health professions in Mongolia?
(Select 2 answers at most)
1) Improving the medical knowledge of health professions
2) Improving the medical skills of health professions
3) Improving the professionalism (attitude) of health professions
4) Improving human resource policies
5) Improving the incentive system
6) Others ( )
2. What are the major obstacles to strengthening the competencies of health professions? (Select 3 answers at most)
1) Poor basic medical (university) education
2) Poor postgraduate (resident) training
3) Poor continuing professional development training
4) Poor medical facilities
5) Poor medical equipment
6) Lack of incentives
7) Lack of support from the hospital or government
8) Others ( )
3. What should be the priority areas for future training programs?
(Select 5 answers at most)
1) Cardiovascular disease
2) Cancer
3) Infectious disease
4) Pulmonary diseases
5) Surgery
6) Obstetrics
7) Pediatrics
8) Traumatology (emergency medicine)
9) Diagnostic laboratory
10) Nursing
11) Hospital management
12) Public health
13) Medical education
14) Medical research
15) Others ( )
4. Who should be targeted as trainees for future international training programs?
(Select 3 answers at most)
1) Directors, heads
2) Senior doctors (more than 20 years of experience)
3) Mid-career doctors (10-20 years of experience)
4) Junior doctors (less than 10 years of experience)
5) Nurses
6) Technicians
7) Health administrators
8) Professors
9) Others (
Appendix 2.
- Interview Questions
1. What are the main healthcare policies of the Mongolian government?
2. Are there other international development projects for HRH?
3. What are the main healthcare objectives of the Mongolian government?
4. What are the major needs of health professions in Mongolia?
5. What are the major obstacles to strengthening the competencies of health professions?
6. What should be the priority areas for future training programs?
7. Who should be targeted as trainees for future training programs?
8. How could program providers select appropriate trainees for future training?
9. What shouldbe considered to maximize the impact of the future training program?
