| Home | E-Submission | Sitemap | Contact us |  
top_img
Korean J Med Educ > Volume 30(3); 2018 > Article
Yang: Comparison of competency level of medical, non-medical students and its relevance for admission policy

Abstract

Purpose

This study aims to analyze the competency of medical students and its relevance for admission policy in medical schools.

Methods

This study examined the competency of 63 medical students from the 6-year program (group A) and 41 medical students from the 4-year program (group B) at Yonsei University using the Korea Collegiate Essential Skills Assessment (KCESA). The competency of groups A and B were compared to the corresponding competency levels of non-medical students (groups C and D). Group C is freshmen and D is senior students in universities. The KCESA is computer-based ability test composed of 228 items. The competency of participants were calculated on a T-scores (mean=50, standard deviation=10) based on KCESA norm-references. We conducted independent t-test for group comparisons of competency levels.

Results

There are no differences in competency levels between groups A and B. Compared with the non-medical students (group B), the medical students showed a significantly stronger ability to use resources, information-technology and higher-order thinking. In the comparison between groups B and D, medical students showed lower levels of self-management, interpersonal, and cooperative skills.

Conclusion

The cognitive ability serves as an important indicator for the decision on admission to a basic medical education program. The efforts should be made to foster the competency that medical students have been found to lack, such as self-management, interpersonal, and cooperative skills. The admission committee should assess the cognitive and non-cognitive competency of applicants in a balanced manner.

Introduction

Medical schools are increasingly giving attention to competency-based education (CBE) [1]. According to literature on curriculum development, CBE is a preferred educational model in medical education [2]. As regards the concept of competency, several researchers have presented different definitions [3-7]. A shared sense is that competency relates to the underlying characteristics of an individual that is causally linked to superior performance in a job [6]. In the medical sphere, CBE has gained increasing attention as emphasis is being placed on graduates’ practical competencies, and because the social responsibility of educational institutions has emerged as an important issue [8]. In the United States, the Accreditation Council for Graduate Medical Education initiated its core competency project in 1999 to identify competencies that residents should be equipped with during the course of graduate medical education. Similar efforts have been implemented in Canada through the CanMEDS 2005 Project, which aimed to establish a CBE model. Other examples of efforts to identify the required components of CBE include the Tuning Project 2000, funded by the European Commission, and Tomorrow’s Doctors, which was published in 2009 by Britain’s General Medical Council. These projects have repeatedly indicated that doctors’ competencies include communication skills, interpersonal skills, ability to collaborate or work in teams, critical thinking, information management, clinical skills, patient care, and self-directed learning.
One central issue in competency-based medical education is the selection of candidates most likely to possess these competencies. Typically, admission criteria serve to select applicants most likely to succeed in the respective undergraduate medical program and become upstanding members of the medical profession [9]. Based on several years of study, the Association for American Medical Colleges defined the following nine competencies that should be considered in admission policy: ethical responsibility to self and others, reliability and dependability, service orientation, social skills, capacity for improvement, resilience and adaptability, cultural competence, oral communication, and teamwork [10]. Thus far, the primary selection criteria used in most of these models are cognitive in nature, and include traditional academic measures, such as undergraduate grade point average (GPA) and medical college admission test scores [11]. Although past academic achievement can be effective variables for predicting academic progress through medical school [12], students’ academic excellence in cognitive domains does not necessarily translate to clinical excellence in the field [13,14]. Therefore, determining admission based on cognitive competencies does students a disservice, as potentially excellent candidates might be eliminated based on the assessment of arguably irrelevant competencies.
It takes more than academic ability to become a good doctor, and thus, selection criteria that assess important generic attributes need to be established [15]. Koenig et al. [16] noted that of the competencies necessary to be a doctor, prospective students’ internal traits should be screened prior to acceptance into medical school. In explaining the structure of doctors’ core competencies, Niven [17] pointed out that an individual’s underlying characteristics, such as beliefs, motivation, values, and personality, are difficult to foster through education. Kulatunga-Moruzi and Norman [11] also argued that medical students’ integrity, leadership ability, communication skills, work ethic, and orientation towards service should be seriously considered during the admission process because it is difficult to teach these skills and qualities. Although there are tools to assess these competencies, certain traits, such as integrity, belief, and values, are not easy to assess during the admission process. Continuing with this line of thought, Harden [18] asserted that institutions should not only assess if applicants possess the necessary competencies at the admission process but also try to assist students in cultivating these competencies during their study through curriculum design.
In South Korea, there has been significant debate regarding undergraduate medical education since 2004. The focus has been on changing the medical education system (from a 6-year medical education program after high school to a 4-year program after undergraduate studies) that cultivates medical professionals. As of 2016, the country had 41 medical education institutions, of which 16 offer the 4-year program type, 14 retain the 6-year program, and 11 provide both. In 2012, the accreditation standard required the identification of competencies medical students must have and the admission policy of the medical school that takes such competencies into account [19]. This change in accreditation standard has driven the development of CBE curriculum and influenced dramatically the admission policy of medical schools. Attention is now given to competency and subjective factors, such as personality traits and behaviours needed be a ‘good doctor’ in the admission policy. Although there are considerable efforts in South Korea to define the required competencies for medical students, no national consensus has been reached on the specific competencies that applicants should have and that should be evaluated in the admission process. Moreover, no objective assessment method has been developed to screen for and evaluate the said competencies. Consequently, the students selected by medical schools tend to be those who obtain high scores on the competencies that can be measured objectively. Powis [20] mentioned that an expert’s subjective judgment on non-cognitive qualities does not necessarily imply a decline in objectivity. Nevertheless, the Ministry of Education of Korea [21] declared that all medical schools should evaluate applicants’ competency based on quantitative data to guarantee objectivity in admission. This rule implies that medical schools have limited autonomy in evaluating applicants in terms of whether they have the personality traits or characteristics required to be a good doctor.
The 6- and 4-year medical education programs follow different admission policies. In the former, new students are selected for admission based on their high school transcript and Korea Scholastic Aptitude Test score (KSAT, similar to the scholastic aptitude test used in the United States), which are purely objective materials. Students are required to submit recommendation letters and take an interview, but these are deemed less important variables than academic performance indicators. Meanwhile, students in the 4-year program are selected based on their university GPA, medical education eligibility test score (similar to the medical college admission test used in the United States), recommendation letters, personal statement, and interview. In this admission process, evaluators consider objective as well as subjective materials, such as letters and interview, to predict the likelihood of a student to succeed in basic medical education and become a good doctor.
The present study aims to identify the characteristics of competency of medical students in Korea and investigate its relevance for admission policy. Thus, this work compares the competency of first-year medical students in both the 6- and 4-year programs, as students in these two tracks are accepted using different admissions processes and materials. The research also analyses the differences in the competencies possessed by medical and non-medical students. Based the results, the nature and types of competencies possessed by medical students and its relevance for admission policy are discussed.

Methods

Study participants included 63 students (95.5% of 66) from the 6-year medical education program (group A) and 41 students (74.5% of 55) from the 4-year medical education program (group B) accepted at Yonsei University, which provides both program types. There are 83 male students (group A, n=50; group B, n=33) and 21 female students (group A, n=13; group B, n=8). Group A is composed of first-year medical students who began their program after high school; the program consists of 2 years of premedical education and 4 years of basic medical education. They are equivalent to university freshmen. Group B is composed of first-year medical students who had completed a bachelor’s degree in university; the program consists of 4 years of basic medical education. The competency of the two groups were also compared to the corresponding competency levels of non-medical students in universities nationwide (group C, 1,000; group D, 1,000), majoring in liberal arts and social science, natural science, engineering, and others. Group C is composed of university freshmen comparable to group A. Group D members are senior students in a university comparable to group B. The admission policy and process of groups C and D are similar to those of the 6-year medical education program.
Participants’ competencies of groups A and B were evaluated using the Korea Collegiate Essential Skills Assessment (KCESA, equivalent to the Graduate Skills Assessment in Australia and Collegiate Learning Assessment or Measure of Academic Proficiency and Progress in the United States). To compare the competency between medical and non-medical students, which may inform the relationship between competency and admission policy, it is desirable to use the KCESA inventory developed in the context of Korea. KCESA was jointly developed by the Korea Ministry of Education and the Korea Research Institute for Vocational Education and Training (KRIVET). KCESA is a standardized inventory that assesses college students’ core competencies. This inventory was developed by the Delphi process to measure the level of knowledge, skills, and attitudes commonly required in various job markets in fiscal years 2006–2009. KCESA’s criterion validity was established through consistency with other assessment tasks, whereas construct validity was determined through confirmatory factor analysis of 813 college students. After then, a total of 48,000 college students participated in the KCESA test by 2011, and over 20,000 students participate in the test every year. This inventory is a standardized tool developed by assuming 4-year college students in Korea [22].
As presented in Table 1, the KCESA is designed to assess six core competencies. The KCESA is a computerbased ability test composed of 228 items including of 110 multiple-choice questions, eight essay items, and 110 items rated on a 5-point Likert scale. All participants (groups A and B) completed the test in a computer lab in Yonsei University, and T-scores (mean scores=50, standard deviation=10) of individuals were calculated based on the mean and standard deviation of the each group. The data for groups C and D is norm-referenced score given by the KRIVET. The score is calculated by sampling 1,000 students in each grade of university considering gender and university location. Therefore, it can be said that the higher the average score of the individual or group is, the higher the competency level is. We conducted independent t-test for group comparison of competency levels. The p-value for the statistical test set to <0.008 (=0.05/6, Bonferroni correction) considering six components of the KCESA. The assumption of homogeneity of variances among groups was met. Data were statistically analyzed using PASW SPSS ver. 18.0 (SPSS Inc., Chicago, USA). Ethical approval was obtained from the Severance Hospital Institutional Review Board (4-2011-0006). Informed consent was obtained from the participants according to the guideline and form of the Institutional Review Board.

Results

1. Competency levels between groups A and B

Table 2 presents the core competency levels of the medical students. No significant differences in core competency levels were observed between groups A and B. As regards the six core competencies, both groups obtained above-average scores of t-distribution on items pertaining to the ability to use resources, information, and technology; communication skills; and higher-order thinking. In contrast, group B obtained below-average t-scores on items pertaining to self-management and interpersonal and cooperative skills. In terms of global competency, group A obtained above-average scores of t-distribution, whereas group B obtained below-average scores. Nonetheless, both groups are regarded as being within the normal range on that dimension because t-scores are within one standard deviation above or below the mean reference value.

2. Comparison of specific competencies between medical and non-medical students

Table 3 presents comparisons of the competency levels of medical and non-medical students (groups A and C, B and D). Compared with the non-medical students (groups A and B), the medical students showed a significantly stronger ability to use resources (e.g., information and technology) and higher-order thinking ability. Meanwhile, no significant between-group differences in global skills and communication were observed. In the comparison between groups B and D, medical students showed lower levels of self-management, interpersonal, and cooperative skills compared with the fourth-year students of group D.

Discussion

This study conducted an analysis of the competency levels of first-year medical students enrolled in the two types of medical school program. No significant difference in general competency level was observed between the two groups (groups A and B); differences were observed in some competencies between first-year medical and non-medical students. It is worth noting that the first-year medical students (group A) accepted into the basic medical education program in Korea are likely to possess such competencies as higher-order thinking and ability to use resources, information, and technology compared with non-medical students (group C) in university. The medical students of group B, however, showed lower self-management, interpersonal, and cooperative skills compared with fourth-year non-medical students (group D). The results of analyzing the data of 28,399 students who participated in KCESA in 2010 are the same as those of this study [22].
A few factors underlie these tendencies. First, students’ cognitive ability is given more importance in the selection of applicants for 6-year basic medical education program after high school. A study on KCESA pointed out that students’ competence are related to the entrance screening system [23]. There is a possibility that the difference in competence among groups may be caused by the selection effect [22]. In admission processes in Korea, many assessors consider objective variables, such as test scores (e.g., KSAT), high school transcript, and GPA, as the best measurement of students’ ability to use resources (including information and technology) and higher-order thinking. As mentioned, past academic achievement can be effective as a variable to predict academic progress through medical school [12]. The results of this study confirm that cognitive ability serves as an important indicator for the decision on admission to basic medical education program. Second, in this study, the students who had been accepted into 4-year basic medical education program after a bachelor degree had lower levels of selfmanagement, interpersonal, and cooperative skills compared with non-medical students. This may be attributed to Korea’s educational environment. Kim [24] reported that Korean students’ learning experiences take place under the supervision of their parents or in private educational institutions (hakwon) rather than on their own. Park [25] also noted that Korean medical students tend to view learning in terms of competing with others and believe that studying is an activity that should be done alone. In such a competitive learning environment where academic rank (established based on a student’s performance compared with others) is perceived as important, there is a perception that interpersonal and cooperative skills are unimportant.
Analysis of the competency of students admitted to medical schools provides meaningful implications for developing admission policy and improving competency through medical curricula. First, the specific competencies medical students should possess at the time of admission should be clarified. Subsequently, admission policies based on these competency requirements should be established. Second, efforts should be made to foster the competencies that medical students have been found to lack, such as self-management, interpersonal, and cooperative skills. Therefore, fostering these competencies, including the ability to connect emotionally with others, work with diverse individuals, work effectively in a team, and act as a mediator, as well as developing leadership skills and systematic thinking, should be regarded as essential goals in basic medical education. Third, the non-cognitive competencies of applicants should be recognized as important admission criteria because they are difficult to cultivate during basic medical education, which means that the admission committee is given no room to assess non-cognitive competencies because of the regulation of the Ministry of Education. A potential approach is to allow admission committees autonomy to evaluate non-cognitive competency.
A limitation of this study is that it could not explore the correlation between first-year medical students’ competency and assessment scores in the admission process directly owing to lack of access to admission data. Such data would be helpful to assess whether differences in admission scores are statistically significant. The admission process of medical schools and measurement of medical students’ competency have largely remained unchanged since 2011, and critics have noted that current research data may be outdated. In a follow-up study, it is necessary to conduct a follow-up study to compare the scores of the KCESA according to students majors, and to track the cause of the differences among the groups. The limitations of the KCESA inventory are as follows. The KCESA does not help to identify the strengths and weaknesses of core competencies based on student background variables. In addition, the KCESA itself does not provide the information necessary for the improvement of education [26]. The KCESA inventory is based on quantitative assessment and tends to simplify and reduce the complexity of college students’ core competencies [27]. Also, there is a limit to diagnose competencies with non-intellectual characteristics because of focusing on specific areas and contents of cognitive domains [28]. This study contributes insights and data for the design of basic medical education curriculum aimed at addressing the deficiencies of first-year medical students in competencies needed not only to survive medical school but also succeed in professional practice, which may be explored in future studies.

Acknowledgments

None.

Notes

Funding
This study was supported by a faculty research grant of Yonsei University College of Medicine for 2012 (6-2012-0022).
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
Authors’ contribution
EBY designed there search questions and administered the survey instruments. EBY also recruited the participants, analyzed the data, and wrote the manuscript. EBY designed there search questions and administered the survey instruments. EBY also recruited the participants, analyzed the data, and wrote the manuscript.

Table 1.
Composition of the Korea Collegiate Essential Skills Assessment Inventory
Competency Definition No. of items Time (min) Type
Resources-information-technology Competency involving collecting, analyzing, and using resources, information, and technology 30 45 Multiple-choice question
Global skills International sensibility and an international worldview 48 30 Multiple-choice question
Communication skills Competency involving understanding, creating, and transmitting a message appropriate for the situation, using communication skills and knowledge 32 80 Multiple-choice question
Higher-order thinking Competency involving solving a problem by diagnosing the situation and using logical reasoning and problem solving to analyze clearly the problem situation through higher-order thinking skills, and offering a valid solution from among various ideas 8 90 Essay
Self-management Self-managing competency involving motivation, cognitive strategies, stress management, planning ability, and profession’s ethic, etc. 60 Unlimited Five-point Likert scale
Interpersonal and cooperative Competency involving cooperating with others and acting appropriately to achieve goals in social situations and maintaining positive relations with a variety of people 50 Unlimited Five-point Likert scale
Table 2.
Comparison of Competency T-Scores between Groups A and B
Competency Group No. of students Mean±SD t-value* p-value
Resources-information-technology A 62 55.51±5.51 0.23 0.8185
B 40 56.11±6.11
Global skills A 63 50.92±9.04 -0.62 0.5358
B 40 49.74±9.74
Communication skills A 34 50.55±0.55 1.11 0.2737
B 16 54.41±4.41
Higher-order thinking A 50 53.93±3.93 0.66 0.5140
B 31 55.39±5.39
Self-management A 63 47.51±11.02 -0.02 0.9804
B 40 47.46±7.46
Interpersonal and cooperative A 62 48.04±8.04 -0.29 0.7712
B 40 47.43±7.43

Group A: 6-year basic medical education program after high school. Group B: 4-year basic medical education program after a bachelor’s degree.

SD: Standard deviation.

* p<0.008 (=0.05/6, Bonferroni correction: significance level 0.05 is divided by six competencies of the Korea Collegiate Essential Skills Assessment).

Table 3.
Comparison of Competencies between Medical and Non-Medical Students
Competency Group (A) Group (C) t-value p-value Group (B) Group (D) t-value p-value
Resources, information-technology 55.50 48.40 5.595* 0.000 56.10 51.40 2.023 0.050
Global skills 51.03 48.50 1.864 0.063 49.85 51.50 -1.118 0.264
Communication skills 50.62 48.20 1.569 0.121 54.44 51.60 1.564 0.118
Higher-order thinking 54.04 47.50 5.186* 0.000 55.55 51.90 2.154 0.032
Self-management 47.46 47.00 0.372 0.710 47.15 52.20 -3.24* 0.001
Interpersonal and cooperative 48.06 47.60 0.272 0.786 47.37 51.90 -2.592* 0.003

Group A: 6-year basic medical education program after high school. Group B: 4-year basic medical education program after a bachelor degree. Group C: first-year non-medical students from other majors. Group D: fourth-year non-medical students from other majors.

* p<0.008 (=0.05/6, Bonferroni correction: significance level 0.05 is divided by six competencies of the Korea Collegiate Essential Skills Assessment).

References

1. Gruppen LD, Mangrulkar RS, Kolars JC. The promise of competency-based education in the health professions for improving global health. Hum Resour Health 2012; 10(1):43.
crossref pmid pmc pdf
2. Rubin P, Franchi-Christopher D. New edition of Tomorrow’s Doctors. Med Teach 2002; 24(4):368-369.
crossref pmid
3. McClelland DC. Testing for competence rather than for “intelligence”. Am Psychol 1973; 28(1):1-14.
crossref pmid pdf
4. Jacobs R. Getting the measure of management competence. Pers Manag 1989; 21(6):32-37.

5. Dubois DD. Competency-based performance improvement: a strategy for organizational change. Amherst, USA: HRD Press; 1993.

6. Spencer LM, Spencer SM. Competence at work: models for superior performance. New York, USA: John Wiley & Sons; 1993.

7. Epstein RM. Assessment in medical education. N Engl J Med 2007; 356(4):387-396.
crossref pmid
8. Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach 2010; 32(8):631-637.
crossref pmid
9. Wilkinson D, Zhang J, Byrne GJ, et al. Medical school selection criteria and the prediction of academic performance. Med J Aust 2008; 188(6):349-354.
pmid
10. Nowacek GA, Bailey BA, Sturgill BC. Influence of the interview on the evaluation of applicants to medical school. Acad Med 1996; 71(10):1093-1095.
crossref pmid
11. Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med 2002; 14(1):34-42.
crossref pmid
12. McManus IC, Woolf K, Dacre J, Paice E, Dewberry C. The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Med 2013; 11: 242.
crossref pmid pmc pdf
13. Mitchell KJ. Traditional predictors of performance in medical school. Acad Med 1990; 65(3):149-158.
crossref pmid
14. Bore M, Munro D, Powis D. A comprehensive model for the selection of medical students. Med Teach 2009; 31(12):1066-1072.
crossref pmid
15. Lambe P, Bristow D. What are the most important non-academic attributes of good doctors?: a Delphi survey of clinicians. Med Teach 2010; 32(8):e347. e354.
crossref pmid
16. Koenig TW, Parrish SK, Terregino CA, Williams JP, Dunleavy DM, Volsch JM. Core personal competencies important to entering students’ success in medical school: what are they and how could they be assessed early in the admission process? Acad Med 2013; 88(5):603-613.
crossref pmid
17. Niven PR. Balanced scorecard step-by-step: maximizing performance and maintaining results. 2nd ed. Hoboken, USA: John Wiley & Sons; 2006.

18. Harden RM. Outcome-based education: the ostrich, the peacock and the beaver. Med Teach 2007; 29(7):666-671.
crossref pmid
19. Korean Institute of Medical Education and Evaluation. Medical college accreditation standards. Seoul, Korea: Korean Institute of Medical Education and Evaluation; 2012.

20. Powis D. Selecting medical students: an unresolved challenge. Med Teach 2015; 37(3):252-260.
crossref pmid
21. Ministry of Education. Plan for medical and dental school transfer in 2017 academic year. http://www.moe.go.kr/boardCnts/view.do?boardID&#x0003d;294&lev&#x0003d;0&statusYN&#x0003d;W&s&#x0003d;moe&m&#x0003d;0503&opType&#x0003d;N&boardSeq&#x0003d;62435. Published 2016. Accessed May 10. 2016.

22. Jin MS, Sohn YM, Chu HJ. A study on development plan of K-CESA for college education assessment. J Educ Adm 2011; 29(4):461-486.

23. Lee JI. A study on university students' essential skills and university admission methods. J Agric Educ Hum Resour Dev 2012; 44(2):73-96.

24. Kim E. Characteristics of the current student generation and considerations for medical education. Korean Med Educ Rev 2012; 14(2):59-63.

25. Park J. Medical students’ general beliefs about their learning. Korean Med Educ Rev 2012; 14(2):64-68.

26. Lichtenberg JW, Portnoy SM, Bebeau MJ, et al. Challenges to the assessment of competence and competencies. Prof Psychol Res Pract 2007; 38(5):474-478.
crossref
27. Koeppen K, Hartig J, Klieme E, Leutner D. Current issues in competence modeling and assessment. J Psychol 2008; 216(2):61-73.
crossref
28. Lee AH, Choi MS. Analysis of the research trend on the college students’ key competencies and diagnosis tools. J Educ Technol 2014; 30(4):561-588.
crossref
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-928-1647
E-mail : kjme_office@daum.net
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2018 by Korean Journal of Medical Education. All rights reserved.                 powerd by m2community