| Home | E-Submission | Sitemap | Contact us
top_img
Korean J Med Educ > Volume 28(2); 2016 > Article
Neyazi, Arab, Farzianpour, and Mahmoudi: Identifying weaknesses in undergraduate programs within the context input process product model framework in view of faculty and library staff in 2014

Abstract

Purpose:

Objective of this research is to find out weaknesses of undergraduate programs in terms of personnel and financial, organizational management and facilities in view of faculty and library staff, and determining factors that may facilitate program quality–improvement.

Methods:

This is a descriptive analytical survey research and from purpose aspect is an application evaluation study that undergraduate groups of selected faculties (Public Health, Nursing and Midwifery, Allied Medical Sciences and Rehabilitation) at Tehran University of Medical Sciences (TUMS) have been surveyed using context input process product model in 2014. Statistical population were consist of three subgroups including department head (n=10), faculty members (n=61), and library staff (n=10) with total population of 81 people. Data collected through three researcher-made questionnaires which were based on Likert scale. The data were then analyzed using descriptive and inferential statistics.

Results:

Results showed desirable and relatively desirable situation for factors in context, input, process, and product fields except for factors of administration and financial; and research and educational spaces and equipment which were in undesirable situation.

Conclusion:

Based on results, researcher highlighted weaknesses in the undergraduate programs of TUMS in terms of research and educational spaces and facilities, educational curriculum, administration and financial; and recommended some steps in terms of financial, organizational management and communication with graduates in order to improve the quality of this system.

Introduction

Fundamental part of good decision making is evaluation which its information will be used in developing a precise perception of outcomes of a program and input requirements like personnel and financial [1].
Evaluation of higher education increasingly is affecting academic quality, accountability of authorities, and universities ranking. Faculty members at universities are considered as effective and main factors in quality insurance of teaching–learning process in educational systems [1]. In many evaluation literatures, the CIPP model or Context, Input, Process, Product model is a very recommendable method for educational evaluation [2,3,4,5]. Complete definition for evaluation in the CIPP model is “Evaluation is a process of specifying, acquiring and providing descriptive information and making judgments about value and utility of goals, plans, performance and results to guide decision making, providing accountability, and greater understanding of phenomena under the study [6].”
From the CIPP model point, we can breakdown these eight evaluated factors into four sections:
(1) Context
  • - Goals, management and organization area

(2) Inputs
  • - Students

  • - Human resources

  • - Research and educational spaces and equipment/spaces and facilities

(3) Process
  • - Research and educational courses and programs, teaching and learning process

  • - Administration and financial

  • - Program evaluation

(4) Product (output)
  • - Graduates

This study aimed to examine faculty and library staff perspectives on undergraduate program quality and weak points within universities and factors that may facilitate program quality–improvement. Although undergraduate programs within Tehran University of Medical Sciences (TUMS) are the focus of this paper, one must also keep in mind the broader context of undergraduate programs and other school’s essential needs which are common among all universities and schools. Objective of this research is to find out weaknesses of undergraduate programs in terms of personnel and financial, organizational management and facilities in view of faculty and library staff, and determining factors that may improve its quality.

Subjects and methods

This is a descriptive analytical study conducted in four selected faculties at TUMS in 2014. We selected our sample only from Public Health School, Faculty of Nursing and Midwifery, Faculty of Rehabilitation and Faculty of Allied Medical Sciences that have undergraduate programs. Statistical population were consist of three sub-groups including department head (n=10), faculty (n=61), and library staff (n=10) with total population of 81 people. After receiving the permission from Ethics Committee of Tehran University of Medical Sciences with number (IR.TUMS.REC.1394.2156), sampling was done as a census among head of departments, faculty and library staff.
We used three separate researcher-made questionnaires inspired from the CIPP model and internal evaluation literatures to collect data, based on eight factors in four area of CIPP evaluation model, consist of demographic data such as place of work, educational degree, teaching status, scientific ranking and multiple choice questions that were used based on Likert scale rating 1 to 5 digits, grading from very low, low, medium, high, and very high. Thus, if the score of a factor became less than 49.9%, it was assumed undesirable; if the score was between 50% and 74.9%, it was considered as relatively desirable situation; and if the score was more than 75%, the factor had a desirable situation. The accuracy of the questions, topic, and research goals were validated by consulting expert, supervisor, and advisor. Cronbach α coefficient formula was used to determine department head, faculty and library staff questionnaire’s reliability which estimated at α=0.97, α=0.92, and α=0.87, respectively that suggests the questionnaires have had the required reliability.
The questionnaires were distributed and collected via face to face approach. The data were then analyzed using descriptive and inferential statistics including distribution of population, frequency, percentage, and Fisher exact test. Collected data was analyzed by SPSS version 20.0 software (IBM Corp., Armonk, USA).
In this study first, the context (goals, management and organization area) was evaluated and then, the education system indicators were determined, by using facilitate decision-making model of CIPP. These factors are including: input (students, human resources, educational and research spaces and equipment/spaces and facilities); process (educational courses and programs, teaching and learning process, program evaluation, and administration and financial); and product (graduates).

Results

The result showed that highest response rate among faculty members is from public health faculty with 38.3%. Also assistant professor is the most frequent ranking among faculty members (43.9%) and most of faculty members teach full time in their faculties. Among library staff, most of them (60%) have bachelor degree. Most frequent scientific ranking and degree among department head is associate professor with 44.4% and PhD with 88.9%, respectively.

1. Context

In this area, goals, management and organization factor was evaluated. Fifty percent of department head (Table 1) and 55.7% of faculty members believed that this factor has a relatively desirable situation (Table 2).

2. Input

In this area, three factors were evaluated. Fifty percent of department head (Table 1), 40% of library staff (Table 3), and 75.4% of faculty members (Table 2) stated that research and educational spaces and equipment/facility and spaces are in undesirable condition. Sixty percent of library staff believed that human resource is in desirable situation (Table 3) and 52.5% of faculty members specified that students are in relatively desirable situation (Table 2).

3. Process

In this area, also three factors were assessed. Seventy percent of department head expressed that educational courses and programs, learning and teaching process is in desirable situation (Table 1) and 52.5% of faculty members pronounced that this factor is in relatively desirable situation (Table 2). Seventy percent of department head believed that administration and financial is in undesirable condition and 40% of department head stated that program evaluation is in desirable condition (Table 1), but 45% of faculty members specified that this factor is in relatively desirable situation (Table 2).

4. Output

Fifty percent of department head expressed that graduates has a relatively desirable situation through these four faculties (Table 1).
The Fisher exact test showed statistically significant association between sampling location and research and educational spaces and equipment/spaces and facilities and between teaching status and research and educational spaces and equipment among faculty members that it means there are differences in these factors among selected faculties (Tables 4-9).

Discussion

Comparing our study and ones which have been conducted in past in same university highlights the significance of periodic evaluation of universities and reflection of its results to top authorities in order to identify gaps and improve universities quality accordingly.
In context area, goals, management, and organization factor was evaluated and has relatively desirable situation in view of department head and faculty members. Our findings is in opposition to results of Farzianpour et al.’s study [7] in Environmental Health and Engineering School of Public Health and Farzianpour et al.’s study [8] in Endodontic Department, Faculty of Dentistry at TUMS which showed desirable situation in management and organization. Findings of Farzianpour et al. [9] showed undesirable situation of goals and rather desirable situation of organizational and management structure in Epidemiology and Biostatistics Department in Public Health School at TUMS.
In input domain, our result showed that research and educational spaces and equipment/facilities and spaces factor was in undesirable situation which is against to result of studies in Biochemistry Department of Medicine Faculty, and in Healthcare Management Department of Allied Medical Sciences Faculty and in Epidemiology and Biostatistics Department in Public Health School at TUMS which showed relatively acceptable situation regarding this factor [9,10]. But findings of evaluation studies in Department of Environmental Health and Engineering, School of Public Health and in Endodontic Department, Faculty of Dentistry, TUMS, are similar to our results and showed undesirable situation in research and educational spaces and equipment [7,8].
Factor of human resources in input area is in desirable situation in view of library staff which is opposed to the results of Farzianpour et al. [10], which showed relatively acceptable situation in Biochemistry Department, Faculty of Medicine, TUMS. Farzianpour et al. [7] also showed relatively acceptable situation in manpower in Department of Environmental Health and Engineering School of Public Health, TUMS. Our finding is similar to results of Farzianpour et al. [8] in internal evaluation of Department of Endodontic, Faculty of Dentistry, TUMS.
In our study, students have relatively desirable situation which is aligned with findings of evaluation studies in Epidemiology and Biostatistics Department, Public Health School at TUMS and in Healthcare Management Department, Allied Medical Sciences Faculty [9]; but this result is in opposition of finding of evaluation studies in Department of Environmental Health and Engineering, School of Public Health and in Endodontic Department, Faculty of Dentistry at TUMS [7,8].
In process domain, in view of department head, educational courses and programs, teaching and learning process is in desirable situation, but in view of faculty members this factor is in relatively desirable situation that is aligned with the findings of Farzianpour et al. [11] in Healthcare Management Department, Allied Medical Faculty, TUMS and finding of Farzianpour et al.’s study [8] in internal evaluation of Endodontic Department, Faculty of Dentistry, TUMS. Results of Farzianpour et al. [9] showed rather desirable situation of educational courses and curriculum in Epidemiology and Biostatistics Department at TUMS.
Administration and financial is in undesirable situation.
Program evaluation is in desirable situation in view of department head and in relatively desirable situation in view of faculty member which is similar to findings of Farzianpour et al. [10], which showed relatively acceptable situation in Biochemistry Department, Faculty of Medicine, TUMS.
In output scope, graduates are in relatively desirable situation that is similar to findings of Farzianpour et al. [9] which showed rather desirable situation of graduates in Epidemiology and Biostatistics Department at TUMS. These similarities and contradiction between their findings and ours in some factors confirm the result of Fisher exact test which showed there are some significant differences between four faculties regarding to evaluated factors in view of faculty and library staff. This comparison also indicates that situation of some factors were desirable or relatively desirable in past but now have undesirable situation, it means that top authorities do not have had equal view to all faculties in terms of financial and providing facilities at TUMS or there has been some neglects during this time. Some similar findings also indicate that despite of undesirable situation in some faculties regarding to evaluated factors in past, they did not take them into consideration and did not act accordingly which affected quality and situation of those faculties in terms of education, research and facilities and they are still in undesirable situation in view of faculty and library staff in our study. These factors have been determined which may facilitate quality–improvement of program: development of a system for reflecting of student and faculty member’s criticism, satisfaction and views about department head; improvement of goals and mission of programs in dealing with graduates; development of a report on changes made during past 3 years by department head; development of documentary regulation in the department; allocating sufficient budget to increase and improve the quality of academic spaces for faculty members and students; establishing a system evaluation in library regarding to services, collection, facilities, quality and appropriateness of text book, internal and external scientific journals based on the field and availability of print and copy in the library; development of framework to estimation of needed human resources; establishing a systematic mechanism for allocation of facilities and services to faculty members in the department; increasing cooperation between universities regarding to research project, consultant services and executive activities. Besides, there should be more seminar and scientific conference for students and graduates and continuous assessment and evaluation for courses and programs; community needs, professional problems, proportional between theoretical and practical work should be considered in curriculum. Also there should be a system to record needed information about graduates, their skills, their satisfaction and their situation after graduation and for communication with them.
In this study, we faced with limitations such as lack of cooperation of faculty due to time constraint that prolonged the time to collect data. Besides, faculty wanted to modify the questionnaires according to their will.

Acknowledgments

The authors would like to appreciate all faculty members, library staff and all officials in public health, nursing and midwifery, rehabilitation and allied medical sciences faculties at Tehran University of Medical Sciences who participated in this study.

Notes

Funding
This research was supported by International Campus, Tehran University of Medical Sciences (2014).
Conflicts of interest
None.

Table 1.
Frequency Distribution of Units under Research among Department Dead according to Evaluated Factors
CIPP component Evaluated factor for head of department Evaluation spectrum
0-49.9 (low) 50-74.9 (middle) ≥75 (high)
Context Goals, management, and organization area 1 (10) 5 (50) 4 (40)
Input Facility and spaces 5 (50) 3 (30) 2 (20)
Process Educational courses and programs, learning and teaching process 0 3 (30) 7 (70)
Administration and financial 7 (70) 1 (10) 2 (20)
Program evaluation 3 (30) 3 (30) 4 (40)
Product Graduates 4 (40) 5 (50) 1 (10)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 2.
Frequency Distribution of Units under Research among Faculty Members according to Evaluated Factors
Component of CIPP Evaluated factors for faculty members Evaluation spectrum
0-49.9 (low) 50-74.9 (middle) ≥75 (high)
Context Goals, organization and management area 19 (31.1) 34 (55.7) 8 (13.1)
Input Learner or student 20 (32.8) 32 (52.5) 9 (14.8)
Spaces, research and education facility 46 (75.4) 13 (21.3) 2 (3.3)
Process Courses, research and educational programs, teaching and learning process 25 (41) 32 (52.5) 4 (6.6)
Evaluation 27 (45) 27 (45) 6 (10)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 3.
Frequency Distribution of Units under Research among Library Staff according to Evaluated Factors
CIPP component Evaluated factors for library staff Evaluation spectrum
0-49.9 (Low) 50-74.9 (Middle) ≥75 (High)
Input Human resources 1 (10) 3 (30) 6 (60)
Facility and spaces 4 (40) 2 (20) 4 (40)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 4.
Frequency Distribution of Units under Research according to CIPP Components and Sampling Location among Faculty Members
CROSS TAB
CIPP components Faculty
p Fisher exact test
Public Health Nursing and Midwifery Allied Medical Sciences Rehabilitation
Context Goals, organization and management area Low 6 (26.1) 6 (42.9) 4 (36.4) 3 (25) 0.285 7.15
Middle 16 (69.6) 7 (50) 4 (36.4) 6 (50)
High 1 (4.3) 1 (7.1) 3 (27.3) 3 (25)
Input Students Low 8 (34.8) 6 (42.9) 2 (18.2) 4 (33.3) 0.44 5.81
Middle 12 (52.2) 8 (57.1) 6 (54.5) 5 (41.7)
High 3 (13) 0 3 (27.3) 3 (25)
Research and educational spaces and equipment Low 22 (95.7) 10 (71.4) 9 (81.8) 4 (33.3) 0.001 16.84
Middle 1 (4.3) 4 (28.6) 2 (18.2) 6 (50)
High 0 0 0 2 (16.7)
Process Educational courses and programs, teaching and learning process Low 8 (34.8) 9 (64.3) 3 (27.3) 4 (33.3) 0.31 6.55
Middle 14 (60.9) 5 (35.7) 6 (54.5) 7 (58.3)
High 1 (4.3) 0 2 (18.2) 1 (8.3)
Program evaluation Low 11 (47.8) 8 (57.1) 3 (27.3) 5 (41.7) 0.34 6.49
Middle 11 (47.8) 6 (42.9) 5 (45.5) 5 (41.7)
High 1 (4.3) 0 3 (27.3) 2 (16.7)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 5.
Distribution of Units under Research according to CIPP Components and Scientific Ranking among Faculty Members
CROSS TAB
CIPP component Scientific ranking
p Fisher exact test
Assistant professor Associate professor Professor Lecturer
Context Goals, organization and management area Low 6 (24) 3 (33.3) 3 (33.3) 6 (42.9) 0.92 2.37
Middle 16 (64) 5 (55.6) 5 (55.6) 6 (42.9)
High 3 (12) 1 (11.1) 1 (11.1) 2 (14.3)
Input Student Low 6 (24) 4 (44.4) 2 (22.2) 6 (42.9) 0.73 3.84
Middle 13 (52) 4 (44.4) 6 (66.7) 7 (50)
High 6 (24) 1 (11.1) 1 (11.1) 1 (7.1)
Research and educational space and equipment Low 16 (64) 8 (88.9) 7 (77.8) 11 (78.6) 0.87 3.12
Middle 7 (28) 1 (11.1) 2 (22.2) 3 (21.4)
High 2 (8) 0 0 0
Process Educational courses and programs, teaching and learning process Low 7 (28) 3 (33.3) 3 (33.3) 9 (64.3) 0.21 7.44
Middle 17 (68) 6 (66.7) 5 (55.6) 4 (28.6)
High 1 (4) 0 1 (11.1) 1 (7.1)
Program evaluation Low 9 (36) 3 (33.3) 5 (55.6) 8 (57.1) 0.83 3.23
Middle 13 (52) 5 (55.6) 4 (44.4) 5 (35.7)
High 3 (12) 1 (11.1) 0 1 (7.1)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 6.
Distribution of Units under Research according to CIPP Components and Teaching Status among Faculty Members
CIPP component Teaching status
p Fisher exact test
Full time Part time
Context Goals, organization and management area Low 13 (28.3) 5 (35.7) 0.38 1.94
Middle 28 (60.9) 6 (42.9)
High 5 (10.9) 3 (21.4)
Input Students Low 14 (30.4) 6 (42.9) 0.40 2.12
Middle 26 (56.5) 5 (35.7)
High 6 (13) 3 (21.4)
Research and educational spaces and equipment Low 39 (84.8) 6 (42.9) 0.002 11.04
Middle 7 (15.2) 6 (42.9)
High 0 2 (14.3)
Process Educational courses and programs, teaching and learning process Low 19 (41.3) 6 (42.9) 1.00 0.284
Middle 24 (52.2) 7 (50)
High 3 (6.5) 1 (7.1)
Program evaluation Low 19 (42.2) 7 (50) 0.61 1.10
Middle 22 (48.9) 5 (35.7)
High 4 (8.9) 2 (14.3)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 7.
Distribution of Units under Research according to CIPP Component and Degree among Library Staff
CIPP component Degree
p Fisher exact test
Diploma Bachelor
Input Human resource Low 0 1 (14.3) 1.00 0.81
Middle 1 (33.3) 2 (28.6)
High 2 (66.7) 4 (57.1)
Facilities and spaces Low 1 (33.3) 3 (42.9) 1.00 0.92
Middle 1 (33.3) 1 (14.3)
High 1 (33.3) 3 (42.9)

Data are presented as frequency (%).

CIPP: Context input process product.

Table 8.
Distribution of Units under Research according to CIPP Components and Scientific Ranking among Department Head
CIPP component Scientific ranking
p Fisher exact test
Assistant professor Associate professor Professor
Context Goals, organization and management area Low 1 (100) 0 0 0.25 5.17
Middle 0 3 (60) 2 (50)
High 0 2 (40) 2 (50)
Research and educational spaces and equipment and human resources Low 1 (100) 3 (60) 1 (25) 0.85 2.97
Middle 0 1 (20) 2 (50)
High 0 1 (20) 1 (25)
Process Educational courses and programs, teaching and learning process Low 0 0 0 0.41 2.33
Middle 1 (100) 1 (20) 1 (25)
High 0 4 (80) 3 (75)
Administration and financial Low 1 (100) 3 (60) 3 (75) 1.00 2.80
Middle 0 1 (20) 0
High 0 1 (20) 1 (25)
Program evaluation Low 1 (100) 2 (40) 0 0.45 4.21
Middle 0 1 (20) 2 (50)
High 0 2 (40) 2 (50)
Product Graduates Low 1 (100) 2 (40) 1 (25) 0.85 3.78
Middle 0 2 (40) 3 (75)
High 0 1 (20) 0

Data are presented as frequency (%).

CIPP: Context input process product.

Table 9.
Distribution of Units under Research according to CIPP Components and Degree among Department Head
CIPP component Degree
p Fisher exact test
PhD Master
Context Goals, organization and management area Low 1 (100) 0 0.1 5.05
Middle 0 5 (55.6)
High 0 4 (44.4)
Research and educational spaces and equipment and human resources Low 1 (100) 4 (44.4) 1.00 1.42
Middle 0 3 (33.3)
High 0 2 (22.2)
Process Educational courses andprograms, teaching and learning process Low 0 0 3.00 0.52
Middle 1 (100) 2 (22.2)
High 0 7 (77.8)
Administration and financial Low 1 (100) 6 (66.7) 1.00 1.51
Middle 0 1 (11.1)
High 0 2 (22.2)
Program evaluation Low 1 (100) 2 (22.2) 0.60 2.26
Middle 0 3 (33.3)
High 0 4 (44.4)
Product Graduates Low 1 (100) 3 (33.3) 0.50 2.28
Middle 0 5 (55.6)
High 0 1 (11.1)

Data are presented as frequency (%).

CIPP: Context input process product.

References

1. Bastani P, Amini M, Nejad KT, Zadeh NS. Faculty members’ viewpoints about the present and the ideal teacher evaluation system in Tehran University of Medical Sciences. J Adv Med Educ Prof 2013; 1: 140-147.

2. Mohebbi N, Akhlaghi F, Yarmohammadian MH, Khoshgam M. Application of CIPP model for evaluating the medical records education course at master of science level at Iranian medical sciences universities. Procedia Soc Behav Sci 2011; 15: 3286-3290.
crossref
3. Mokhtarzadegan M, Amini M, Takmil F, Adamiat M, Sarveravan P. Inservice trainings for Shiraz University of Medical Sciences employees: effectiveness assessment by using the CIPP model. J Adv Med Educ Prof 2015; 3: 77-83.
pmid pmc
4. AbdiShahshahani M, Ehsanpour S, Yamani N, Kohan S, Hamidfar B. The evaluation of reproductive health PhD program in Iran: a CIPP model approach. Procedia Soc Behav Sci 2015; 197: 88-97.
crossref
5. Allahvirdiyani K. Evaluate implemented academic advisor of Shahed students in Tehran State Universities through CIPP evaluation model. Procedia Soc Behav Sci 2011; 15: 2996-2998.
crossref
6. Stufflebeam DL. The CIPP model for program evaluation. In: Madaus GF, Scriven MS, Stufflebeam DL, eds. Evaluation models. Dordrecht, The Netherlands: Kluwer Academic Publishing Group; 1983.

7. Farzianpour F, Hosseini S, Mirsepasi A, Honary H, Hosseini SS, Hosseini S. Internal evaluation of the Department of Environmental Health Science and Engineering. Am J Appl Sci 2013; 10: 58-63.
crossref
8. Farzianpour F, Zarabian M, Sharifian MR. Evaluation of management of Department of Endodontics, Faculty of Dentistry, Tehran University of Medical Sciences. Int Rev Manag Bus Res 2014; 3: 1394-1400.

9. Farzianpour F, Mohammad K, Malekafzali H, Nejat S. Determining the quality of management and structural elements of the Epidemiology and Biostatistics Department in Tehran University of Medical Sciences. J Serv Sci Manag 2014; 7: 430-439.
crossref
10. Farzianpour F, Ansari M, Hosseini S. Internal evaluation in Biochemistry Department, Medicine Faculty, Tehran University of Medical Sciences. Am J Agric Biol Sci 2014; 9: 517-522.
crossref
11. Farzianpour F, Dargahi H, Hosseini S, Hosseini SS, Hosseini S. Program evaluation of health care management: is it adjusted for students needs? Am J Econ Bus Adm 2011; 3: 420-424.
crossref
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-928-1647
E-mail : kjme_office@daum.net
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2017 by Korean Journal of Medical Education. All rights reserved.                 powerd by m2community