Korean J Med Educ. 2007; 19(3): 185-195.
Published online 2007 September 30.
Item Analysis of Clinical Performance Examination Using Item Response Theory and Classical Test Theory
1Department of Medical Education, Korea University, College of Medicine, Seoul, Korea.
2Department of Medical Statistics, Korea University, College of Medicine, Seoul, Korea.
3Department of Software Engineering, Sungkonghoe University, Seoul, Korea.
ABSTRACT
PURPOSE: The objectives of this study were: 1) to analyze Clinical Performance Examination(CPX) items using item response theory(IRT) and classical test theory(CTT) and 2) to discuss how to apply and interpret these results in order to improve the quality of CPX items. In addition, we intended to explore statistical procedures in order to merge examination data from several different medical schools.
METHODS: The subject of the study was the 2005 CPX examination data from 10 medical schools located in Seoul and the Kyunggi province. For merging data from ten different medical schools, Levene's test for homogeneity of variances was used. Homogeneous group selection was conducted based on ANOVA or Kruskal-Wallis' test and Tukey's multiple comparisons appropriately. The generalized partial credit model was applied to analyze polytomous items and the 2-parameter logistic model was used to analyze dichotomous items.
RESULTS: Data from 8 medical schools were incorporated into the analysis. The result of the discrimination index by IRT was different from that of CTT in both polytomous and dichotomous items. Discrimination index from IRT tended to be lower than that of CTT. Difficulty index of dichotomous items of two models was correlated well with each other.
However, for polytomous items, IRT model provided more information than CCT.
CONCLUSION: We discovered that the CPX items were mostly easy in terms of difficulty index, and the result from IRT and CCT model did not correlated well in the discrimination index. IRT may provide more detailed information for polytomous items, but the checklist and criteria of scoring system should be cautiously reviewed.
Keywords :
Clinical performance examination;Item response theory;Classical test theory
