Verbal communication of students with high patient–physician interaction scores in a clinical performance examination assessed by standardized patients
Article information
Abstract
Purpose
Standardized patients (SPs) tend to rate medical students’ communication skills subjectively and comprehensively, in contrast to such objective skill set defined in the clinical performance examination (CPX). Meanwhile, medical school instructors have a different approach in their evaluation of students’ communication skills. We aim to analyze medical students’ verbal communication skills using objective methods, and to determine the contributing factors of a patient–physician interaction (PPI) score.
Methods
Students with high- and low-ranking scores for PPI in CPX were selected. The Roter interaction analysis system was used to compare verbal communication behaviors of the students and SPs. Patient-centeredness scores (PCSs), physician’s verbal dominance, and number of utterances were compared between the two groups.
Results
PCSs and physician’s verbal dominance had no difference between the groups. The number of utterances during the limited time of 5 minutes of CPX was higher for the high-ranking students. They tended to employ more paraphrase/check for understanding, and closed questions for psychosocial state and open questions for medical condition. The SPs interviewed by high-ranking students gave more medical information and requested for more services.
Conclusion
In the case of the routine checkup, smooth conversations with more frequent utterances were detected in the high-ranking students. More medical information exchange and requests for services by SPs were higher for the high-ranking students. Medical communication instructors should keep in mind that our results could be indicators of a high PPI score.
Introduction
Communication skills have been made essential in medical education curricula because it is fundamental to facilitate rapport building and the efficient exchange of information during consultation [1]. A good patient–doctor relationship has an effect on optimal clinical outcomes [2].
Medical communication skills are evaluated as part of clinical performance, especially in the patient–physician interaction (PPI) section in the clinical performance examination (CPX). Standardized patients (SPs) evaluate the medical students based on their observations as soon as the student leaves the exam room. The professors and SPs showed no difference in their assessment of students’ attitudes, empathy and communication skills in CPX [3,4]. However, another study indicated that assessments may be vulnerable to errors, such as when SPs at the same CPX station have different physical presentations and facial expressions [5]. Consequently, the accuracy and reliability of SPs’ marks on the CPX checklist have been questioned for years. For items with clear evaluation criteria, such as history taking, SPs and professors tend to give mostly similar marks, but for those involving subjective judgment, such as PPI, they give significantly different scores [6]. Professors evaluate PPI as a direct or indirect observer, whereas SPs evaluate PPI after they have experienced consulting with a medical student. McLaughlin et al. [7] reported that SPs gave higher scores to students than physicians, and found a weak correlation between the SPs ’ and physicians’ scores. In addition, CPX scores by SPs were not correlated with multiple choice questions examination scores, unlike physician scores. Therefore, SPs may have better evaluated communication skill, whereas physicians have better evaluated problem-solving and clinical skills [7].
Well-trained and experienced SPs can evaluate PPI, and they can give feedback to medical students as replacement for professors. However, they are not the ones who teach or train medical students in the aspects of medical communication skill and clinical competency. It is the medical college professor, generally physicians who teach communication skills to medical students because SPs do not have knowledge of medical competency. The professors’ knowledge and understanding of the PPI may differ than those of the SPs who experience the PPI directly. Therefore, professors and SPs might have different standards for evaluation.
As mentioned, SPs tend to assess students subjectively and comprehensively in communication skills in CPX, whereas PPI is needed to facilitate analysis using objective and clear methods. The linguistic dialog analysis approach has been used to analyze PPI in CPX. The SPs and authors had differences in terms of PPI evaluation, but linguistic dialog analysis approach is considered a subjective tool [8]. In the present study, the verbal communication behaviors of medical students and SPs were analyzed and compared between a high- and a low-ranked group based on the PPI score. Analyses of medical encounters using objective tools can provide detailed and structured tips to students with a lower PPI score. Further, even high-ranking students may learn areas for improvement with respect to better communication.
Methods
1. Setting and samples
The course of doctor competency development runs for 4 consecutive weeks for first-year medical students at Inje University as of March 2015. It consists of modules for basic communication skills, basic physical examination, introduction to problem-based learning, self-development of competence, and clinical reasoning. The basic communication skills course consists of six topics: medical interview experience, agenda setting, attentive listening, empathy, medical interview with pediatric patients, and patient education. Each topic is covered in 4-hour class: (1) 1-hour lecture, (2) 2-hour exercise with SPs and formative feedback, and (3) 1-hour discussion and summary with the instructor. During the 2-hour exercise with SPs, six to seven students meet one SP. Each student takes turns in the practice medical interview with an SP for 5 minutes. Feedback by the SP and peer students is given to the student immediately after the interview.
Three stations of CPX are integrated into the course as summative assessment at the end of the course: physical examination for the abdomen, physical examination for the heart, and interview for routine checkup. Each student has three CPX stations. Three copies of three CPX were operating at the same time. Built-in video camera equipment and portable microphones were used for recording all three routine checkup CPX rooms. These recordings were originally made available for the remediation of students who fail to pass the communication CPX. Each student had only one video recording for the routine checkup CPX. As routine checkup is the only one to assess communication skills, this station was selected for the present study. Video recordings were used for our study after approval by the Institutional Review Board of Inje University Busan Paik Hospital (IRB number: 2015-08-0145).
The case was developed by the medical communication committee at the Inje University College of Medicine. As first-year students do not learn clinical medicine yet, a regular checkup case was developed. The case was of a 44-year-old female with a 5-minute interview for regular checkup. The SPs learned to present the social background as well as family and personal medical histories. They were also trained to provide specific answers to the expected specific questions of the medical student according to the scenario, and then rate the students’ history taking and PPI by SP trainer at Inje University College of Medicine. The instructions on what to do in a station were presented before the students enter the examination room. The student should build rapport, set the agenda, take medical history, family history, social history, and educate the patient. Three female SPs were recruited for the examination, and their real age was early fifties and have more than five experiences as SPs. One SP in each station met 32 to 33 medical students. History taking included four items, to be scored 0 or 4 (score of 0 to 16). PPI included five items, and scored from 0 to 4 (score of 0 to 20). The PPI rating scale used was the same marking sheet of the Busan–Gyeongnam consortium (Appendix 1) [9].
A total of 98 students participated in the CPX in March 2015. From this sample, 66 video recordings from two CPX rooms were available for analysis. The rest of the students were not included because of technical failure in the video recording from one CPX room. For this study, the study subjects were selected based solely on ranks of their PPI score. Ten students with high marks and another set of 10 students with low marks were selected at the author ’s discretion.
2. Roter interaction analysis system and coding
Roter interaction analysis system (RIAS) was chosen to code the verbal communication skill test in CPX. RIAS is a specific tool for coding medical dialogue between patients and providers, and the dynamics and consequences between them can be viewed [10]. It is widely used in medical exchange research for its reliability and validity. RIAS categories generally consist of task-focused behaviors and affective dimensions. Task behaviors include gathering data for understanding the patient’s problems as well as providing education and counseling to patients on illness and motivating patients to undergo the appropriate treatment. Affective behaviors include building relationships and patient–physician partnership [10]. Each utterance is assigned to one of 41 categories. An utterance was defined as a complete thought by either patient or physician, and ranges from a single word to a complete sentence. The 41 types of utterances can be classified under larger functional groupings: building a relationship, patient activating and partnering, data gathering (or question asking for patients), patient education, and counselling [10,11].
Coding is conducted directly from the audiotapes or videotapes without transcription [10]. In the present study, subjects’ videotaped CPX interviews were transcribed into text to avoid coding error, and then the two authors (KHP, HR) independently assigned each utterance to one of the 41 RIAS categories after watching videotaped recordings. Finally, one author (KHP), who attended a RIAS workshop, confirmed the types of each utterance, after discussion between two coders.
3. Study measures
Frequency (the number of utterances) of students’ communication behaviors were compared between the two groups.
Patient centeredness scores (PCS) were calculated based on the RIAS coding, and then compared between the two groups. PCS is defined in the Table 1. The difference between PCS 1 and PCS 2 is whether a physician’s biomedical information giving is treated as patient or doctor centered. Physician biomedical information giving is considered a biomedical detail in PCS 1, but as a patient-centered detail in PCS 2. Both scores are considered patient centered when they are over 1. PCS 1 has been the most commonly used in RIAS research, and published scores usually range from 0 to 5 [11,12].
Basic Characteristics, Patient Centeredness, and Physician’s Verbal Dominance between the Two Groups
Physician’s verbal dominance, or the ratio of the sum of physician statements divided by the sum of patient statements, was also calculated [13].
4. Statistical analysis
Continuous variables between the two groups were analyzed with Mann-Whitney U-test: PCS 1, PCS 2, physician’s verbal dominance, and frequencies of students’ communication behaviors. All continuous variables were then described as mean and standard deviation. The IBM SPSS ver. 20.0 (IBM Corp., Armonk, USA) was used for the analyses, and significance was declared at p<0.05.
Results
1. Student characteristics and basic results
Table 1 showed that the mean of the history taking score and PPI score in the total of 66 students were 8.06±4.12 and 9.55±1.81, respectively. Based on PPI scores, 11 high-ranking students (two students scored tie) and 10 low-ranking students were selected. The mean PPI score was 12.00 for the high-ranking students and 7.30 for the low-ranking ones. The former group also scored higher in history taking.
A total of 1,571 utterances were detected. For the high-ranking students, 895 utterances were detected, with the average number of utterances of 81.36 per exam. For the low-ranking students, 676 utterances were detected, with an average of 67.60 per exam. The mean of the frequency of utterances in all students, all SPs, and both groups were significantly higher for the high-ranking students.
PCS 1 and PCS 2 had no differences between the two groups. The same trend was seen for physician ’s verbal dominance (Table 1).
2. Patterns of utterances
Interrater agreement was high (κ=0.935, p<0.001). The most common type of utterance for high-ranking students was a paraphrase/checks for understanding, whereas that for low-ranking students was closed questions for the medical condition (Table 2). Medical information giving is the most common utterance of SPs in both group (Table 3).
Comparison of Communication Frequencies of Student Doctors: Type of Utterances
Comparison of Communication Frequencies of Standardized Patients: Type of Utterances
In terms of type of utterance, the high-ranking students used more paraphrase/checks for understanding, legitimizing statements, closed questions about psychosocial state and open questions about medical information compared with the low-ranking students. SPs who had consulted with high-ranking students requested for more services and more freely gave information on their medical condition compared with the low-ranking group (Table 3). Requests for services by SPs were detected in the four samples of high-ranking students group. An example was that the SP said, “Please perform a checkup of my body, and examine other areas that may need attention.”, when the student asked the SP her health problems and negotiated the agenda. In addition, the frequency with which both student groups used closed questions for medical information was similar, but high-ranking students used open questions more often. Accordingly, the SPs gave more medical information to high-ranking students.
Discussion
In this study, high-ranking students were determined by the PPI score evaluated by SPs. We consequently analyzed communication success from the SPs ’ viewpoint.
The frequency of utterance during the limited time of 5 minutes was higher in high-ranking students and their SPs, which indicates that high-ranking students had a smooth conversation with their SPs. In other words, they conversed with their SPs without any unnecessary pause, and more contents were exchanged. These results are similar to the previous study. Shin et al. [14] reported that female physicians had patient-centered encounters because they had more dialog with SP.
In medical practice, relationship building begins with personal remarks and social conversation. As all students initiate their medical encounters with personal remarks and social conversation, no difference was observed in the relationship building performance of both groups. Students might be nervous and stressed during encounters, based on the occurrence of few negative and positive talk. Emotional talk was also rare in both groups, but high-ranking students used legitimizing statements more often. Legitimizing statements indicated that it is normal for or understandable why SPs feel or act a certain way, and is a relatively easy form of emotional talk.
High-ranking, compared with low-ranking, students used paraphrasing/checks for understanding more often, a technique for participatory facilitation. Paraphrasing or clarification is an easy way to demonstrate attentive listening to SPs, while checks for understanding are a form of shared understanding. In RIAS coding, back-channel responses refer to the typical method of responding and listening attentively to a patient, encouraging them to talk. Interestingly, back-channel responses rarely occur in the Korean language; therefore, paraphrasing is used like the back-channel response in English. SPs might feel supported and encouraged to talk by students’ paraphrases. Since PPI is determined by the SPs’ subjective and comprehensive assessment, PPI can be correlated to their satisfaction about encounters with students. In the study conducted in the emergency department, patient satisfaction with a consultation is reportedly associated with the amount of talk relating to relationship building, patient activating, and partnering [15].
High-ranking students used more closed questions about psychosocial state and open questions about medical condition, when compared with their low-ranking counterparts. There was no difference in the frequency of closed questions about medical condition. Nevertheless, the SPs in high-ranking group showed the higher tendency to provide more medical information when responding to more open questions, leading to more freely shared medical information. This result might be attributed to the rapport between student and SP in the high-ranking group.
SPs who had consulted with high-ranking students requested for services more, such as those related to their concerns on their condition, when the student negotiated the agenda. Thus, the student should thoroughly explore the problems brought up by the SP.
In the current study, PCSs were not different between the two groups. The mean PCS 1 and 2 in both groups were greater than 1.0. As PCS is considered patient centered when greater than 1.0, both groups’ verbal communication can be considered patient centered. A reason is that, as aforementioned, the students and SPs in both groups had fewer emotional rapport-building opportunities. Meanwhile, when a physician’s biomedical information giving is included in the patient-centered side for PCS 2, PCS increased. Educating and counseling related to therapeutic regimens led to higher patient satisfaction [15]. Thus, biomedical information giving is regarded as a crucial patient-centered behavior.
In a research on actual patients, patient satisfaction was correlated with rapport building and psychosocial communication [16]. The factors contributing to patient satisfaction are various and dependent on the type of situation. For example, emergency patients were more satisfied with their consultations with emergency nurse practitioners than doctors, because emergency nurse practitioners provided more education and counseling on therapeutic regimens and patients could understand their condition and participate in the decision making [15].
Awkward laughs or pauses were noticed in the recordings of a number of students, such as when they could not find anything to say during the interview. As these utterances are part of paraverbal communication, they cannot be analyzed by RIAS. In addition, these behaviors might be a part of the reason that the count of utterances in the low-ranking students is low. Words themselves comprise only 7% of communication [17]. Therefore, nonverbal and paraverbal communication also need to be analyzed.
The present study has several limitations. First, the sample size is small. The authors selected only 11 high-ranking students and 10 low-ranking students to maximize the difference between the groups. In addition, the analysis was limited to only one CPX, routine checkup. Second, one author selected study subjects, and then conducted coding without being blind to the ranks of the students. The other authors conducted coding fully blinded. Although not being blind to the subjects could lead to an unfair result, interrater agreement was high. Third, nonverbal behaviors cannot be evaluated by RIAS. A description of nonverbal behaviors by SPs is the weakest category for medical students [18]. In addition, RIAS cannot measure all the potential domains of patient-centered communication [12]. RIAS categories are not equivalent to those in the PPI checklist. The RIAS can be used only for verbal communication behaviors, but not the propriety of the communication and sequence of utterances [19].
The number of utterances during the limited time of CPX was higher for the high-ranking students, which indicates that the process of conversation was active and smooth. Provision of more medical information (by SPs responding to open questions about medical conditions from students) could lead to high PPI scores during routine checkups. It may be the same when high-ranking students use more legitimizing statements and closed questions for psychosocial states and SPs request a service. Furthermore, nonverbal behaviors are considerable factors in medical communication, and therefore, nonverbal behaviors need to be analyzed.
Acknowledgements
The authors thank Mr. Kyung Seok Park who transcribed video files.
Notes
Funding
This work was supported by the 2015 Inje University research grant.
Conflicts of interest
None.
Authors’ contribution
KHP conceived the design of the study and drafted the initial manuscript. KHP and HR contributed to the construction of the evaluation criteria, and acquisition and analysis of the data. KHP interpreted the data. HR and SYP revised the initial manuscript critically for overall contents. All authors read, critically reviewed, and approved the final manuscript.
