| Home | E-Submission | Sitemap | Contact us |  
top_img
Korean J Med Educ > Volume 36(1); 2024 > Article
Mahmud, Sari, Sari, Arfian, and Zucha: The application of augmented reality for improving clinical skills: a scoping review

Abstract

Augmented reality technology had developed rapidly in recent years and had been applied in many fields, including medical education. Augmented reality had potential to improve students’ knowledge and skills in medical education. This scoping review primarily aims to further elaborate the current studies on the implementation of augmented reality in advancing clinical skills. This study was conducted by utilizing electronic databases such as PubMed, Embase, and Web of Science in June 2022 for articles focusing on the use of augmented reality for improving clinical skills. The Rayyan website was used to screen the articles that met the inclusion criteria, which was the application of augmented reality as a learning method in medical education. Total of 37 articles met the inclusion criteria. These publications suggested that using augmented reality could improve clinical skills. The most researched topics explored were laparoscopic surgery skills and ophthalmology were the most studied topic. The research methods applied in the articles fall into two main categories: randomized control trial (RCT) (29.3%) and non-RCT (70.3%). Augmented reality has the potential to be integrated in medical education, particularly to boost clinical studies. Due to limited databases, however, any further studies on the implementation of augmented reality as a method to enhance skills in medical education need to be conducted.

Introduction

Medical education consists of various methods, primarily on theoretical learning in classrooms and clinical skills in hospitals, where students gain clinical experiences. These learning methods mostly employ the traditional method. Traditional method involves having students come to class multiple times a week. The instructor of the class will often concentrate on instructing, assigning homework, and completing assessments like quizzes, examinations, or projects. The most popular method of instruction is the use of aural and visual aids or media, such as films, demonstrations, or case studies [1].
Rapid innovations and advancement in medical education technology, such as simulation technologies and open-access medical education have occurred recently. The application of augmented reality (AR) as one of simulation technologies in medical education is currently gaining popularity because of its potential to improve the learning qualities [2]. AR can improve the quality of educational settings by providing a life-like experience and interaction of the physical world through the addition of computer-generated sensory input such as sound, video, graphics, or GPS data [2,3]. Previous studies have demonstrated the implementation of AR in multiple settings including medication management and planning, learning tools in medical education and training, simulating objects or organs in surgery for better visualization, improving techniques in rehabilitation for improved outcomes, and many more. As a result, AR system applications will continue to advance and have potential use to facilitate the process of education in healthcare and medical education [2].
As a result of the use of digital strategies, modern healthcare professional education has undergone a considerable paradigm shift which is now represented in many curricula. Digital learning methods such as using software, videos, gaming, online classes, augmented and virtual reality has proven to have many advantages to improve students’ knowledge and skills. Digital methods are not limited by time or space, and they are flexible, allowing students to learn and practice repeatedly anywhere and anytime. Meanwhile, the disadvantages of this strategy involve higher costs for preparation and platform maintenance, the dependence on the device such as computer or smartphone and Internet connection and the lack of instructors’ presence [1]. The advantages and disadvantages of digital and traditional methods are detailed in Table 1.
Students’ self-confidence is the key factor that influences their success in demonstrating the acquisition of skills. Students want to feel safe during their practices, both for themselves and the patients. Digital learning, for example, AR could be an alternative to improve knowledge and skills in medical students and staffs by providing simulation for trainings [3]. This scoping review mainly purposes to describe the existing research on the use of AR for improving clinical skills.

Methods

1. Information sources

This study conducted by having a scoping review in which the researchers select relevant articles through medical journals and databases such as PubMed, Embase, and Web of Science. A scoping review examines an exploratory research issue and aims to identify important ideas (components), forms of evidence, and research gaps through knowledge synthesis. The titles and abstracts of the papers were examined and analyzed for relevance to the use of AR for enhancing clinical skills using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

2. Eligibility and inclusion criteria

This research involved all articles with the population or participants who were provided intervention using AR technology to compare the technology with the existing traditional methods to improve their clinical skills. This study used randomized control trial and experimental studies which compared two methods of study using AR and the traditional methods.

1) Exclusion criteria

Non-English language articles; discussion, book chapters, proceedings, and dissertations/thesis were excluded.

2) Databases

From the inception of the research to June 2022, the researchers had searched three databases; PubMed, Embase, and Web of Science.

3) Search strategy

This research employed a combination of Medical Subject Headings (MeSH) and free-text search terms for AR and skills as described in Table 2.
This study used Rayyan Intelligent Systematic Review (https://www.rayyan.ai/), a free online application that assisted systematic review methodology and meta-analysis projects, to filter the articles that met the inclusion criteria. This application allows user to search the existing citation records such as title, abstract, and author.

3. Data extraction and analysis

The extracted data obtained from various literatures were analyzed using both qualitative and descriptive method. The researchers retrieved descriptive data concerning year of publication, type of artificial intelligence used in the study, number of subjects, outcome of study, and parameter/assessment tool as well as took note on the results of the studies. In the analysis stage, all data were meticulously analyzed using Braun and Clarke’s qualitative thematic analysis, a widely used method in scoping reviews. In addition, this review applied a comprehensive thematic representation of certain research areas, and artificial intelligence application to improve skills of medical students were carefully selected. Familiarization of data set included type of study, type of AR, and outcome/parameter of assessment. The authors discussed theme review, data, and collation and excluded articles with had little engagement with the key issue (AR and skill improvement) and those that mentioned the bias issues.

Results

1. Synthesis of results

The initial search yielded a total of 4,258 entries. After the process of filtering, eliminating duplicates, screening, and excluding them based on titles and abstracts, 85 publications were reviewed in the study. A total of 37 articles were then selected for the full-review study. The PRISMA flow diagram of the study selection is presented in Fig. 1. This research focused on several countries: the United States (n=10), Canada (n=4), the United Kingdom (n=4), Germany (n=4), Switzerland (n=4), Netherlands (n=3), Serbia (n=2), Austria (n=2), Japan (n=2), Portugal (n=1), and Thailand (n=1). All study’s characteristics included in this study were described in Table 3 [4-40].

2. Randomized control trial articles

The previous studies used randomized control trials (RCTs) to investigate the application of AR. Eleven publications (29,7%) of 37 publications reviewed in this study employed RCT as the research method. These publications originated from a variety of medical science fields, the majority of which dealt with surgery. Ten articles investigated skill enhancement after adopting AR as a novel learning technique. By giving participants a new method of mentoring or teaching, AR system was applied to improve skills in cricothyroidotomy [4], transthoracic echocardiogram [5], and laparoscopic suturing task [6], Robotic system, simulator, and instructions using AR were also used in urethrovesical anastomosis [7], ophthalmoscopy and ophthalmic microsurgery [8-10], local anesthesia of inferior alveolar nerve block [11], dynamic hip screw guidewire [12], and single extracorporeal membrane oxygenation (ECMO) cannulation [13]. One study used AR with flashcards to improve emergency ultrasonography knowledge [14]. Ten articles found that the AR intervention group outperformed the traditional approach control group.

3. Non-randomized control trial articles

There were 26 articles (70,3%) of 37 articles examined in this study employed non RCT study methods. Besides, 11 articles concentrated on skill improvement, and all of the studies focused on improvement. AR is essential to help medical practitioners do laparoscopic suturing, knot-tying training, and better visualization for digital rectal examination. AR has been used in tele-mentoring to assist students in achieving better skills in port placement and abdominal incision. The implementation of AR to provide learning assisting is considered advantageous for acetabular cup orientation in total hip arthroplasty. The growing popularity of AR in the medical field aids numerous departments in developing their skills and simplifying their work. The non-RCT articled reviewed stated that Immersive Touch simulator can increase success rate of ventriculostomy catheter placement [15]. OsiriX assists surgeons in navigating patients’ anatomy [16]. The improvement of clinical skills is possible using intelligent tutoring system for micromanipulation in ophthalmic surgery [17]. Despite requiring improvement, ODG (Osterhout Design Group) R-7 Smartglasses (https://www.osterhoutgroup.com/products-r7HL/) is used in temporal bone surgery experience [18], Meanwhile, other 15 articles had investigated the application of AR in skill improvement, with an emphasis on the system’s usability or feasibility.

4. Augmented reality in laparoscopic surgery skills

Minimally invasive surgery technique such as using laparoscopy for surgery has become widespread in various areas of surgery. This technique has numerous benefits including a lower risk of infection, less surgical stress and complications, and a shorter hospital stay and recovery time. The application of AR is able to facilitate the process of learning and training of laparoscopic surgery skills. The majority of articles in this review investigated the utilization of AR to enhance laparoscopic surgical skills. One study examined how instructional video scheduling has affected learning curves for laparoscopic skill training. The participants first received a course on the basic skills on virtual reality simulator before divided into two groups of instructional videos either preoperatively or intraoperatively. Both groups improved their laparoscopic skills, with no significant differences [19]. Another study looked on the use of optical see-through AR to view an interactive virtual pointer on the laparoscopy. The participants would be separated into two groups: HoloPointer and standard. The results revealed that the HoloPointer group outperformed the regular group. The HoloPointer group also exhibited a reduction in mistakes and increased movement economy [20]. The ProMIS Augmented Reality laparoscopic simulator was investigated to improve laparoscopic suturing abilities and found a substantial difference in the participants’ performance curves and metrics [21,22]. Nineteen participants completed a 6-week training program utilizing a virtual reality simulator (LapSim) and were evaluated using an AR simulator (ProMIS) for cholecystectomy performance, demonstrating improvement in surgical skills [23]. The use of AR telementoring for laparoscopic surgery intracorporeal suturing task shown faster skill development and fewer failures than standard learning methods [6].

5. Augmented reality in ophthalmology

In order to establish a diagnosis, indirect ophthalmoscopy was a necessary examination in ophthalmology. Training for these skills requires trial and error, and numerous approaches, including fundus dummies, have been devised. Augmented reality ophthalmoscopy (ARO) was created to assist students in learning and practicing binocular indirect ophthalmoscopy. The ARO group improved their ophthalmoscopy skills faster than the traditional group [9]. EyeSI, which was comparable to ARO, an AR binocular indirect ophthalmoscopy simulator developed and investigated in Canada to be contrasted with traditional teaching methods, was developed and studied. The EyeSI group outperformed the typical group and improved significantly [8]. Students are trained for skill improvement in microsurgical operations using AR. One study looked into the use of AR to improve motor abilities and intelligent tutoring systems to improve training efficiency. The results demonstrated that adopting AR as a training tool improved skill. Students’ performance improved with both intelligent tutoring system and natural sequential steps or classic training [10,17].

Discussion

1. Principal findings

AR is a novel technology that has a variety of applications that are constantly being developed. AR is a technology that combines two-dimensional or three-dimensional virtual objects into a three-dimensional real environment, then projects these virtual objects in real time. One of the elements contributing to its growing popularity is the ability to observe and interact with digital objects without having to turn away from the real world to view the monitor displaying the appropriate medical imaging [41]. The difference between AR and virtual reality are AR using real-world setting and enhances both virtual and real world, users can control their presence in real world and AR can be accessed with smartphone while virtual reality is completely virtual, controlled by the system and required headset device. This scoping analysis was carried out to describe the existing research on the use of AR in improving medical practitioners’ clinical abilities. This scoping review discovered 37 papers that met the criteria. More than 80% of the publications in this review stated that using AR could improve clinical skills. AR is able to outperform the conventional methods of learning by providing more improvement of students’ knowledge and skills.
A comparison of the effects of mobile AR learning and textbook learning on medical students revealed that the mobile AR group improved their knowledge more than the control group [24]. These findings were also seen in the publications we examined for this investigation. They discovered that AR improved and increased the effectiveness of learning.
Because of the disparities in what medical students learned in school and in clinical settings, it was discovered that surgery and interventional procedures received more attention in the development of new digital learning tools. Students’ knowledge and abilities will not improve until they practice them. Chowriappa et al. [7] used an RCT to compare the use of AR, namely hand-on surgical training, and traditional approaches in skills training for robot-assisted urethrovesical anastomosis. The results demonstrated that the AR training increased students’ skills [7]. Other studies focusing on skill improvement in surgery and the interventional procedure category were computed tomography (CT)-guided intervention [25], port placement and abdomen incision [26], basic and laparoscopic skills and suturing [16,22,27], cholecystectomy [20,23], total hip arthroplasty [28], percutaneous renal access and nephrolithotomy [29,30], percutaneous rhizotomy [31], ECMO cannulation, and transthoracic echocardiogram [5,13]. It is found that AR is used to improve language learning, mechanical skills, and spatial abilities in addition to clinical skills.
Each AR application or device has its unique user interface and benefits that may or may not be applicable in all circumstances. The standard approaches evaluated still increase clinical skill learning, but the AR groups demonstrated greater improvement. Previous research has demonstrated the advantages of using AR in medical education, such as minimizing radiation exposure. Park et al. [42] investigated how the use of AR could increase procedural efficiency in terms of process time, puncture, and radiation exposure for CT-guided intervention. Park et al. [42] discovered that utilizing AR in CT-guided intervention reduced the mean CT dose index from 28.7 to 16.9 mGy.
AR also shows its effectiveness in improving surgical skills through telementoring, which offers experts in faraway locations real-time guidance on surgical operations. This can be a potential to use AR in the future as a tool for supervision and skill development. For surgical novices, expert supervision is crucial, and AR telementoring may provide more flexibility and efficiency than traditional “direct” supervision, which in certain cases might be demanding and ineffective. Also, considering that objective performance indicators may be monitored utilizing AR platforms in a simulated or, ideally, real-life operational environment to assess technical ability, the function of AR as an evaluation tool is worth taking into account [43,44].
AR applications are most successful in motivating users for training and knowledge transmission. It serves as a tool to provide interactive experiences and makes users feel comfortable with decisions and supported while doing skills or activities. However, technological concern such as latencies, eyesight, perception, a lack of ergonomic gear, location and accuracy failures, illumination occlusions, and other technical issues remain as the disadvantages of this digital method of learning and needed to be solved [45]. As this technology develops, further research will be required to assess various modalities that AR may be applied to. To be effective, future AR devices would also need to be able to interface and link up with other medical equipment, such emergency medical record systems, ultrasound machines, and vital monitors. It will be necessary to create software standards for this [2].

2. Strength and limitations

This review may provide insight into the potential application of AR as a learning tool in medical education and training. It also could inform the application of AR across various areas or departments in medical fields. The majority of articles on the development of AR technology highlight industrialized countries and significant medical education centers. AR is still considered a novel and expensive technology in medical sectors. Another limitation is that the included studies only discussed small-scale AR implementations and used a variety of AR platforms. For further research on the use of AR to improve students' skills, the researchers advise using and applying AR by using an application (mobile or PC) or online that provides AR simulation, if access and expense are limited.

Conclusion

AR has the potential to be integrated in medical education, particularly to boost clinical studies. Due to limited databases, however, any further studies on the implementation of AR as a method to enhance skills in medical education need to be conducted.

Acknowledgments

None.

Notes

Funding
None.
Conflicts of interest
No potential conflict of interestrelevant to this article was reported.
Author contributions
Conception or design of the work: MM, DCR, DS, NA; data collection: MM; data analysis: MM, DCR, DS, NA; data interpretation: MM, DCR, DS, NA, MAZ; drafting the article: MM; critical revision of the article: MM, DCR, DS, NA, MAZ; and final approval of the version to be published: DCR, DS, NA.

Fig. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses Diagram of Study Selection

kjme-2024-285f1.jpg
Table 1.
The Advantages and Disadvantages of Digital and Traditional Learning Method [1]
Method Advantages Disadvantages
Digital (mLearning) - Unlimited time and place - Requires higher material preparation and platform maintenance costs
- Flexible so students can move at their own learning pace - Lack of presence of instructors or tutors
- Students can study, repeat material and practice many times outside class hours - Depends on computer or smartphone, internet connection, learning system
- It can give rise to distrust or cheating by students by asking other people to replace their presence during class or exams
Traditional - Direct and face-to-face interaction between students and teachers - Implementation depends on the presence of students and teachers at a specific place and time
- Students and teachers can immediately get feedback from the implementation of learning
- Teachers and students have a better connection - The speed of learning or the material delivered is determined by the teacher (some students may feel the material is delivered too quickly, but there are also students who feel the material is delivered too slowly)
- Teachers can assess students who actively participate and understand the material or students who are not interested or who do not understand the material
Table 2.
Search Strategy
Search strategy
Embase ‘medical student’/exp OR ‘resident’/exp OR ‘consultation’/exp OR ‘postgraduate student’/exp OR ‘medical student*’:ab,ti OR ‘fellow*’:ab,ti OR ‘residen*’:ab,ti OR ‘trainee*’:ab,ti OR ‘consultan*’:ab,ti OR ‘postgraduate*’:ab,ti AND ‘augmented reality’/exp OR ‘augmented realit*’:ab,ti OR image*:ab,ti AND ‘skill’/exp OR skill*:ab,ti OR proficienc*:ab,ti
Web of Science medical student OR medical students OR fellow OR fellowship OR fellowships OR resident OR residents OR trainee OR trainees OR consultant OR consultants OR postgraduate OR postgraduates AND augmented reality OR image OR augmented realities OR images AND skill OR skills OR proficiency OR proficiencies
Table 3.
Summary of the Articles for Scoping Review
Author (year) Procedure Study design No. of subjects Intervention Control Outcome Assessment tool/parameter Results
Albrecht et al. [24] (2013) Legal medicine Non-RCT 6 mARble app on mobile device NA Usability of system AttrakDiff2 questionnaire based on Hassenzahl’s paradigm of experience design Preliminary evaluation from Hassenzahl’s AttrakDiff2 questionnaire showed an average value of 0.381 (CI, ±0.492) for pragmatic quality.
Amiras et al. [25] (2021) CT-guided intervention Non-RCT 16 CT-guided intervention with AR NA Feasibility of system Feedback and questionnaire From 16 participants, 11 felt the accuracy and realism was adequate for training and 12 felt more confident about their CT biopsy skills after training session.
Andersen et al. [26] (2016) Port placement + abdominal incision Non-RCT 20 STAR Conventional system Skill improvement Placement error, number of focus shift, time to task completion STAR group had less placement error (45% and 68%) and fewer focus shifts (86% and 44%), but they were more slowly (19% for each task). rather than the conventional group. This means that STAR system gave less placement error and fewer focus shifts in the procedure but giving more time to complete the task.
Andersen et al. [4] (2019) Surgical telementoring of cricothyroidotomy RCT 20 Telementoring using a visualization of future steps–STAR Conventional telestrator-based telementoring (Telestrator) Skill improvement Idle time ratio, recall error, and task performance Trainees in the future step visualization group had 48% smaller idle ratio, 26% less recall error, and 10% higher task performance score.
Botden et al. [21] (2009) Laparoscopic suturing/knot tying Non-RCT 18 Performing suturing knots on suturing model on AR ProMIS ver. 2.0 NA Skill improvement Time spent during knot tying and quality of knot The result showed a statistically significant difference between the baseline knot and the knot at top of performance curve. Significant difference also found between both of assessment scores.
Broekema et al. [19] (2016) Laparoscopic skill Non-RCT 21 Preoperative AR instructional video Interactive instructional video Skill improvement MAP consist of left and right pathlength, insertion, angular area, volume, and time Both groups showed significant improvements from first task to fourth. After the fourth repetition, when the performance of the two groups was compared, no significant difference in improvement were found.
Cecilio-Fernandes et al. [5] (2020) Transthoracic echocardiogram RCT 36 AR visual feedback (HS) (n=16), combination expert and augmented feedback (EF+HS) (n=10) EF (n=10) Skill improvement Time of obtaining images, image quality On the first test EF group were faster at obtaining images. On the retention test, there were no difference in speed of obtaining images for all group, but EF+HS group had higher score on image quality than the other groups.
Chowriappa et al. [7] (2015) Robot-assisted UVA RCT 52 HoST AR simulator Conventional method Skill improvement UVA evaluation score (needle driving, needle positioning and suture placement); GEARS score HoST group had significant difference of evaluation score in needle driving, needle positioning and suture placement, this group also had higher GEARS score.
Glossop et al. [32] (2004) Projecting preoperative plans Non-RCT NA The XarTrax AR system Conventional robotic telesurgery system Evaluation of system Accuracy of system on device The system and device were recommended for carrying out precise preoperative planning and for teleconsultation applications, which allow for the effective transmission of scheduled or real-time consultations to a local caregiver with less training.
Heinrich et al. [20] (2021) Laparoscopic surgery (cholecystectomy) Non-RCT 10 AR interactive virtual pointer Conventional method Applicability and feasibility Economy of movement, error rates, user performance The use of interactive virtual pointer for training showed a significant improvement in economy of movement (p=0.047) and error rates (p=0.047), the user performance also got good scores (total z-score; p=0.031).
Ienghong et al. [14] (2021) Emergency ultrasound knowledge RCT 46 Practical ultrasound flashcards with AR and traditional POCUS learning (n=23) Traditional POCUS learning only (n=23) Knowledge improvement Pretest and post-test using multiple-choice question, questionnaire Participants in flashcard group had better performance on POCUS knowledge post-test than control group. Participants were satisfied with the flashcards.
Kollmann et al. [33] (2021) Learning knobology of ultrasound equipment Non-RCT Students Detection of the equipment using AR app NA Usability of application Questionnaire The app had the ability to detect specific ultrasound equipment in real environment and can be used as preparation of practical course or orientation within the lab.
Kratzke et al. [34] (2022) Tumor ablation performance Non-RCT 15 AR Emprint SX navigation Standard ultrasound guidance Evaluation of ablation (efficacy and efficiency) Percentage under-ablation and over-ablation, time of trial, number of attempts of antenna position 281 trials were done with the results volume decreased of under-ablations and over-ablations, less task completion time and fewer antenna placement attempts with Emprint SX. In novice group, all metrics were improved by using Emprint SX navigation.
Leitritz et al. [9] (2014) Binocular indirect ophthalmoscopy RCT 37 ARO (n=19) Conventional ophthalmosco py (n=18) Skill improvement Ophthalmoscopy training score Median of training score for conventional group was 1.2 and showed significant difference to ARO group.
Logishetty et al. [28] (2019) Acetabular cup orientation in total hip arthroplasty Non-RCT 24 AR guidance One-on-one instruction from surgeon Skill improvement Accuracy of orienting cup, errors, feedback/ questionnaire of users’ experience AR-guidance group had smaller mean errors in orientation than control group. Both groups showed skill improvement. Eleven of 12 participants said would use the AR-guidance method as training tool and 10 of 12 for procedure-specific training.
Mendes et al. [35] (2020) Insertion of intravenous needles Non-RCT 18 Needle insertion with PINATA Conventional method Skill improvement Task completion time and number of needle insertion errors; semi-structured interviews Correlation was found between task completion time in both methods. The differences of time and errors were found between expert and novice participants. Fourteen participants considered that PINATA can outperform the conventional system of needle insertion.
Menozzi et al. [10] (2020) Ophthalmic microsurgery training RCT 23 AR ITS Natural sequence method Skill improvement Motor skill performance Participants underwent training in 5 days. The result showed both training method gave improvement of motor skill. There was no significance difference in both groups, but for the first 3 days of training, the ITS group showed steeper training progress than natural sequence group.
Mladenovic et al. [11] (2019) Local anesthesia of IANB RCT 41 Mobile AR simulator (n=22) Control group (n=19) Skill improvement Time for performing anesthesia, anesthetics success, post-clinical questionnaire The experimental group had shorter period of time (50.0±14.3 seconds vs. 68.4±25.5 seconds) in performing IANB and higher success rate than the control group. AR group also had 90.9% anesthesia success rate, while control group was 73.7%.
Mladenovic et al. [36] (2022) Local anesthesia administration Non-RCT 19 E-learning training using AR dental simulator (simulated-based games) NA Effectiveness of simulation-based games Validated post-training survey All participants felt comfortable, and over 90% believed that the games could facilitate learning process and had benefit.
Mu et al. [29] (2020) Ultrasound-guided PCA Non-RCT 30 AR simulator for ultrasound-guided PCA NA Effectiveness of simulator Face validity, content validity, performance score The average face validity score of 4.39 and content validity score of 4.53. There was a a significant difference (p<0.05) in performances between experts and novices. Significant performance improvement was found in novice participants.
Muangpoon et al. [37] (2020) Digital rectal examination Non-RCT 19 AR model Real model Movement in examining finger is accurately aligned the virtual and real model Questionnaire of system usefulness Movement in examining finger were accurately aligned the virtual and real model with a frame rate of 60 fps and mean error of 3.9 mm. Participants also found that the movement was realistic (mean±SD, 3.9±1.2).
Müller et al. [30] (2013) Percutaneous nephrolithotomy Non-RCT 53 Mobile AR NA Feasibility of system Radiation exposure, puncture time The trainee had best performance in puncturing time by using AR, while experts did their best with fluoroscopy. Mobile AR lowered radiation exposure by a factor of 3 for trainee and 1.8 for experts. Mean visualization accuracy was 2.5 mm.
Nagayo et al. [27] (2021) Suture training Non-RCT 2 AR motion capture system of surgical instruments and a 3D replication system of captured procedures on the surgical field NA Usability of system Tracked time of AR marker, procedure time; performance and usability of replication system ANOVA demonstrated a significant difference among the three AR markers for both experts. Tukey-Kramer test showed significant differences (p<0.05) for all pairs of AR markers in both experts. The 3D replication system can show the entire 3D models of surgical instruments withing FOV and users can see both models and actual instruments then imitate the replicated procedure.
Nomura et al. [23] (2015) Laparoscopic skills (cholecystectomy) Non-RCT 19 Virtual reality simulator (LapSim) NA Skill improvement Pretest and post-test using ProMIS AR Performance score was better after LapSim training in execution time of the task (p<0.001), left and right instrument path length (p=0.001), left and right instrument economy of movement (p<0.001) than before training.
Oostema et al. [22] (2008) Laparoscopic skills Non-RCT 46 ProMIS AR simulator NA Feasibility of system Time, path length smoothness of the procedure The result showed a significant correlation between experience and performance in time, path length and smoothness for task 2 to 4. The strongest correlation was found for the knot-tying task (R2=0.60 for time and 0.59 smoothness).
Rai et al. [8] (2017) Binocular indirect ophthalmoscopy RCT 28 EyeSI AR BIO (n=13) Conventional BIO (n=15) Skill improvement Raw score, total time elapsed and performance score of 3 tasks The control group outperformed the AR simulator group in three metrics of evaluation.
Rochlen et al. [38] (2017) Central line insertion Non-RCT 40 First-person POVAR NA Usability and feasibility of first POV AR Open ended questions and free-text responses Participants reported that the AR system was realistic (77.5%) and the ability to view the internal anatomy was helpful (92.5%). The success rate of placing the needle was similar between experienced and non-experienced participants.
Ropelato et al. [17] (2020) Micromanipulation performance in ophthalmic surgery Non-RCT 50 ITS using AR system Fixed task sequence Skill improvement Performance score of three repetition (before first training, before second training, after second training A significant improvement in the first session of micromanipulation performance for both groups. For the second session, only ITS group had further improvement.
Shakur et al. [31] (2015) Percutaneous trigeminal rhizotomy Non-RCT 92 AR simulator for percutaneous trigeminal rhizotomy NA Usefulness of simulator Post graduate year, number of fluoroscopy shots, distance from the ideal entry point, distance from ideal target (lower score is better) The mean distance from the entry point (9.4 mm vs. 12.6 mm, p=0.01), the distance from the target (12.0 mm vs. 15.2 mm, p=0.16), and final score (31.1 vs. 37.7, p=0.02) of senior residents was lower than the junior residents. The mean number of fluoroscopy shots was similar between two groups. Increased postgraduate year level had significant result with decreased distance from ideal entry point, shorter distance from target, and better final score.
Sugand et al. [12] (2019) Dynamic hip screw guidewire insertion RCT 45 FluoroSim AR simulator Conventional method Skill improvement Tip-apex distance, predicted cut-out rate, total procedural time, total number of radiographs, total number of guidewires retries The training cohorts had significant improvement in all metrics. Significant difference of both groups was found in procedural time (25%), number of radiograph (57%), and number of guidewires retries (100%). After the 1-week washout period, there was also an improvement in the baseline of the learning curve, which suggests skill retention.
Vera et al. [6] (2014) Fundamental of laparoscopic surgery suturing task RCT 19 ART Traditional mentoring Skill improvement Time and errors of the performance ART group gained faster (b=-0.567, R2=0.92) skill improvement than the control group (b=-0.453, R2=0.74), even after 10 repetition or an hour of practice (mean 167.4 seconds vs. 242.4 seconds, p=0.014). ART group also had fewer errors (8 vs. 13).
Volonté et al. [16] (2011) Laparoscopic and robotic surgery Non-RCT Patients who underwent laparoscopic operations AR OsiriX NA Making surgical interventions easier, faster, and safer User performance OsiriX gave benefits to surgeon such as navigations through patients’ anatomy, high lightning regions, and marked pathologies.
Wagner et al. [39] (1999) Craniomaxillofacial surgical procedure Non-RCT 27 AR tele-assisted surgery method NA Feasibility of system SPS Tele-assisted surgery was applied successfully. There were found technical problems in six cases but did not interfere the system performance.
Wolf et al. [13] (2021) Single ECMO cannulation training RCT 21 AR instructions Conventional instructions Skill improvement Training times, detailed error protocol, standardized UEQ AR instructions was associated with slightly higher training times and lower error counts. Training times and error count did not have significant correlation.
Yong et al. [18] (2019) Temporal bone surgery Non-RCT 7 ODG R-7 AR Smartglasses Conventional method Rating of experience and user-friendliness, ability to perform/teach tasks, quality of communication Visual analog score was used to obtain mean, SD and range. Open-ended questions were used for collecting the strength and weakness data Both residents and instructors enjoyed being able to connect by voice, text message, or sending and receiving editable still images. Technical issues were still found and need further improvement.
Yudkowsky et al. [15] (2013) Ventriculostomy catheter placement Non-RCT 16 Immersive touch AR system NA Success rate and improvement of the procedure Pre-test and post-test, performance score, questionnaire The result was the success rate of simulator cannulation was increased and actual procedure outcomes showed improvement. Residents also reported that the simulations were realistic and helpful in improving their skills.
Zhao et al. [40] (2020) Neonatal endotracheal intubation Non-RCT 45 Capturing motions and visualization using AR HMD NA Accuracy of learning model The classification accuracy of machine learning model in this study was 84.6%.

RCT: Randomized control trial, NA: Not applicable, CI: Confidence interval, CT: Computed tomography, AR: Augmented reality, STAR: System for telementoring with AR, MAP: Motion analysis parameter, HS: Help screen, EF: Expert feedback, UVA: Urethrovesical anastomosis, HoST: Hands-on surgical training, GEARS score: Global Evaluative Assessment of Robotic Skills score, POCUS: Point-of-care ultrasound, ARO: AR ophthalmoscopy, PINATA: Pinpoint insertion of intravenous needles via AR training assistance, ITS: Intelligent tutoring system, IANB: Inferior alveolar nerve block, PCA: Percutaneous renal access, SD: Standard deviation, 3D: Three-dimensional, ANOVA: Analysis of variance, FOV: Field of view, POV: Point-of-view, ART: AR telementoring, SPS: System performance scale, ECMO: Extracorporeal membrane oxygenation, UEQ: User experience questionnaire, HMD: Head-mounted display.

References

1. Tran B. Educational experiences with traditional, online and hybrid learning environments. J Educ Soc Policy 2016;3(3):60-63.

2. Munzer BW, Khan MM, Shipman B, Mahajan P. Augmented reality in emergency medicine: a scoping review. J Med Internet Res 2019;21(4):e12368.
crossref pmid pmc
3. Kurt Y, Öztürk H. The effect of mobile augmented reality application developed for injections on the knowledge and skill levels of nursing students: an experimental controlled study. Nurse Educ Today 2021;103:104955.
crossref
4. Andersen DS, Cabrera ME, Rojas-Muñoz EJ, et al. Augmented reality future step visualization for robust surgical telementoring. Simul Healthc 2019;14(1):59-66.
crossref pmid
5. Cecilio-Fernandes D, Cnossen F, Coster J, Jaarsma AD, Tio RA. The effects of expert and augmented feedback on learning a complex medical skill. Percept Mot Skills 2020;127(4):766-784.
crossref pmid pdf
6. Vera AM, Russo M, Mohsin A, Tsuda S. Augmented reality telementoring (ART) platform: a randomized controlled trial to assess the efficacy of a new surgical education technology. Surg Endosc 2014;28(12):3467-3472.
crossref pmid pdf
7. Chowriappa A, Raza SJ, Fazili A, et al. Augmentedreality-based skills training for robot-assisted urethrovesical anastomosis: a multi-institutional randomised controlled trial. BJU Int 2015;115(2):336-345.
crossref pmid
8. Rai AS, Rai AS, Mavrikakis E, Lam WC. Teaching binocular indirect ophthalmoscopy to novice residents using an augmented reality simulator. Can J Ophthalmol 2017;52(5):430-434.
crossref pmid
9. Leitritz MA, Ziemssen F, Suesskind D, et al. Critical evaluation of the usability of augmented reality ophthalmoscopy for the training of inexperienced examiners. Retina 2014;34(4):785-791.
crossref pmid
10. Menozzi M, Ropelat S, Köfler J, Huang YY. Development of ophthalmic microsurgery training in augmented reality. Klin Monbl Augenheilkd 2020;237(4):388-391.
crossref pmid
11. Mladenovic R, Pereira LA, Mladenovic K, Videnovic N, Bukumiric Z, Mladenovic J. Effectiveness of augmented reality mobile simulator in teaching local anesthesia of inferior alveolar nerve block. J Dent Educ 2019;83(4):423-428.
crossref pmid pdf
12. Sugand K, Wescott RA, Carrington R, Hart A, van Duren BH. Training and transfer effect of FluoroSim, an augmented reality fluoroscopic simulator for dynamic hip screw guidewire insertion: a single-blinded randomized controlled trial. J Bone Joint Surg Am 2019;101(17):e88.
pmid
13. Wolf J, Wolfer V, Halbe M, Maisano F, Lohmeyer Q, Meboldt M. Comparing the effectiveness of augmented reality-based and conventional instructions during single ECMO cannulation training. Int J Comput Assist Radiol Surg 2021;16(7):1171-1180.
crossref pmid pmc pdf
14. Ienghong K, Kotruchin P, Tangpaisarn T, Apiratwarakul K. Practical emergency ultrasound flashcards with augmented reality in teaching point-of-care ultrasound in ER. Open Access Maced J Med Sci 2021;9(E):39-42.
crossref pdf
15. Yudkowsky R, Luciano C, Banerjee P, et al. Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul Healthc 2013;8(1):25-31.
crossref pmid
16. Volonté F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P. Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 2011;18(4):506-509.
crossref pmid
17. Ropelato S, Menozzi M, Michel D, Siegrist M. Augmented reality microsurgery: a tool for training micromanipulations in ophthalmic surgery using augmented reality. Simul Healthc 2020;15(2):122-127.
pmid
18. Yong M, Pauwels J, Kozak FK, Chadha NK. Application of augmented reality to surgical practice: a pilot study using the ODG R7 Smartglasses. Clin Otolaryngol 2020;45(1):130-134.
crossref pmid pdf
19. Broekema TH, Talsma AK, Wevers KP, Pierie JE. Laparoscopy instructional videos: the effect of preoperative compared with intraoperative use on learning curves. J Surg Educ 2017;74(1):91-99.
crossref pmid
20. Heinrich F, Huettl F, Schmidt G, et al. HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training. Int J Comput Assist Radiol Surg 2021;16(1):161-168.
crossref pmid pdf
21. Botden SM, de Hingh IH, Jakimowicz JJ. Suturing training in augmented reality: gaining proficiency in suturing skills faster. Surg Endosc 2009;23(9):2131-2137.
crossref pmid
22. Oostema JA, Abdel MP, Gould JC. Time-efficient laparoscopic skills assessment using an augmented-reality simulator. Surg Endosc 2008;22(12):2621-2624.
crossref pdf
23. Nomura T, Mamada Y, Nakamura Y, et al. Laparoscopic skill improvement after virtual reality simulator training in medical students as assessed by augmented reality simulator. Asian J Endosc Surg 2015;8(4):408-412.
crossref pmid
24. Albrecht UV, Folta-Schoofs K, Behrends M, von Jan U. Effects of mobile augmented reality learning compared to textbook learning on medical students: randomized controlled pilot study. J Med Internet Res 2013;15(8):e182.
crossref pmid pmc
25. Amiras D, Hurkxkens TJ, Figueroa D, et al. Augmented reality simulator for CT-guided interventions. Eur Radiol 2021;31(12):8897-8902.
crossref pmid pmc pdf
26. Andersen D, Popescu V, Cabrera ME, et al. Medical telementoring using an augmented reality transparent display. Surgery 2016;159(6):1646-1653.
crossref pmid
27. Nagayo Y, Saito T, Oyama H. A novel suture training system for open surgery replicating procedures performed by experts using augmented reality. J Med Syst 2021;45(5):60.
crossref pmid pmc pdf
28. Logishetty K, Western L, Morgan R, Iranpour F, Cobb JP, Auvinet E. Can an augmented reality headset improve accuracy of acetabular cup orientation in simulated THA?: a randomized trial. Clin Orthop Relat Res 2019;477(5):1190-1199.
crossref pmid
29. Mu Y, Hocking D, Wang ZT, Garvin GJ, Eagleson R, Peters TM. Augmented reality simulator for ultrasound-guided percutaneous renal access. Int J Comput Assist Radiol Surg 2020;15(5):749-757.
crossref pmid pdf
30. Müller M, Rassweiler MC, Klein J, et al. Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. Int J Comput Assist Radiol Surg 2013;8(4):663-675.
crossref pdf
31. Shakur SF, Luciano CJ, Kania P, et al. Usefulness of a virtual reality percutaneous trigeminal rhizotomy simulator in neurosurgical training. Neurosurgery 2015;11 Suppl 3:420-425.
crossref pdf
32. Glossop N, Wang Z, Wedlake C, Moore J, Peters T. Augmented reality laser projection device for surgery. Stud Health Technol Inform 2004;98:104-110.

33. Kollmann C, Santner L. A prototype of a handheld augmented reality application to get familiar with ultrasound equipment knobology. Med Ultrason 2021;23(4):438-442.
crossref pmid pdf
34. Kratzke IM, Goss RS, Razzaque S, et al. Navigation improves tumor ablation performance: results from a novel liver tumor simulator study. Am Surg 2023;89(5):1693-1700.
crossref pmid pdf
35. Mendes HC, Costa CI, da Silva NA, Leite FP, Esteves A, Lopes DS. PIÑATA: pinpoint insertion of intravenous needles via augmented reality training assistance. Comput Med Imaging Graph 2020;82:101731.
crossref pmid
36. Mladenovic R, AlQahtani S, Mladenovic K, Bukumiric Z, Zafar S. Effectiveness of technology-enhanced teaching methods of undergraduate dental skills for local anaesthesia administration during COVID-19 era: students’ perception. BMC Oral Health 2022;22(1):40.
crossref pmid pmc pdf
37. Muangpoon T, Haghighi Osgouei R, Escobar-Castillejos D, Kontovounisios C, Bello F. Augmented reality system for digital rectal examination training and assessment: system validation. J Med Internet Res 2020;22(8):e18637.
crossref pmc
38. Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc 2017;12(1):57-62.
pmc
39. Wagner A, Millesi W, Watzinger F, et al. Clinical experience with interactive teleconsultation and teleassistance in craniomaxillofacial surgical procedures. J Oral Maxillofac Surg 1999;57(12):1413-1418.
crossref
40. Zhao S, Xiao X, Wang Q, et al. An intelligent augmented reality training framework for neonatal endotracheal intubation. Int Symp Mix Augment Real 2020;2020:672-681.
crossref pmid pmc
41. Barcali E, Iadanza E, Manetti L, Francia P, Nardi C, Bocchi L. Augmented reality in surgery: a scoping review. Appl Sci 2022;12(14):6890.
crossref
42. Park BJ, Hunt SJ, Nadolski GJ, Gade TP. Augmented reality improves procedural efficiency and reduces radiation dose for CT-guided lesion targeting: a phantom study using HoloLens 2. Sci Rep 2020;10(1):18620.
crossref pmid pdf
43. Kovoor JG, Gupta AK, Gladman MA. Validity and effectiveness of augmented reality in surgical education: a systematic review. Surgery 2021;170(1):88-98.
crossref pmid
44. El-Sabawi B, Magee W 3rd. The evolution of surgical telementoring: current applications and future directions. Ann Transl Med 2016;4(20):391.
crossref pmid pmc
45. Villagran-Vizcarra DC, Luviano-Cruz D, Pérez-Domínguez LA, Méndez-González LC, Garcia-Luna F. Applications analyses, challenges and development of augmented reality in education, industry, marketing, medicine, and entertainment. Appl Sci 2023;13(5):2766.
crossref
TOOLS
PDF Links  PDF Links
PubReader  PubReader
ePub Link  ePub Link
XML Download  XML Download
Full text via DOI  Full text via DOI
Download Citation  Download Citation
  Print
Share:      
METRICS
0
Crossref
0
Scopus
848
View
25
Download
Editorial Office
The Korean Society of Medical Education
(204 Yenji-Dreamvile) 10 Daehak-ro, 1-gil, Jongno-gu, Seoul 03129, Korea
Tel: +82-2-2286-1180   Fax: +82-2-747-6206
E-mail : kjme@ksmed.or.kr
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © 2024 by Korean Society of Medical Education.                 Developed in M2PI