The Common Achievement Tests Organisation (CATO) was established in 2005 as a collective of medical and dental universities throughout Japan [1]. CATO is responsible for administrating the computer based testing (CBT), the pre-clinical and the post-clinical clerkship objective structured clinical examination (OSCE). In 2021, the Medical Practitioners’ Act was amended to allow medical students who have passed the CBT and the pre-clinical clerkship OSCE to perform medical procedures under the supervision of a physician [2]. As a result, this examination has become increasingly important in ensuring that medical students are competent to provide patient care during clinical training. In Japanese medical education, the nationally standardized preclinical clerkship OSCE has assessed clinical examination skills since 2023 [3]. CATO is involved in creating and reviewing OSCE stations for each medical school, and it also contributes to academic research activities for developing and enriching educational content in medical universities and other institutions by consolidating the results of these evaluations. It also serves as a focal point for responding to objections from each medical university. As a medical educator who is both a faculty member at a Japanese medical university and an active member of CATO, I would like to reflect on the lessons and paradoxes that have emerged from the transformation of the pre-clinical OSCE into an official examination, which may be of value to educators in Japan and abroad.

One significant change accompanying the pre-clinical OSCE was the standardization of the number of stations. Previously, each university determined this independently; however, there are now eight stations (Table 1): medical interviews, head and neck examinations, chest examinations, abdominal examinations, neurological examinations, general examination and vital signs, basic clinical procedures, and emergency medicine [2]. The Angoff method, which has been established as an internationally useful method, is used as the criterion for scoring the OSCE [4], and the pass criterion is set before the test is administered. Based on this method, the pass criterion for each station is set with the cooperation of university faculty members at medical schools nationwide, and the criteria are also reviewed periodically. Although there was a proposal to increase the number of stations from eight to 10, the current design of eight stations was retained owing to the present heavy burden of implementing the OSCE. There is tension between the opinion that the number of stations should be increased to 10, including limbs, spine, and infection control as soon as possible, considering the educational aspect, and the opinion that it is not possible to ensure that the operation of the standardized 10 stations is transferred to all medical universities nationwide.

Another important development was establishing a new certification system for evaluators to ensure fairness and impartiality. Each station now requires two or more assessors certified by the CATO. To gain certification, faculty members and task moderators must attend CATO training sessions, typically held several times a year in urban areas, and pass the certification exam within a limited period before the OSCE is conducted. Furthermore, the simulated patients who participate in the medical interviews must be certified by CATO. In the past, the training methods for simulated patients in medical interviews differed depending on the organization to which they belonged, but standardization was needed. Standardization initiatives to ensure the quality of assessors were considered essential for the standardized pre-clinical OSCE, but these initiatives also burdened faculty members and simulated patients.

For rural universities, where faculty numbers are limited, securing enough certified evaluators and attending training sessions posed significant challenges. Across the country, many universities struggled to meet the requirements. Moreover, with the officialization of the OSCE, universities were required to include external evaluators sent by CATO and standardized patients for medical interviews certified by CATO, further straining personnel and resources.

The situation regarding solutions to these problems remains chaotic. Additionally, there is a fear that without continued financial support, it will not be possible to maintain a strict testing system. In light of the above, consideration is currently being given to holding evaluator training workshops at each medical university in collaboration with the person responsible for this task at CATO. In addition, CATO committee members are holding discussions about the efficient training of evaluators through online training to reduce the burden of training evaluators and about the consolidation of the OSCE implementation period to reduce the burden on OSCE implementing organizations. My view is that in the future, we should consider flexible and varied responses based on each university’s circumstances. This also reflects broader difficulties in improving education and assessment while managing faculty workloads. Although no single solution exists to address these challenges, I believe continuous reflection and equitable, non-hierarchical dialogue between CATO and universities is crucial for progress. Ultimately, I hope these initiatives will lead to safe, high-quality medical care for patients. I believe that policymakers and central organizations need to thoroughly verify whether the financial and human resources required to implement the public recognition of examinations, including the OSCE, are sustainable with medical school educators in advance. I also believe that learning about Japan’s efforts to publicize its examinations in the medical education systems of other countries will provide an opportunity to reflect on or relativize their own examination systems.