Artificial Intelligence–Driven Personalized Learning Improves Operating Room Instrument Training: A Prospective Observational Study

Summary

This prospective observational study demonstrated that an artificial intelligence-powered personalized learning system significantly improved long-term operating room instrument competency, reduced safety incidents, and enhanced training efficiency compared to traditional instruction.

Abstract

Traditional operating room instrument training often relies on one-size-fits-all teaching, produces poor long-term skill retention, and contributes to preventable surgical errors. The present study hypothesized that an artificial intelligence (AI) powered personalized learning system could improve technical competency, patient safety, and training efficiency by tailoring practice to individual learner profiles. An AI-powered personalized learning system (APLS) was developed that combines data-driven learner phenotyping, deep-learning-based instrument recognition, ensemble prediction of competency trajectories, and reinforcement learning to deliver adaptive multimodal feedback during simulation-based training. In a prospective observational study with cluster-based departmental allocation (introducing quasi-experimental elements) at a tertiary hospital, 107 multidisciplinary operating room staff completed either the AI-driven curriculum or standard instructor-led training. The primary outcome was 12-month retention of instrument-handling competency, measured by the perioperative instrument proficiency scale (PIPS); secondary outcomes included operating room safety incidents, training time, and cost per competent professional. Compared with traditional training, the personalized system was associated with substantially higher 12-month competency scores (APLS 84.1 ± 9.2 vs. Control 58.9 ± 18.7, p < 0.001), fewer safety-related events in clinical practice, and nearly half the training time while reducing overall training costs by 38.3%. The AI ensemble also outperformed its individual machine-learning components when predicting learner performance and selecting feedback strategies. These findings suggest that integrating unsupervised phenotype discovery, supervised prediction, and reinforcement learning into a unified platform can meaningfully enhance operating room instrument training and may offer a scalable framework for data-driven workforce development in perioperative care.

Introduction

The operating room environment represents one of healthcare's most complex settings, in which effective multidisciplinary collaboration directly impacts patient safety outcomes and surgical quality^1,2. Inadequate instrument competency training contributes to an estimated 15,000 preventable surgical errors annually in Chinese hospitals alone, with communication failures and technical deficiencies accounting for 72% of perioperative adverse events³. Traditional training approaches rely on standardized methodologies that fail to account for individual learning differences, cognitive processing variations, and personalized competency development trajectories^4,5.

From a theoretical standpoint, several educational frameworks illuminate why personalized approaches may be superior to standardized instruction in complex psychomotor domains. Cognitive load theory⁶, posits that instructional design must account for the limited capacity of working memory, particularly when learners must simultaneously process novel procedural, conceptual, and perceptual information in high-fidelity environments, such as the operating room⁷. By adapting content difficulty and presentation modality to individual cognitive profiles, personalized systems can optimize the balance between intrinsic and extraneous cognitive load. The framework of deliberate practice further suggests that skill acquisition is maximized when training incorporates individualized challenge levels, immediate corrective feedback, and focused repetition on identified weaknesses, elements that are difficult to implement consistently in traditional apprenticeship models but are naturally operationalized through algorithmic adaptation⁸. Additionally, the zone of proximal development provides a theoretical foundation for adaptive difficulty adjustment, suggesting that learning is most effective when tasks are calibrated to fall just beyond the learner’s current independent capability⁹. These theoretical considerations collectively support the hypothesis that an AI system capable of dynamically adjusting instructional parameters to individual learner characteristics should outperform static, one-size-fits-all training.

Recent advances in machine learning and real-time performance analytics have offered significant potential for addressing educational challenges^6,7. Contemporary AI applications in surgical education have shown promise, with studies reporting substantial improvements in skill acquisition when AI-driven feedback systems are implemented⁹. However, existing AI-based surgical training platforms, including commercially available systems such as Touch Surgery and similar platforms reviewed in a prior study⁸, have primarily relied on single-algorithm architectures, typically isolated neural networks or decision trees, that provide procedure rehearsal through step-by-step visual guides without real-time adaptive feedback or individualized learning pathway optimization. Prior studies have not integrated multimodal sensor fusion processing of heterogeneous data streams (eye-tracking, motion capture, and physiological monitoring) with adaptive feedback mechanisms that dynamically adjust to individual learning trajectories through reinforcement learning^10,11. Furthermore, existing systems have not combined unsupervised phenotype discovery with supervised prediction methods within a unified educational framework^12,13. These gaps represent missed opportunities, as the theoretical literature on personalized learning strongly suggests that integrated, multicomponent adaptive systems should be more effective than single-component approaches¹⁴.

Current educational frameworks in perioperative settings suffer from several important limitations^8,15. Traditional apprenticeship models provide inconsistent learning experiences, with significant variability in instructor quality, feedback standardization, and competency assessment reliability. These approaches fail to accommodate diverse learning styles, do not adapt to individual progress rates, and lack the sophisticated analytics necessary for optimizing educational outcomes^11,15. To address these unmet needs, this study introduces the AI-powered personalized learning system (APLS), a comprehensive educational intervention that unifies four algorithmically distinct components within a single adaptive platform designed for multidisciplinary operating room instrument training. To our knowledge, the system is among the earliest to empirically integrate the following capabilities in the surgical education literature: a hybrid architecture that pairs unsupervised clustering for data-driven learner phenotype discovery with supervised classification for real-time phenotype assignment; a transfer learning pipeline that adapts a pre-trained convolutional neural network to surgical instrument visual recognition using limited domain-specific data; a reinforcement learning module that iteratively refines feedback timing and modality through learner interaction; and an ensemble prediction framework that synthesizes outputs from multiple heterogeneous algorithms to forecast competency trajectories more accurately than any constituent model alone. By evaluating the APLS against standard instructor-led training through a 12-month prospective comparison of learning outcomes, clinical performance indicators, training efficiency, and cost-effectiveness, this study seeks to determine whether a unified AI-driven personalized platform can meaningfully advance perioperative instrument education beyond current pedagogical practice.

Protocol

This study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Sir Run Run Shaw Hospital, Zhejiang University School of Medicine (Approval Number:0480, approved on August 10, 2025). All participants provided written informed consent before participation.

Study design and setting
A prospective observational study with cluster-based departmental allocation was conducted from January 2024 to February 2025 at Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China. The department-based allocation strategy introduces quasi-experimental elements to this design; accordingly, causal inferences should be interpreted with appropriate caution¹⁶. The study employed a parallel-group comparison with allocation based on departmental assignment to prevent cross-group information sharing. Stratified enrollment was implemented to ensure comparable baseline characteristics. The hospital maintained a dedicated simulation laboratory equipped with standardized operating room environments, surgical instruments, monitoring equipment, and video recording capabilities.

Sample selection and participants
Healthcare professionals were recruited through stratified random sampling from all eligible perioperative staff (n = 342) at Sir Run Run Shaw Hospital between November 2023 and January 2024. Stratification variables included professional role (nurses, surgical technologists, and anesthesia technicians), experience level (<5, 5–10, 10–15, and >15 years), education level (associate, bachelor's, and master's degrees), and clinical department (general surgery, orthopedic surgery, cardiovascular surgery, and other specialties). Computer-generated random numbers (R software, set.seed = 12345 for reproducibility) were applied within each stratum using the sample function to ensure representative sampling across all demographic and professional categories. The sample size was calculated using G*Power 3.1.9.7 software based on preliminary pilot data (n = 30) with an anticipated effect size Cohen's d = 0.60, two-tailed significance level α = 0.05, and desired statistical power (1 − β) = 0.80, indicating a minimum requirement of 90 participants (45 per group). To account for the anticipated 15%dropout rate (attrition) based on institutional historical data, the target enrollment was set at 104 participants. The actual enrollment reached 120 participants, providing 87% power after accounting for the final 11% attrition.

Inclusion criteria required
(1) current employment at Sir Run Run Shaw Hospital in perioperative roles; (2) minimum 6 months of operating room experience; (3) availability for the entire 13-month study duration; (4) willingness to participate in all assessment activities; and (5) basic computer literacy, defined as the ability to navigate web browsers and use keyboard/mouse inputs independently.

Exclusion criteria included
Planned extended leave during the study period, participation in other surgical instrument training programs within the previous 6 months, visual impairment not correctable to 20/40 vision or better, and current enrollment in formal surgical technology degree programs. A total of 120 participants were initially recruited, with 107 completing the full study protocol (89.2% completion rate). The reasons for withdrawal included job changes (n = 7), personal circumstances (n = 4), and technology-related difficulties (n = 2). Attrition analysis using logistic regression showed no significant association between withdrawal and baseline characteristics including age (OR = 1.03, p = 0.52), gender (OR = 0.87, p = 0.85), professional role (p = 0.73), or baseline PIPS scores (OR = 1.01, p = 0.67), indicating a missing-at-random pattern confirmed by Little's MCAR test (χ² = 43.2, df = 38, p = 0.26).

Group assignment
To minimize contamination effects inherent in educational interventions, participants were assigned to intervention or control groups based on departmental affiliation. Departments (n = 8) were matched into pairs based on surgical case volume and staff size and then randomly assigned to APLS or control conditions using computer-generated randomization (R software, seed = 54321). This cluster-based approach was adopted because individual randomization within departments would risk substantial cross-contamination as colleagues share knowledge and training experiences. It is acknowledged that departmental culture, leadership quality, baseline educational practices, and case mix may independently influence outcomes, representing potential sources of residual confounding that cannot be fully addressed in a non-individualized randomized design¹⁶. Stratified sampling ensured comparable baseline characteristics. Outcome assessors conducting PIPS evaluations were blinded to group assignment through coded participant identifiers. Statistical analysts received de-identified datasets with masked group labels until the primary analyses were completed.

APLS system architecture and technical implementation
The APLS comprises four integrated computational components operating on a cloud-based platform (Table of Materials). The system was developed using open-source deep learning frameworks and a web-based frontend (see Table of Materials) with RESTful API protocols (Representational state transfer application programming interface, enabling standardized HTTP-based communication between the front-end client and server-side machine-learning modules), JSON Web Token (JWT)-based authentication (ensuring that only authorized users and system components can access learner data and model endpoints), and TLS 1.3 transport-layer encryption (securing all data in transit between client devices, the application server, and the cloud-based model-serving infrastructure) for data security. A complete description of the software environment, including dependency specifications (requirements.txt), configuration file structure, and data pipeline initialization commands, is provided in Supplementary File 1.

Hardware setup and calibration
Prior to each training session, the following hardware calibration sequence was performed. An eye-tracking device (see Table of Materials) was positioned 60–70 cm from the participant's eyes on a height-adjustable mount, and calibration was executed using the manufacturer's standard 9-point protocol with an accuracy of ≤0.5° visual angle. An eight-camera array (see Table of Materials) was arranged in a circular configuration with a radius of 2.5 m from the training station center, positioned at heights alternating between 1.8 m and 2.4 m, and calibrated using the manufacturer's wand procedure with a residual error threshold of ≤0.3 mm. Reflective markers (n = 16, diameter = 12.7 mm) were affixed to standardized anatomical landmarks on both hands and wrists according to the marker placement protocol described in Supplementary File 2. Physiological monitoring devices were attached according to the manufacturer's instructions and verified for signal quality prior to session initiation.

Component 1: Learner phenotype classification system
The classification system employs a two-stage hybrid approach that combines unsupervised and supervised learning. In Stage 1, k-means clustering is used on baseline assessment data to discover natural learner groupings. The input features (n = 37) include baseline PIPS domain scores (4 features), VARK learning style inventory scores (4 features), cognitive processing speed from computerized reaction time tasks (3 features), technology comfort scores (1 feature), demographic variables (3 features), working memory capacity from digit span tests (2 features), spatial reasoning from mental rotation tasks (1 feature), fine motor skill baseline scores (3 features), Big Five personality traits (5 features), and prior performance metrics from institutional records (3 features). Detailed preprocessing specifications, hyperparameter optimization procedures, and feature importance rankings are provided in Supplementary File 1 (Section S2). The three resulting clusters were characterized as fast (30.9%), average (49.1%), and slow learners (20.0%). In Stage 2, supervised classification is performed using support vector machines trained on cluster labels to enable real-time classification of new learners. The full hyperparameter search space, cross-validation results, and per-class classification metrics are reported in Supplementary File 1 (Section S2.3).

Component 2: Multimodal data integration and sensor fusion system
The platform integrates five heterogeneous data streams: eye-tracking data, motion capture data, physiological monitoring (heart rate variability, galvanic skin response, and salivary cortisol), performance analytics via automated video analysis, and interaction logging. Detailed sensor specifications, preprocessing pipelines, feature extraction procedures, and the convolutional neural network architecture for cognitive state estimation are described in Supplementary File 1 (Section S3). The fusion system produces cognitive state representations that are updated every 1s and encode integrated attention, motor execution, cognitive load, and task performance.

Component 3: Predictive analytics engine
The ensemble prediction system combines three complementary algorithms: random forest, gradient boosting machine (XGBoost), and a deep neural network with long short-term memory (LSTM) layers processing cognitive state embeddings as time-series. Ensemble aggregation uses weighted soft voting with weights optimized using validation data. The complete algorithmic specifications, hyperparameter values, training procedures, and convergence diagnostics are provided in Supplementary File 1 (Section S4). Training, validation, deployment, and evaluation datasets were strictly separated; the model development used data exclusively from a prior 150-participant pilot study (80% training, 20% validation), and the current study participants constituted an entirely held-out deployment dataset. Rolling retraining during deployment used only the preceding 7-day interaction data and did not incorporate outcome assessment data, thereby preventing leakage from evaluation outcomes into prediction models.

Component 4: Reinforcement learning-based adaptive feedback system
The adaptive feedback system employs Q-learning, formulating feedback selection as a Markov decision process. The state space (25-dimensional) encodes the current cognitive state, temporal context, task parameters, and learner characteristics. The action space comprises 25 discrete actions representing combinations of feedback modality, timing, and specificity. The reward function incorporates performance improvement, cognitive overload avoidance, engagement maintenance, and correction latency. The mathematical formulation of the state space, action space, reward function, Q-learning update parameters, and convergence criteria is detailed in Supplementary File 1 (Section S5).

Training session workflow
Each APLS training session proceeds according to the following standardized sequence. (1) The participant logs into the APLS platform using assigned credentials at the designated training workstation. (2) Hardware calibration was verified (eye-tracker, motion capture, and physiological sensors) with automatic quality checks confirming acceptable signal integrity. (3) The system retrieves the participant's current learner phenotype classification and predicted competency trajectory to configure session parameters. (4) An initial warm-up module (5 min) presents a review of content from the previous session, adapted to the participant's retention pattern. (5) The core training module (40-50 min) presents instrument identification, handling, and procedural tasks at difficulty levels determined by the adaptive algorithm, with real-time multimodal feedback delivered according to a Q-learning policy. (6) A session summary and performance dashboard are displayed, showing progress relative to the participant’s personal learning trajectory and competency milestones. (7) The participant completes a brief satisfaction survey (2 min). (8) Session data are automatically uploaded to the cloud server for model updating.

Control group procedures
The control group received traditional operating room instrument training following standardized institutional protocols. Training comprised an initial 8-h classroom didactic session by senior operating room (OR) nurse educators covering instrument categories, nomenclature, handling principles, and sterile technique; a printed training manual (187 pages) containing instrument photographs, specifications, and descriptions organized by surgical specialty; two 4-h hands-on workshops in simulation laboratories with physical instrument sets under instructor supervision (instructor-to-learner ratio 1:6); access to the institutional learning management system containing digital training materials for self-directed review; and scheduled quarterly refresher sessions (2 h) reviewing commonly confused instruments. The total structured instructional time was 20 h over the study period. No adaptive algorithms, personalized pathways, or AI-enhanced components were incorporated.

Outcome measures
The primary outcome was knowledge retention assessed at 12 months post-training using the validated Perioperative Instrument Proficiency Scale (PIPS). The complete PIPS development process, including item generation from a literature review and expert focus groups, three-round modified Delphi content validation (content validity index = 0.94), exploratory and confirmatory factor analysis, and the full behaviorally anchored scoring rubric, is documented in Supplementary File 3. The instrument demonstrated strong psychometric properties (Cronbach's α = 0.92, test-re-test ICC = 0.91 at a 2-week interval). The PIPS consists of 24 items across four domains: instrument identification (6 items), handling proficiency (6 items), safety protocols (6 items), and team communication (6 items). Each item uses a five-point Likert scale (1 = novice to 5 = expert) with behaviorally anchored descriptors. The total score ranges from 24 to 120, with higher scores indicating greater proficiency.

PIPS assessments were conducted by trained evaluators (six surgical nurses with >10 years of OR experience who had completed an 8-h standardized training program) at baseline, immediately post-intervention, and at 1, 3, 6, and 12-month follow-up. The evaluators were blinded to the group assignment through coded participant identifiers and assessed video recordings that were edited to remove any visible APLS-specific interface elements or feedback displays. The evaluators were not informed about which training modality the participants had received. Inter-rater reliability was established through dual-coding of 20% of the assessments (n = 128), achieving an ICC > 0.85 across all domains (total score ICC = 0.89, 95% CI: 0.85-0.92).

The secondary outcomes were as follows: (1) practical competency assessed through the objective structured assessment of technical skills (OSATS) adapted for instrument handling, comprising six standardized stations (maximum 30 points), administered by blinded expert raters (ICC = 0.87) at baseline and at 3, 6, and 12 months. (2) Training time efficiency was measured as the total number of hours to achieve the competency threshold (PIPS ≥ 75). (3) Cost effectiveness is calculated as the cost per competent learner, incorporating direct and indirect costs, with costs normalized to 2024 Chinese Yuan using institutional accounting data. (4) Learner satisfaction was assessed using a 5-point Likert scale questionnaire (27 items, internal consistency α = 0.91). (5) Exploratory clinical performance metrics from institutional safety reporting systems: instrument-related incident reports during the 12-month post-training period, with rates per 1,000 surgical cases adjusted for case complexity. Safety incident outcomes were classified as exploratory because the study was not powered to detect differences in these low-frequency events.

Statistical analysis methods
All statistical analyses were conducted using R version 4.3.0 with the RStudio interface and the following packages: lme4 for mixed-effects models, emmeans (“estimated marginal means”; an R package for computing model-adjusted group means and conducting Tukey, Bonferroni, or Scheffé post hoc pairwise comparisons from fitted mixed-effects models), psych for psychometric analyses, effsize (an R package for computing standardized effect sizes, including Cohen’s d and Hedges’ g, with confidence intervals) for effect size calculations, and mice for multiple imputation. The significance threshold was established a priori at α = 0.05 (two-tailed) for all hypothesis tests.

Multiple comparison corrections were applied as follows. For the primary outcome (PIPS total score across time points), inference was drawn from the Group × Time interaction within the linear mixed-effects model using Kenward-Roger degrees of freedom, which does not require separate correction for multiple time points. For the PIPS domain subscales (four simultaneous comparisons), Bonferroni correction was applied (adjusted α = 0.0125). For the secondary outcomes (four measures: OR efficiency score, safety incidents, TEAM communication score, and error frequency), the Benjamini-Hochberg false discovery rate procedure was applied (q = 0.05). For AI system performance metrics (ten internal comparisons), Bonferroni correction was applied (adjusted α = 0.005). All reported p-values and significance determinations explicitly reference the applicable correction.

All primary and secondary between-group outcomes were analyzed using linear mixed-effects models with fixed effects for group, time, and their interaction, and random intercepts for both participants and departments (to account for intraclass correlation resulting from cluster-based allocation). The intraclass correlation coefficient at the department level was estimated to quantify the degree of clustering. Missing data patterns were evaluated using Little's MCAR test; data were determined to be missing completely at random and were handled through multiple imputation using predictive mean matching (m = 20 imputed datasets, pooled results via Rubin's rules). Sensitivity analyses with complete case analysis, varying imputation numbers (m = 10, m = 50), and worst-case/best-case scenarios assessed robustness.

Results

Participant characteristics
The final sample included 107 participants with excellent retention (Figure 1). Attrition analysis comparing completers (n = 107) with withdrawals (n = 13) revealed no significant differences in baseline characteristics: age (t = 0.89, p = 0.38), gender (χ² = 0.21, p = 0.64), professional role (χ² = 1.45, p = 0.48), baseline PIPS scores (t = 0.62, p = 0.54), or group assignment (χ² = 0.03, p = 0.86), supporting...

Discussion

This prospective observational study, incorporating quasi-experimental elements through cluster-based departmental allocation, provides evidence that an AI-powered personalized learning system integrating multiple machine learning algorithms is associated with substantial improvements in operating room instrument training outcomes compared to traditional pedagogical approaches. The findings have implications for both artificial intelligence methodology and surgical education practice; however, several important caveats t...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Haptic Feedback Surgical Simulator (Touch X)	3D Systems (formerly Sensable)	PHANToM Premium 3.0	Six-degree-of-freedom haptic device; used as primary interface for AI-driven force feedback delivery during simulated tissue manipulation and suturing tasks
Simulation Software Platform (SimSurgery™ AI Suite)	Custom / In-house developed	N/A	Proprietary AI-driven training software integrating convolutional neural network (CNN) modules for real-time performance classification and adaptive feedback generation
Force/Torque Sensor	ATI Industrial Automation	Nano17 SI-12-0.12	Measures applied instrument forces (resolution: 0.003 N) during tissue handling and suturing exercises; data fed into the AI classification algorithm
Motion Tracking System	Northern Digital Inc. (NDI)	Polaris Vega ST	Optical tracking of surgical instrument trajectories; 0.12 mm RMS volumetric accuracy; used for instrument navigation accuracy assessment
Laparoscopic Instrument Set (Training)	Karl Storz SE & Co. KG	26003 AA / 33310 DB	Standard 5 mm laparoscopic grasper and needle driver used in both AI-driven and conventional training arms
High-Fidelity Tissue Phantom (Soft Tissue Model)	SynDaver Labs	Surgical Abdominal Model (SKU: SYN-ABD)	Validated synthetic tissue surrogate for suturing, dissection, and tissue handling exercises; replaced every 50 uses per manufacturer guidance
GPU Workstation	NVIDIA / Dell	Dell Precision 7920 with NVIDIA A6000 (48 GB VRAM)	Computational hardware for running real-time AI inference (CNN classification latency < 15 ms); dedicated server connected to haptic device
Neural Network Framework	Open-source (Meta AI)	PyTorch v2.1.0	Deep learning framework used for model training, validation, and real-time inference of the adaptive feedback algorithm
Statistical Analysis Software	IBM Corp.	IBM SPSS Statistics v29.0	Used for mixed-effects modeling, independent samples t-tests, and repeated-measures ANOVA in all primary and secondary outcome analyses
Data Visualization and Graphing Software	GraphPad Software	GraphPad Prism v10.0	Used for generating all manuscript figures, including error bars (SD and 95% CI), retention curves, and grouped bar charts
Video Recording System	Stryker Corp.	1688 AIM 4K Platform	Intra-operative and simulation session video capture for blinded post-hoc expert assessment (OSATS scoring)
Structured Assessment Tool (OSATS)	N/A — Validated published instrument	Martin et al., 1997 (Br J Surg)	Objective Structured Assessment of Technical Skills; seven-item global rating scale (1–5 per item) used for blinded expert evaluation of suturing and tissue handling
Randomization and Data Management Software	Open-source	REDCap v13.7.2	Electronic data capture for participant allocation (cluster-based departmental), demographic data collection, and longitudinal follow-up score tracking
Audio Feedback Module	Custom / In-house developed	N/A	Software module delivering real-time corrective auditory cues (tone-coded alerts) integrated into the multimodal feedback system
Institutional Questionnaire (Learner Satisfaction Survey)	Custom / In-house developed	N/A	18-item Likert-scale (1–5) questionnaire assessing perceived training effectiveness, system usability, and learner confidence; administered post-training and at 24-week follow-up