Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
August 12th, 2021
The human depth perception of 3D stereo videos depends on the camera separation, point of convergence, distance to, and familiarity of the object. This paper presents a robotized method for rapid and reliable test data collection during live open-heart surgery to determine the ideal camera configuration.
It enables capturing of high-quality 3D videos for training and education. It can be applied to most operating room settings with open surgery. With a robotized approach, we can quickly and accurately collect the data to evaluate the camera positions.
This method is used to rapidly test camera configuration for 3D visions applications. It can also be used to investigate how different subjects perceive depth. To begin mounting the camera to the robot, first, mount the lens to the camera, and then the camera to the adapter plate with three M2 screws.
Next, mount the circular mounting plate to the camera adapter plate with four M4 screws to the opposite side of the camera. Once all the cameras are mounted to the robot, observe the mirroring of the resulting assemblies. Mount the adapter plate to the robot wrist with four M2.5 screws.
For a table-mounted robot, attach the left camera to the right robot arm. Connect the USB cables to the cameras and the Ubuntu computer. On the robot touch display, press the Menu button and select Stereo 2 to start the robot application.
On the main screen, press Go to start for 1, 100 millimeters in the robot application, and wait for the robot to move to the start position. Remove the protective lens cap from the cameras. Place a printed calibration grid at 1, 100 millimeters from the camera sensors.
To correctly identify the corresponding squares, place a small screw nut or mark somewhere in enter of the grid. Start the recording application on the Ubuntu computer to start the interface. And then, adjust the aperture and focus on the lens with the aperture and focus rings.
In the recording application, check Crosshair to visualize the crosshairs. In the recording application, ensure that the crosshairs align with the calibration grid in the same position in both camera images. After pressing the gear icon in the robot application, position the cameras relative to the patient.
Change the X direction by pressing the plus or minus for Hand distance from robot, and Z direction by pressing the plus or minus for Height to capture the surgical field in the images. Change the Y direction by manually moving the robot or patient. After pausing the surgery and informing the OR personnel about starting the experiment, press Record in the recording application, and then press Run Experiment in the robot application.
After a while, when the experiment is finished, Done will appear on the touch display. Then, press Stop Recording in the recording application to stop recording. Inform the OR personnel that the experiment has finished and resume the surgery.
For evaluation, display the video in top-bottom 3D format with an active 3D projector. The correct evaluation video with a right image placed at the top in top-bottom stereoscopic 3D is shown. A successful sequence should be sharp, focused, and without unsynchronized image frames.
The unsynchronized video streams create blur. The convergence point should be centered horizontally, independent of the camera separation. With a very large separation between the right and left images, the brain cannot fuse the images into one 3D image.
Several reasons can cause failure in the center positioning of the heart, such as the distance from the convergence point to the heart. For correct configuration of the camera tool coordinate system, ensure that the crosshairs in the recording application point at the same object, the calibration object is in the center position, and the correct debayering format of colors. 3D videos with changing baseline distances enable us to study how depth information affects anatomical understanding among medical staff.
View the full transcript and gain access to thousands of scientific videos
Overview
This study presents a robotized method for collecting test data during live open-heart surgery, focusing on the ideal camera configuration for 3D stereo videos. The method aims to enhance depth perception evaluation in surgical settings.
Key Study Components
Area of Science
- Neuroscience
- Biomedical Engineering
- Surgical Techniques
Background
- Depth perception in 3D videos is influenced by various factors.
- Camera configuration plays a critical role in visual clarity.
- Understanding depth perception can improve surgical training.
- Robotized approaches can streamline data collection in operating rooms.
Purpose of Study
- To evaluate the ideal camera positions for 3D vision applications.
- To investigate how different subjects perceive depth in surgical contexts.
- To enhance the quality of training and education through high-quality 3D video capture.
Methods Used
- Robotized camera mounting for rapid data collection.
- Evaluation of camera separation and point of convergence.
- Testing in various operating room settings.
- Collection of data during live open-heart surgeries.
Main Results
- High-quality 3D videos can be captured effectively.
- Camera configurations significantly affect depth perception.
- Rapid testing methods yield reliable data for analysis.
- Familiarity with objects influences depth perception outcomes.
Conclusions
- The robotized method is effective for camera configuration testing.
- Understanding depth perception can enhance surgical training.
- Future applications may extend to various surgical procedures.
Frequently Asked Questions
Executive Industry Relevance
This method enables rapid, repeatable evaluation of camera configurations for 3D surgical visualization, supporting improved depth perception in medical training. By minimizing OR interference and standardizing data collection, it enhances the reliability of stereoscopic video for surgical education and device evaluation. The approach addresses a key challenge in translating 3D imaging benefits into consistent, scalable training tools.
Strategic Applications in Biopharma R&D
Early Discovery & Target Validation
- Scientific Value: Enables objective assessment of visual fidelity in 3D surgical models, supporting hypothesis testing around spatial anatomy comprehension.
- Operational Value: Provides standardized, robotized data capture to reduce variability in visual model evaluation across studies.
Screening & Assay Development
- Scientific Value: Generates quantitative video datasets at defined baseline and convergence distances for comparative analysis of 3D rendering quality.
- Operational Value: Supports assay-like screening of camera setups using repeatable positional steps and controlled convergence points.
Translational & Preclinical Research
- Scientific Value: Facilitates evaluation of 3D video realism in open-heart surgery models, aiding translational assessment of training tool effectiveness.
- Operational Value: Enables cross-functional teams to benchmark visual systems using standardized video outputs from controlled camera configurations.
Pipeline & Workflow Integration
This method fits within early discovery workflows where visual model fidelity impacts biological interpretation, particularly in procedure-based training and device-tissue interaction studies.
- Discovery Biology: Supports hypothesis testing regarding depth perception and spatial understanding in surgical training models.
- Screening: Enables rapid, reproducible evaluation of camera parameters to identify optimal configurations for 3D video capture.
- Analytics: Produces time-synchronized video samples at incremental baseline distances for quantitative comparison of stereoscopic quality.
- Translational Research: Connects camera configuration testing to the realism of surgical video used in preclinical training and device evaluation.
- Enterprise Reuse: Establishes a reusable robotic platform for standardized visual system validation across multiple surgical models and training scenarios.
Operational & Enterprise Impact
- Scientific Value: Improves predictive confidence in 3D training tools by enabling objective, repeatable assessment of visual fidelity.
- Operational Value: Standardizes camera positioning and data capture, reducing setup time and inter-operator variability in OR environments.
- Strategic Value: Supports faster go/no-go decisions on 3D visualization investments by de-risking visual performance early in development.
- Portfolio Impact: Enables risk-adjusted prioritization of training technologies based on validated 3D video quality and depth perception outcomes.
Implementation Considerations
- Requires expertise in robotic systems, camera calibration, and 3D video synchronization.
- Needs dual-arm robotic arm, stereo cameras, calibration grid, and synchronized recording software.
- Demands coordination with OR staff for timed pauses during surgery to capture video samples.
- Must account for variations in surgical field anatomy and lighting across different procedures.
- Limited to open-surgery settings where robot mounting and line-of-sight are feasible.
