Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone
January 2nd, 2020
19.6K views
Presented here is a method to design an augmented reality smartphone application for the visualization of anatomical three-dimensional models of patients using a 3D-printed reference marker.
Overview
This article presents a method for developing an augmented reality smartphone application that visualizes anatomical 3D models using a 3D-printed reference marker. The protocol is designed for users with no prior experience in medical imaging or software development.
Key Study Components
Area of Science
- Augmented Reality
- 3D Printing
- Medical Imaging
Background
- Augmented reality can enhance visualization in medical scenarios.
- 3D printing provides a tangible reference for anatomical models.
- This method is accessible to inexperienced users.
- Combining these technologies can improve patient understanding.
Purpose of Study
- To empower users to create their own augmented reality applications.
- To facilitate the visualization of patient anatomy.
- To simplify the integration of 3D models into medical practice.
Methods Used
- Utilization of 3D Slicer software for medical image segmentation.
- Exporting segmented anatomy into 3D model formats.
- Designing a smartphone app using Unity Engine and Vuforia.
- 3D printing of reference markers for augmented reality.
Main Results
- Successful creation of 3D models from medical images.
- Integration of augmented reality with 3D printed markers.
- Development of a user-friendly app for visualizing anatomy.
- Demonstrated potential for application in various medical scenarios.
Conclusions
- The method allows for effective visualization of anatomical structures.
- Augmented reality can enhance patient education and engagement.
- This approach is scalable and adaptable to different medical contexts.
Frequently Asked Questions
What is the main goal of the augmented reality application?
The main goal is to visualize anatomical 3D models to enhance understanding in medical scenarios.
Who can use this protocol?
It is designed for users with no prior knowledge of medical imaging or software development.
What software is used for creating 3D models?
3D Slicer software is used for medical image segmentation and model creation.
How are the 3D models integrated into the app?
The models are integrated using Unity Engine and Vuforia for augmented reality support.
What materials are recommended for 3D printing the markers?
A white colored material for the TwoColorCubeMarker and a black colored material for the supporting structure are recommended.
Can this method be applied to different medical scenarios?
Yes, it can be adapted to various medical scenarios where 3D printed references can be used.
Executive Industry Relevance
This protocol enables biopharma R&D teams to rapidly prototype patient-specific anatomical models for target validation and mechanistic studies using accessible AR and 3D printing tools. By lowering technical barriers, it supports early-stage hypothesis testing and reduces reliance on specialized engineering resources. The approach enhances predictive confidence in preclinical models by providing tangible, visualizable systems for pathway clarification and functional validation.
Strategic Applications in Biopharma R&D
Early Discovery & Target Validation
- Scientific Value: Supports therapeutic hypothesis interrogation through patient-derived 3D models that clarify anatomical context of disease targets.
- Operational Value: Enables rapid iteration of model designs using free software and standard 3D printers, accelerating target validation cycles.
- Predictive Value: Improves mechanistic de-risking by allowing visualization of target engagement in spatially accurate, patient-relevant systems.
Screening & Assay Development
- Scientific Value: Provides standardized, reproducible 3D-printed reference markers for consistent AR-based model positioning in screening workflows.
- Operational Value: Facilitates assay readiness by generating quantifiable visual outputs that can be correlated with compound effects.
- Scalability: Enables platform reuse across multiple targets and disease models through adaptable marker and model designs.
Translational & Preclinical Research
- Translational Continuity: Bridges discovery and preclinical stages by maintaining anatomical fidelity from patient imaging through to model validation.
- Risk-Adjusted Advancement: Supports go/no-go decisions by enabling visual assessment of target modulation in disease-relevant systems.
- Biomarker Alignment: Allows correlation of AR-visualized model changes with potential translational biomarkers during preclinical evaluation.
Pipeline & Workflow Integration
The method integrates into early discovery workflows by enabling rapid generation of patient-specific models for target validation, progressing to assay development and preclinical validation stages with maintained model fidelity.
- Discovery Biology: Enables hypothesis testing and pathway clarification through direct visualization of patient-derived anatomical models in AR.
- Screening: Supports assay standardization via reproducible 3D-printed markers that ensure consistent model positioning for compound screening.
- Analytics: Generates quantifiable spatial and visual readouts from AR visualization that help compare experimental conditions.
- Translational Research: Connects discovery to preclinical work by preserving patient-specific anatomy through all model iterations.
- Enterprise Reuse: Establishes a reproducible capability for generating AR-visualizable models across multiple projects and therapeutic areas.
Operational & Enterprise Impact
- Scientific Value: Increases predictive confidence in target validation by reducing ambiguity in anatomical context and target localization.
- Operational Value: Enhances reproducibility and scalability through standardized software workflows and accessible 3D printing.
- Strategic Value: Improves capital efficiency by enabling in-house model generation and reducing dependency on external visualization resources.
- Portfolio Impact: Supports risk-adjusted prioritization by providing clearer visual data for target validation and mechanistic studies.
Implementation Considerations
- Requires familiarity with 3D Slicer for segmentation and Unity/Vuforia for app development.
- Needs access to a dual-extruder 3D printer for high-quality marker fabrication.
- Demands cross-team standardization of file formats and model naming conventions.
- Involves adaptation considerations when applying to different anatomical regions or disease models.
- Limited by the size and complexity of models that can be effectively visualized via smartphone AR.
Biopharma FAQs
Why does null hypothesis testing matter for target validation in AR-visualized models?
Null hypothesis testing helps determine whether observed changes in AR-visualized patient models are statistically significant, supporting confident target validation decisions by distinguishing true biological effects from variability in model positioning or segmentation.
How does independent variable isolation fit the discovery pipeline when using AR and 3D printed models?
Isolating independent variables such as compound concentration or genetic modification allows researchers to attribute changes in AR-visualized model appearance to specific interventions, improving target validation rigor in early discovery.
What quantitative dependent variable measurements enable target validation in this AR-3D printing workflow?
Quantitative measurements such as model position, orientation, and scaling relative to the 3D-printed marker enable objective assessment of target engagement, supporting data-driven target validation in preclinical studies.
Why do replication requirements matter for cross-functional collaboration in AR-based model visualization?
Replication ensures that AR-visualized models can be consistently reproduced across teams and sites, which is essential for reliable cross-functional target validation and assay transfer in drug discovery programs.
What statistical analysis capabilities are required before implementing this AR and 3D printing method in target validation?
Basic statistical analysis capabilities are needed to quantify and compare AR-visualized model outputs across conditions, enabling researchers to assess the significance of observed changes in target validation studies.
Chapters in this video
0:04
Title
1:01
Biomodel Creation
2:08
Biomodel Positioning: Visualization Mode
2:53
Biomodel Positioning: Registration Mode
4:00
3D Printing and Augmented Reality (AR) App Deployment and Visualization
7:25
Results: Representative AR Visualization of 3D-Printed Markers
8:38
Conclusion
