Multimodal Cross-Device and Marker-Free Co-Registration of Preclinical Imaging Modalities

We developed software tools for preclinical imaging, including industry construction, fusion, segmentation, and quantification. Our software is meant to be user friendly, efficient, and compatible with any small animal imaging device to enable reproducible quantitative analysis of the acquired multimodal image data. Automation is an important topic in our field because manual analysis is often too time consuming, error prone, and hardly reproducible.

Software-based automation can make the analysis efficient and reproducible, but the challenge is in making the automation robust enough. Small animal imaging devices provide volumetric image data of anesthetized laboratory mice and rats, and allow non-invasive assessment of pathological and physiological changes. Integrated multimodal devices combine the strengths of multiple modalities, such as micro CT and PET.

It is also possible to combine two separate devices. We developed software for the automated fusion of special modalities, such as micro CT and FMT. We also provided efficient software for interactive image fusion and defined a new file format to store the image data considering aspects from data curation.

Last year we published an approach for retrospective respiratory gating and the high throughput settings. Our protocol results in a transformation matrix, which is generated using a calibration step that involves visible structures such as fiduciary markers. Then the image fusion can be automated using this transformation matrix, which eliminates the need for markers.

Begin by loading the images of both modalities. To do so, open the analysis software. To load the CT Image A as underlay, go to File, followed by Underlay, and click Load Underlay.

In the dialogue that appears, choose and open the image file. Similarly, to load the PET Image A as an overlay click File, Overlay, Load Overlay, and choose the image file to open it. For marker based fusion, hide the underlay by navigating to Menu, View, and Layer Visibility.

Then uncheck overlay and keep underlay. Switch the selected layer to the underlay if required. To show multiple slice views, press alt plus A for the axial, alt plus S for the sagittal, and alt plus C for the coronal views.

If necessary, press CTRL plus W and adjust the left and right vertical bars in the windowing dialogue to better see the fiducials. Click on Okay to close the dialogue. Click the marker symbol on the vertical toolbar on the left side to activate the mouse action mode, Create Marker.

Navigate to one fiducial of the phantom. Place the mouse pointer in its center and left click. Select Okay To save the marker, place the mouse pointer over the view of a plane, and use alt plus mouse wheel to slice through the planes.

Adjust the viewing settings by navigating to Menu, View, Layer Visibility, and check Overlay to show the overlay. Switch the selected layer to overlay. Adjust the windowing as demonstrated if necessary.

Navigate to each fiducial of the phantom. Place the mouse pointer in the center of the fiducial and left click. Click on Okay to save the marker.

Place the mouse pointer over the view of a plane and use alt plus mouse wheel to slice through the planes. Align the markers of underlay and overlay by navigating to Fusion, followed by Register Overlay to Underlay, and selecting Compute Rotation and Translation. Note this measurement and click Okay.

Check the result of the alignment. As demonstrated previously, activate the views of underlay and overlay layers. Click on the symbol on the left side vertical toolbar to activate the mouse mode Interactive Image Fusion.

Check overlay, uncheck underlay and segmentation. Perform translation by placing the mouse pointer near the center of a view. Hold the left mouse button and move the mouse to move the overlay.

Perform rotation by placing the pointer near the edge of a view, axial, coronal, or sagittal. Hold the left mouse button and move the mouse to rotate the overlay. Keep performing rotations and translations on the different views until the underlay and overlay are aligned as best as possible.

Create and save the differential transformation by navigating Fusion, Overlay Transformation, and select Create and Save Differential Transformation. In the first dialogue, select the original overlay file and click on Open. In the second dialogue, enter a file name for the differential transformation and press Save.

To begin, launch the analysis software. Import the CT file as an underlay by clicking on File, Underlay, and Load Underlay. In the ensuing dialogue, select the CT image file to open it.

Load the PET file as an overlay by sequentially selecting File, Overlay, and Load Overlay. In the dialogue that appears, choose the PET image file. To enable the display of both layers, go to Menu, View, followed by Layer Visibility, and check both Underlay and Overlay.

To load and apply the previously saved differential transformation matrix, click on Fusion, Overlay Transformation, and then Load and Apply Transform. Then select the file containing the matrix from the calibration process to open it.