Protocol for Producing Three-Dimensional Infrared Video of Freezing in Plants

Summary

Here, we present a protocol to image a strawberry plant freezing in 3 dimensions. Two infrared cameras positioned at slightly different angles are used to produce a red-blue anaglyph video to observe the freezing of the plant in 3 dimensions.

Abstract

Freezing in plants can be monitored using infrared (IR) thermography, because when water freezes, it gives off heat. However, problems with color contrast make 2-dimensions (2D) infrared images somewhat difficult to interpret. Viewing an IR image or the video of plants freezing in 3 dimensions (3D) would allow a more accurate identification of sites for ice nucleation as well as the progression of freezing. In this paper, we demonstrate a relatively simple means to produce a 3D infrared video of a strawberry plant freezing. Strawberry is an economically important crop that is subjected to unexpected spring freeze events in many areas of the world. An accurate understanding of the freezing in strawberry will provide both breeders and growers with more economical ways to prevent any damage to plants during freezing conditions.

The technique involves a positioning of two IR cameras at slightly different angles to film the strawberry freezing. The two video streams will be precisely synchronized using a screen capture software that records both cameras simultaneously. The recordings will then be imported into the imaging software and processed using an anaglyph technique. Using red-blue glasses, the 3D video will make it easier to determine the precise site of ice nucleation on leaf surfaces.

Introduction

Despite living in a world of three physical dimensions, researchers are often limited to reporting visual observations in 2D. Although 2D images are generally sufficient to convey important information, this lack of information about depth restricts our ability to perceive and understand the complexity of real-world objects.¹

This deficiency in information about the depth provided an incentive to produce 3D videos mainly in the commercial film industry since the early 1900s¹. However, generating clear 3D information in still images and video is hindered by the complexities involved in producing those images. The simplest approach to generating 3D film is based on principles used in stereoscopic photography. Stereoscopic photography utilizes two images of the same object from slightly different angles that conveys a 3D image in the brain.To make this possible, each eye must look only at its respective image (i.e., the left eye at the left image and the right eye at the right image). Since the eyes will not naturally do this, stereoscopic headgear was designed to make this possible¹. Several stereoscopic viewing techniques, as well as polarization-interlaced, time-multiplexed, and head-mount display techniques, have been used during the development of 3D films, but the color-interlacing or anaglyph method using red and green (or cyan) glasses is one of the simplest and least expensive techniques. For a comprehensive review of 3D imaging and the various techniques involved, see the review by Geng¹.

Monitoring freezing in plants using IR thermography is based on the principle that when water freezes, it must give up internal energy². This energy is in the form of heat, which is detectable in the IR region of the electromagnetic spectrum. Cameras able to record the IR energy have been in use since 1929³. The first published report using IR technology to film freezing in plants is from Cecardi et al.², but the resolution of the camera used makes it difficult to accurately determine the tissue where the freezing is initiated. Wisniewski et al.⁴ determined more precise sites of ice nucleation in several plant species using a higher resolution camera. As the technology used in IR thermography improved, higher resolution images led to discoveries such as barriers to freezing⁵ and the precise cellular localization of ice formation⁶.

One difficulty in filming subjects in IR is caused by small differences in temperatures. This will cause most objects in the field of view to be a similar color, making it difficult to determine precisely which object(s) is/are freezing. This can be important when determining the order of freezing in specific tissues, such as leaves or roots in wheat⁶. If the IR video of plants freezing could be imaged in 3D, the accuracy of determining which part of the plant is freezing at a certain point in time could be improved.

Strawberry is a crop in certain areas of the United States in which freezing temperatures are of considerable concern for the growers. Under some growing conditions, it is common for strawberry flowers to appear 2 - 3 weeks before the average last spring freeze. A freeze event can occur as late as June in some areas of the Appalachian Mountains⁷ and usually results in the death of the flower. Frost protection is, therefore, critical for strawberry growers in areas subject to these freeze events. Strawberry growers in North Carolina, for example, must frost-protect, on average, between 4 - 6 frost events before bloom and 1 - 2 hard freezes during the early bloom period⁸. To help develop strawberry genotypes that are more freezing tolerant, it is important to understand various aspects of the freezing, such as the sites of ice nucleation and propagation into other parts of the plant. IR thermography provides an effective means to address these issues.

Here, we use strawberry to illustrate a technique for recording freezing events in 3D using the anaglyph method. Strawberry is well suited for this example because the leaves and flowers are widely distributed in the 3D space and can be difficult to differentiate when viewed in 2D infrared videos.

Protocol

1. Preparation

1. Gather equipment, materials, and software to record and process the video of plant freezing.
   1. Start a programmable freezer by setting the power switch to On, and set the temperature to 0 °C. Program the freezer to reach -8 °C at 1 °C/h.
2. Place one 6-weeks-old strawberry plant with 2 - 5 flowers that was grown in a 1 L container into the freezer.
3. Set up 2 IR cameras (e.g., FLIR T620 cameras) using fastening straps and a small block of wood to produce the correct convergence angle of the lenses.
   Note: The optimum distance to space the center of the lenses of the 2 cameras is generally considered to be the same as the distance between the eyes¹ or approximately 7 cm.
4. Mount both cameras to a 10 x 10 cm laboratory jack and position the jack in the freezer close enough to the plant to allow the image to be focused. Adjust the cameras vertically and horizontally so that the same part of the plant is visible from both cameras. Use the jack to position both cameras vertically in such a way that the 2 images contain the whole plant and a portion of the soil.
5. Connect the 2 cameras using a USB connector to the USB outlets on the computer.
6. Plug the 2 A/C outlets into both cameras to allow a continuous monitoring of the plants.

2. Computer and Software Setup for Recording

1. Open 2 windows (1 window for each camera) of the software by double-clicking the icon for the IR camera software 2x. Follow the instructions in the help menu to connect the left camera in the left window and the right camera in the right window.
   Note: The details for using the software can be accessed through the help menu. A monochromatic palette is best suited for this example due to the necessity of using red-blue tinting for the 3D rendering.
2. Open the screen capture software by double-clicking the icon for the program. Adjust the capture frame by clicking and dragging the frame so it includes both cameras to allow a screen capture of both cameras simultaneously.
   Note: Screen capturing the video stream from both cameras simultaneously is crucial because it allows a flawless synchronization of both left and right views.
3. Record the video in 3-h increments for an easier processing in the video processing software.
   Note: It is impossible to know exactly when the plant will freeze, so it is important to record for some time prior to the freeze event. The option to record in segments is a feature of this software, so it is recommended that this is set to record for 3 h. The software will automatically save the 3-h recording and then begin a new recording. The file for each 3-h recording will automatically be given a numerical sequence after the name. Each video file will be from 10 to 20 GB, so ensure that sufficient space is available on a hard drive for multiple files of this size.
4. Start the freezer program by selecting Run in the controller menu and begin the screen capture. Then press the Rec button on the window. Ensure that the outline showing the region of the screen being captured turns red.
5. Record the strawberry plant freezing down to -8 °C and hold the temperature of the freezer for 1 h.
6. Raise the temperature of the freezer at 2 °C/h until the freezer is at +2 °C. Stop the recording.
   Note: The total freezing time is 14 h.
7. Convert the file(s) of interest from .mp4 format to .mov using a file conversion software.
   Note: In this case, a single 3 h file containing 1 or more freeze events will be used.

3. Processing Video Using a Video Imaging Software

Note: Video imaging software will be used in this example. Tutorials on how to use the software are available online. This example will assume a basic knowledge of the software. The understanding of terms such as “composition”, “layer”, and “render queue”, as well as the various panels and how to manipulate them, is assumed.

1. Double-click anywhere inside the project panel to import the .mov file of interest into the imaging software and drag the file to the Composition icon at the bottom of the project panel. Then save the project to the same folder containing the original videos.
   Note: The video recorded will be visible in the preview pane.
2. Click on the Region of Interest icon along the bottom of the preview window and, using the cursor, outline only the recording from the left camera.
3. Drag the same .mov video to the Composition icon to create a second composition of the same video. Repeat step 3.3, but this time, use the cursor to select only the right camera.
4. Select Composition > Crop Comp to Region of Interest for the left view. Repeat this for the right view. Rename each composition to indicate which is left and right.
5. Highlight the left composition by clicking on it and, in the main menu at the top, selecting Composition > Add to Render Queue.
6. In the render queue, click on Output Module and make sure the video will be rendered as a video (e.g., a QuickTime video). Click on the Render Settings to reduce the resolution to allow a faster render. Click on Output to, name the video Strawberry Left, and save it to the same folder as the original recording and the project. Click Save, and then click the Render button at the top right side of the render panel.
7. Repeat step 3.6 for the Strawberry Right composition.
8. Double-click the project panel and import the Strawberry Left and Strawberry Right videos that were just rendered.
9. Highlight both videos and drag them to the composition icon at the bottom of the project panel. In the pop-up screen asking for the Still Duration, enter 3 with 5 zeros for a 3-h duration.
   Note: Both videos, precisely synchronized, will be in the project panel, but only the topmost video in the composition panel will be visible.
10. To view the other image, click on the little eyeball to turn off the layer. Press Control/W to allow a rotational control of the images in the preview panel using the cursor. Using the cursor and clicking the top layer on and off, adjust the rotational aspect of either the top or the bottom view to make sure both images are in the same rotational plane. Then adjust the X- and Y-plane directly within the 3D glasses subroutine.
11. Highlight the top layer of the composition panel and select Effect > Perspective > 3D Glasses from the menu at the top.
    Note: The parameters for the 3D Glasses Effect will pop up inside the control panel.
12. In the control panel, click the box to the right of “left view” — if the control panel is not separated from the project panel, click the control panel tab at the top of the project panel. List the 2 videos in the composition panel in a drop-down menu, highlight the video in the list for the “left view”. Repeat this step for “right view”.
13. In the box to the right of 3D View, select Red Blue LR.
14. Using red-blue glasses, inspect the view in the project panel. If the 3D view seems to be incorrect, try clicking Swap Left-Right. Adjust Scene convergence and Vertical alignment to eliminate any ghosting and eye strain.
15. When the 3D aspect of the video is acceptable, highlight the composition by clicking on it and select Composition > Add to render queue as was done in step 3.7. Render the video into the same folder as the other files in the project.
    Note: This file will be rather large. Once the file has been rendered, it can be re-rendered to a smaller file size using the video processing software.

Representative Results

Surprisingly, the IR video of the strawberry plant freezing (Supplemental Video 1) indicated that not all leaves/flowers froze at the same time. The leaves and flowers both froze individually at different temperatures, but the leaves froze earlier than the flowers and at a higher temperature. In addition, the freezing began in the leaves but not necessarily at the same position on each leaf. While these results have not been described previously in strawberry, similar results have been found in other plant species⁶. Once the leaves were frozen, the ice progressed down the petiole to the crown of the plant. When the freezer temperature became 1 or 2 degrees colder, the flowers froze beginning at the calyx and quickly spreading into the petals and receptacle (Figure 1). The receptacle remained a lighter color (warmer) longer than most other plant parts, suggesting a greater amount of water freezing.

When comparing the 2D infrared image with the 3D (wearing glasses), the 3D image makes it easier to precisely determine the order in which the leaves and the flowers froze (Figure 1). When viewing the video in 3D, it is also easier to determine the exact position on the leaves where the freezing began (Supplemental Video 1).

The survival results (not shown) indicated that despite the freezing, the leaves were not killed (not shown) by the freeze. The flowers that froze, on the other hand, died within 3 or 4 days.

A second video, this time of wheat roots (Supplemental Video 2), showed an interesting sequence of freezing. The base of these roots was submerged in growing medium consisting primarily of peat. Ice shavings were added prior to the freezing to ensure the roots would freeze. Freeze nucleation occurred at about -0.5 °C midway along a root on the right side. The freezing then progressed upwards to the crown of the plant causing the base of the outer leaves to freeze. The freezing then progressed downwards in the roots at the back of the plant. Note that, without the 3D perspective, it is nearly impossible to determine the order in which the specific roots froze (Figure 2).

When considering freezing in roots (Figure 2 and Supplemental Video 2), if only a 2D perspective was viewed, it would be nearly impossible to determine which root was freezing due to the lack of information about depth. The 3D perspective of this freeze event represents the event as it occurred in the real world and greatly improves the ability of the viewer to distinguish the sequence of freezing in individual roots.

Figure 1: A comparison of an image of strawberries in 2D to the same image in 3D. These images are freeze-frames from Supplemental Video 1 showing 2 leaves and a single flower of a strawberry plant freezing. (A) This panel shows the left view only, in 2D. (B) This panel shows the 3D anaglyph view. Red-blue glasses must be worn to see this image in true 3D. A comparison between the two panels illustrates the improvement in visual perception when the subject has been captured in 3D. Please click here to view a larger version of this figure.

Figure 2. A comparison of images of a root mass of wheat in 2D to the same image in 3D. These images are freeze-frames from Supplemental Video 2. Panels A and B show the root mass in 2D. (A) This is an image of the roots prior to freezing, while (B) this is about midway through the freezing event. Panels C and D show the same images as Panels A and B but in anaglyph format. (C) This panel shows the root mass prior to the freezing (corresponding to panel A). (B) This is an image of the roots at the same point in the freeze event as in panel D. Panels C and D must be viewed with red-blue glasses to see the images in 3D. Please click here to view a larger version of this figure.

Supplemental Video 1: A red-blue anaglyph video showing freezing in a strawberry plant in 3D. This video was generated using the protocol demonstrated here. Note that red-blue glasses are necessary to observe the video in 3D. Please click here to download this file.

Supplemental Video 2: A red-blue anaglyph video showing freezing in wheat roots in 3D. This video was generated using the protocol demonstrated here. Note that red-blue glasses are necessary to observe the video in 3D. Please click here to download this file.

Discussion

Two IR cameras are necessary for this protocol, and they must be aimed at the subject from slightly different angles¹. This will require the lenses to be from 5 - 8 cm apart, but both must be aimed at the same place on the subject to be filmed. Think of the 2 camera lenses as a kind of surrogate for the viewer's eyes. The left camera is analogous to the left eye and the right camera to the right eye. The post-processing software will tint the left image to a red color and the right image to a blue color, so by wearing red-blue glasses, the left eye can only see the left image and the right eye only the right image. This means that it is important to use the gray-scale palette of the IR camera software when recording the freeze event. The brain will combine the 2 images which the viewer will observe in 3D¹.

Another critical step to this protocol is the use of screen capture software to capture the output of both cameras simultaneously. By capturing the output of both cameras simultaneously, a perfect synchronization of the output from both cameras is guaranteed. Synchronizing the right and left images is a crucial aspect of producing 3D films and are discussed elsewhere in detail.¹

To prevent any eye strain, it is important that the vertical and the horizontal convergence of the left and right images are correct. While the cameras should be positioned to ensure a correct convergence prior to the recording, they do not have to be perfect. The post-production software described here will allow adjustments in right-left, up-down, and rotational convergence. The software will also allow a red-green anaglyph video to be produced if red-blue glasses are not available.

One limitation of the technique is the requirement of red-blue glasses to view the 3D video. It is likely that many individuals will not have red-blue glasses readily available. Also, while producing a red-blue anaglyph video is the easiest and least expensive way to produce a 3D video, red-blue anaglyph videos can only convey a limited chromatic view of their subject. However, this is arguably an insignificant limitation since IR radiation, in reality, can only be observed in grayscale. Colors are only perceived by humans in the visible portion of the electromagnetic spectrum.

The limited resolution in early IR technology made it difficult to determine the precise locations of ice nucleation as well as which tissues the ice propagated into. Differential thermal analysis⁹ has improved the ability to detect sites of ice nucleation; however, it remains a 2-dimensional perspective that lacks information about depth. The lack of information provides a limited perspective and does not fully represent freezing as it occurs in the real world.

Commercial films use various techniques for visualizing images in 3D, the most common being polarization-interlacing¹. The most popular techniques require headgear that is specific to the interlacing process, but auto-stereoscopic techniques that do not require headgear are in developmental stages¹. None of the 3D rendering techniques, however, are available for viewing the IR video in 3 dimensions. In addition, while these techniques provide the clearest 3D video available, they require synchronization and special projection devices as well as reflective surfaces on which to project the images¹.

Communicating scientific findings in the clearest possible manner is essential for creating a community that will promote an efficient and timely progress in scientific discoveries. Observations of the world we live in are always in 3 dimensions, but it is difficult to accurately represent those observations using only 2D images. For example, it would be difficult, if not impossible, to determine precisely which root(s) had frozen in the IR imaging of the freezing in wheat roots (Figure 2B). However, using a 3D anaglyph process makes it relatively simple to determine exactly which root froze at what time (Figure 2D). Admittedly, it remains to be determined what new information (not obtainable from 2D videography) might be gleaned from a 3D perspective of freezing in plants. However, it is not unusual for unique information to be obtained when analyzing plant material in 3D¹⁰. By using screen capture software to precisely synchronize the right-left images and commercially available software to create an anaglyph video, any laboratory that uses visual data to understand biological processes can generate images and video in 3D.

Materials

Name	Company	Catalog Number	Comments
T620 Infrared Camera and software	FLIR	55903-5122	2 cameras are needed. Software works only on a Windows-based computer
After Effects	Adobe	15.0.1.73	Post-Production Video Editing Software
Bandicam	Bandisoft	4.1.2.1385	Screen Capture Software
Laboratory Scissor Jack	Eisco	CH0642A	Steel Platform 13X15 cm
Fastening Strap	Velcro	90441	To hold camera on jack.  Should be at least 60cm long by 2cm wide
Media Converter	iSkysoft	10.0.6	Software to convert mp4 files to .mov