Automated Analysis of C. elegans Fluorescence Images using SegElegans

Summary

Here we provide instructions on effectively utilizing SegElegans, a deep learning system we developed for the automated segmentation of individual worms in widefield microscopy images, for subsequent use in image analysis software such as ImageJ. We provide ways to use the system both online and offline.

Abstract

Microscopy, particularly of the fluorescent kind, is a frequently used tool in C. elegans research. The analysis of data from microscopy experiments can, however, be quite tedious and time-consuming. Thus, automation is desirable. We developed SegElegans, a two-headed U-net-based convolutional neural network system that is specifically designed for the automated segmentation of worms, even in images with large numbers of touching or overlapping individuals. The first part of SegElegans consists of one encoder and two decoders. The encoder, based on the SmaAt AT model, applies double convolution layers followed by a Convolutional Block Attention Module (CBAM). Both decoders use convolutional LSTMs: one performs semantic segmentation of worm images (body, edge, background, or overlap), while the other extracts a linear skeleton along each worm. The second part is a post-processing algorithm that combines the outputs of the two decoders and uses them to generate accurate instance segmentations. These segmentations can then be fed to ImageJ or other appropriate image analysis tools. Here we present instructions on how to access and run this system. We provide an online, cloud computing-based implementation as well as two methods to use the SegElegans models offline, on a local machine, should the required hardware be available.

Introduction

Caenorhabditis elegans is a eukaryotic model organism with wide use in the fields of cellular and molecular biology^1,2,3. It is a small (~1 mm for wild type adults) soil-living nematode worm with a fast life cycle (~three weeks), and a large number of progeny (typically genetically identical to the self-fertilizing hermaphrodite parent). Despite its relative simplicity, the animal is still a complex multicellular organism with defined and distinct tissues and organs, which, thanks to its small size and transparency, can be studied in vivo without any need for fixing or other invasive treatments⁴. The capacity of the worm for in vivo study through microscopy can be further amplified through the use of fluorescent reporters, that permit the easy distinguishment of individual cells or even the direct visualization of sub-cellular organelles/components such as mitochondria, lipid droplets, or protein aggregates, and of processes such as signal transduction, gene expression, vesicle fusion, autophagy, neuronal and muscle action potentials, etc.^{5,6,7,8,9,10,11}. In addition, the nematode is also highly amenable to genetic manipulation through mutagenesis¹², transgenesis via microinjection or microparticle bombardment¹³, CRISPR editing¹⁴, and, most importantly, when it comes to easy and cost-effective genetic screening, RNAi¹⁵. This manipulation permits the extensive study of genes and their role in the nematode itself, while simultaneously, thanks to the significant genetic homology that C. elegans exhibits to other model organisms, making foundational discoveries with relevance all the way to humans^16,17. Finally, this homology/conservation also makes the worm an ideal initial testing ground for drugs and chemical agents, allowing the elucidation of the mechanisms of drug activity, the identification of potential activity-modifying genetic variations, and the discovery of potential unwanted interactions and off-target effects^18,19,20.

All the above-mentioned advantages have rendered C. elegans an attractive model for cellular and molecular biology studies, particularly for scientists interested in the in vivo and often real-time monitoring of processes via microscopy. Such studies typically involve a significant amount of image analysis through special software, with the most widely used option being ImageJ^21,22,23. A common characteristic of the analysis through such software is the need for the user/researcher to specify Regions of Interest (ROIs) for analysis through the use of selection tools. Quite frequently, each individual worm in an image will be selected as its own ROI, with the aim to acquire per-worm information about simple morphological features (such as body dimensions) or more "advanced" experimental readouts such as the expression levels of a gene (measured with a fluorescent reporter), the readout of a ratiometric reporter, the number and size of lipid droplets, the formation of protein aggregates, the morphology of individual organelles and their networks etc.^{5,8,10,24,25,26,27}. These per-worm selections are typically made manually, since automatic selection methods that are based on standard image processing algorithms are usually not capable enough to distinguish the shapes and features of the worm, especially in darkfield images where the outlines of the worms are not clearly visible or in brightfield images with significant numbers of animals that touch and overlap. The manual process is, however, slow (20 s to 1 min per worm depending on the precision needed and the experience of the user), laborious, and subject to user bias and error.

An alternative and much more powerful approach for the generation of individual C. elegans ROIs (a task typically referred to as segmentation in the field of computer vision) is to automate it with the help of deep learning/neural network techniques. Convolutional neural networks based primarily on the Mask R-CNN²⁸ and the U-net architectures²⁹ have produced decent results in segmentation tasks on various biological model systems^30,31,32 including in C. elegans^{33,34,35,36,37,38,39}, but none have provided a satisfactory solution to the problem of generating full body segmentations that correctly distinguish individual animals (instance segmentation) in high resolution images with large numbers of touching or outright overlapping worms. In order to meet this need, we developed SegElegans, a deep learning model specifically designed and optimized for this task⁴⁰.

SegElegans is comprised of two major parts (Figure 1). The first part is a two-headed U-net variant convolutional neural network. It is composed of one encoder block and two decoder blocks. The encoder block is based on the SmaAt AT model⁴¹ and uses double convolution blocks for each layer feeding into a Convolutional Block Attention Module (CBAM)⁴². The two decoder blocks are based on the convolutional long short-term memory (LSTM) network⁴³. One decoder is responsible for categorizing each pixel of the images as a part of the main body of a worm, a part of the edge of a worm, a part of the background, or a part of an area where worms overlap (this is typically referred to as semantic segmentation). The other decoder is responsible for drawing a linear "skeleton" along the length of each worm. The second part is a post-processing algorithm that combines the outputs of the two decoders and uses them to generate accurate instance segmentations. It initially identifies the segments of true overlaps by comparing the semantic segmentation output to the skeleton one. Then it outputs the instance segmentation directly for worms without overlaps or by assembling it from segments for overlapping ones⁴⁰. These segmentations are saved as binary masks as well as ImageJ compatible ROIs.

Protocol

The following sections include detailed instructions on how to make effective use of SegElegans (Figure 2). They include instructions on preparatory image acquisition (Section 1), how to run the model online (Section 2) or offline (Sections 3-5), and the import of ROIs to ImageJ or other tools (Section 6). Since the evaluation of images through the model requires a PC with a CUDA compatible Graphics Processing Unit (GPU) that has at least 6 GB (ideally more) of video random access memory (VRAM), most users (provided they have access to Google's services) are advised to utilize the online version of the model which satisfies and surpasses these hardware requirements through cloud computing (Section 1 > Section 2 > Section 6). Alternatively, users who have access to appropriately powerful hardware and a basic understanding of command line/terminal use (or users who lack access to the aforementioned cloud computing options) may find running the model locally more convenient (Section 1 > Section 3 > Section 4 or 5 > Section 6).

1. Acquisition of sample and guide images

1. Use SegElegans to acquire segmentations of adult worms from any worm strain, regardless of what fluorescent reporters they may or may not express. If the phenotype(s)/data of interest are measured in brightfield images, acquire them normally using a widefield microscope and compatible software, using a 4x or similar objective lens.
   NOTE: SegElegans can analyze images with a wide range of brightness and contrast, although it is preferable that there are no saturated bright or dark pixels. In regard to dimensions, the system has been trained with 1328 x 1048 images but should work with any image that is at least 512x512 in size.
2. If the phenotype(s)/data of interest are measured in darkfield images (typical when using one or more transgenic fluorescent reporters or dyes), acquire brightfield images alongside the data to be used as guide images (as explained in step 1.1). Ensure the guide images perfectly match the darkfield images of the fluorescence channel(s) spatially and as close as possible timewise, ideally by using the multichannel acquisition options that are provided by the microscope's operating software. Save the guide images in a separate folder from the corresponding darkfield data images, using the same name for both.

2. Running the online version of SegElegans

1. Log in to a Google account on a web browser. Creating a fresh dedicated lab account for this purpose is recommended.
2. Enter Google Drive (https://drive.google.com) and upload the folder with the brightfield images (data or guide). As mentioned before, if guide images are used to generate ROIs for darkfield images, do not include said darkfield data images in this folder.
3. Go to https://github.com/KonstantinosKounakis/SegElegansOnline/tree/v1.0 and click on the SegElegans Body Prediction Interface.ipynb file. Click on the Open in Colab button at the top of the file that opens. The Jupyter notebook/interface for SegElegans will open on Google Colab.
4. Execute code block 1 by pressing the play () button under the title. Grant the runtime permission to run the code and observe the output after execution is over (a green checkmark appears next to the play button). Colab should automatically run the correct, CUDA-compatible runtime and display an NVIDIA-SMI output table. Otherwise, force the use of a T4 GPU runtime through the Runtime > Change runtime type menu.
5. Execute code block 2 to load the contents of Google Drive into the runtime. Accept all confirmation dialogues and grant all requested permissions.
6. Execute code blocks 3 then 4. Block 3 will take a couple of minutes to finish, so ensure the green checkmark has appeared before proceeding to 4.
7. Open the icon with the folder tab to the left of the Colab interface. This will display the files that are loaded into the runtime, including the imported contents of Drive as /content/drive/MyDrive. Right-click on the folder with the images, copy the path, and paste it into the test_images input form of code block 5. Ensure it looks like this: /content/drive/MyDrive/Imagefolder. Similarly, specify a path for the analysis output (e.g., /content/drive/MyDrive/Imagefolder/Output). Note that paths in this environment use a forward slash to separate folders (/).
8. Execute code block 5.
9. Specify the exact extension of the images to be analyzed by the model in the provided form of code block 6. For the purpose of this input ".tif", ".TIF", and ".tiff" are treated as different extensions.
10. Do not adjust the batch_crop_img input in code block 6 if the system is running in the default T4 GPU runtime (which is provided for free to all users for a limited number of hours per day, depending on usage levels). Reduce it if the runtime raises memory availability issues.
11. Execute code block 6. This will take some time, so ensure the green checkmark has appeared before proceeding.
12. Execute code block 7. Do not adjust its inputs for the default T4 runtime, but reduce the number of subprocesses or outright disable parallel processing if memory availability issues emerge. This will also take significant time.
13. Access Google Drive in a different browser window or tab and navigate to the output folder designated in 2.7.
    NOTE: At this point, SegElegans has concluded the initial evaluation and post processing of segmentations and has created several subfolders with outputs. 0_summary results contain graphs summarizing the output for each image, with an index number assigned to each worm. 1_complete_mask contains the curated binary masks for all segmentations the algorithm has decided are good for analysis. 1_edge_small_mask contains binary masks for all segmentations the algorithm has decided to reject because the animals are too small or partially obscured at the edge of the image. 1_overlap_mask contains binary masks for all segmentations of worms that exhibited real overlaps, and are, by default, not added to the curated output. 1_all_rois_results contains the segmentations of ALL 3 types, regardless of curation, in the ImageJ format (zip files with ROIs).
14. At this point, choose one of the three options:
    1. Use the ROIs provided in 1_all_rois_results in ImageJ and reject unwanted ROIs after the import there (see protocol section 6).
    2. Accept the curated good masks without any manual correction (and without the inclusion of overlapping worms) (skip to 2.16).
    3. Manually adjust the curation by selecting the worms (including the overlapping ones) to be included in the output (continue to 2.15).
15. In order to manually adjust the curation, use code block 8. Examine the results of the initial curation from the summary graphs in the 0_summary results folder. For each image that needs correction, input the full name (with extension) of the original input image in the "name_image_change" form and the numbers of the masks to be kept (from the summary graph), separated by commas in the "index_images" form. Execute code block 8. Repeat this step for any other image that needs correction.
16. After correction or if users choose to accept the initial curation without corrections (and without the overlapping worms), execute code block 9. A new subfolder will be created in the outputs folder in Google Drive called 2_curated_rois_results. It includes all the final curated segmentations in the ImageJ format (zip files with ROIs).
17. In order to execute the process again for a different folder of images, reset the runtime through the menu Runtime > Restart Session and start from the beginning.

3. Preparing the offline version of SegElegans before the first use

1. As an alternative to running SegElegans online on Colab, an offline version of SegElegans is available for users who cannot or prefer not to use Google's cloud computing services. This requires a PC (Windows or Linux) with a CUDA-compatible GPU (https://developer.nvidia.com/cuda-gpus) that has at least 6 GB of VRAM (although 16 is recommended if available and affordable). 
   NOTE: The following instructions in sections 3-5 are focused on Windows, but the software can also run on Linux-based systems with some small changes in the commands used.
2. Download the CUDA toolkit installer from https://developer.nvidia.com/cuda-downloads following the instructions to get the version that is appropriate for the device used. 
   NOTE: SegElegans has been tested with versions 12.9 and 13.0 of the toolkit and CUDA Version 11 in Windows, however future versions are expected to retain backwards compatibility.
   1. Install the toolkit, following the instructions of the installer itself. Visual Studio is not needed.
3. Download the appropriate version of the Python installer for the device from https://www.python.org/downloads/. Install Python, following the instructions of the installer itself, and ensuring python.exe is added to PATH.
   NOTE: SegElegans was built and tested on Python 3.13, but future versions are expected to retain backwards compatibility.
4. Go to https://github.com/KonstantinosKounakis/SegElegansOffline/releases/tag/v1 . Click on the Source code (zip) link to download the package with the necessary files to set up and run SegElegans locally.
5. Extract the zip file. Place it in a high or even top-level folder, such as C:\SegElegans. The contents of the zip file already come inside a subfolder SegElegansOffline-1.
6. Open the Windows command prompt (cmd.exe). Type the command python -m venv "fullsegeleganspath" where "fullsegeleganspath" is the full path, in quotes, into the SegElegansOffline-1 subfolder.
   NOTE: For example, if the zip file was extracted in C:\SegElegans, the full path is C:\SegElegans\SegElegansOffline-1 and the command will be python -m venv "C:\SegElegans\SegElegansOffline-1" . Note that paths in this environment use a backslash to separate folders ("\").
7. In the command prompt, use standard controls to navigate inside the SegElegansOffline-1 folder and execute the command Scripts\activate. A (SegElegansOffline-1) label should appear before the next prompt cursor, indicating the Python environment is now active.
8. While inside the active (SegElegansOffline-1) environment, run the command pip install -r requirements.txt . The process will take some time.
9. Input and execute the command python assemblenetworks.py .
   NOTE: SegElegans is now ready to run. Two ways to achieve this are provided: a quick script that will automatically provide all segmentations as ImageJ ROIs without the option to manually correct the curation (section 4), and a local Jupyter notebook that will provide the option to overview and correct the curation (section 5). The latter approach is recommended for most users.

4. Running the offline version of SegElegans with a quick script

1. Open the Windows command prompt (cmd.exe). As described in step 3.7, navigate to the SegElegans environment folder and activate it.
2. In order to use the quick script, while in the command prompt with the active (SegElegansOffline-1) environment input and execute the command python SegElegansBodyQuickEval.py -i "pathtoinputfolder" -x where "pathtoinputfolder" is the full path, in quotes, to the folder with the brightfield images (e.g. "D:\Data\Experiment12365\Condition 1") and is the precise file extension of the images users wish to analyze (Example ".TIF"). Type the command python SegElegansBodyQuickEval.py -h for instructions on additional parameters that can be changed to accelerate the processing by the script if the device can handle it.
   NOTE: The output folder in this case will be created inside the input folder (So for the above example: "D:\Data\Experiment12365\Condition 1\Output"). Inside, there is a series of subfolders. 0_summary results contain graphs summarizing the output for each image, with an index number assigned to each worm. 1_complete_mask contains the curated binary masks for all non-overlapping segmentations the algorithm has decided are good for analysis. 1_edge_small_mask contains binary masks for all segmentations the algorithm has decided to reject because the animals are too small or partially obscured at the edge of the image. 1_overlap_mask contains binary masks for all segmentations of worms that exhibited real overlaps. 1_all_rois_results contains the segmentations of ALL 3 types, regardless of curation, in the ImageJ format (zip files with ROIs). Since there is no way to manually correct curation here, 2_curated_rois_results includes by default all segmentations deemed good by the algorithm AND all overlapping segmentations in the ImageJ format (zip files with ROIs).

5. Running the offline version of SegElegans with a Jupyter notebook

1. Open the Windows command prompt (cmd.exe). As described in step 3.7, navigate to the SegElegans environment folder and activate it.
2. In order to use the Jupyter interface, while in the command prompt with the active (SegElegansOffline-1) environment, input and execute the command jupyter notebook. This will open a web browser window/tab with a locally hosted site. In the initial screen, select the file SegElegansOfflineJupyterInterface.ipynb.
3. Execute the code in code block 1 by selecting it (click on the left outside of the actual code) and pressing the play () button on the toolbar above. A confirmation message will appear below the code block when the execution is complete.
4. Execute code block 2. This will generate some input forms in the readout underneath the code block. Fill in the generated forms with the path to the input folder with the brightfield images (Example: "D:\Data\Experiment12365\Condition 1"), the desired path for the analysis output (Example: "D:\Data\Experiment12365\Condition 1\Output"), and the precise file extension of the images to be analyzed (Example ".TIF"). Adjust the additional provided settings (increase the number of subcrops per batch to 9 or 16, and run multiple post-processing parallel processes) if the device is sufficiently capable.
5. Execute code block 3.
6. Execute code block 4. This will take some time, so ensure it is complete before proceeding.
7. Execute code block 5. Again, this will take some time. After this, navigate to the output folder designated above.
   NOTE: At this point, SegElegans has concluded the initial evaluation and post processing of segmentations and has created several subfolders with outputs. 0_summary results contain graphs summarizing the output for each image, with an index number assigned to each worm. 1_complete_mask contains the curated binary masks for all segmentations the algorithm has decided are good for analysis. 1_edge_small_mask contains binary masks for all segmentations the algorithm has decided to reject because the animals are too small or partially obscured at the edge of the image. 1_overlap_mask contains binary masks for all segmentations of worms that exhibited real overlaps, and are by default not added to the curated output. 1_all_rois_results contains the segmentations of ALL 3 types, regardless of curation, in the ImageJ format (zip files with ROIs).
8. Similar to the online version of SegElegans, use any of the three options listed below:
   1. Use the ROIs provided in 1_all_rois_results in ImageJ and reject unwanted ROIs after the import there (see protocol section 6).
   2. Accept the automatically curated good masks without any manual correction (and without the inclusion of overlapping worms) (skip to step 5.11).
   3. Manually adjust the curation by selecting the worms (including the overlapping ones) to be included in the output (continue to step 5.9).
9. In order to manually adjust the curation, use code blocks 6 and 7. First execute 6. This will create input forms in the readout underneath the code.
10. Examine the results of the initial curation from the summary graphs in the 0_summary results subfolder. For each image that needs correction, input the full name of the INPUT image (with extension) in the Image to correct: form and the numbers of the masks to be kept (from the summary graph) separated by commas in the Masks to keep: form. Execute code block 7 and ensure execution is complete. Repeat this step for any other image that needs correction.
11. After the curation correction is done or if users choose to accept the initial curation without corrections (and without the overlapping worms), execute code block 8. A new subfolder will be created in the designated outputs folder called 2_curated_rois_results. It includes all the final curated segmentations in the ImageJ format (zip files with ROIs).
12. In order to execute the process again for a different folder of images, reset the runtime through the menu Kernel > Restart Kernel and Clear Outputs of all Cells and start from the beginning.

6. Importing the segmentations to ImageJ (or alternatives)

1. Open one of the actual data images on ImageJ.
2. Open the corresponding zip file with the ROIs of that image. This will load the selections into the ImageJ ROI manager. If they are ROIs from the 2_curated_rois_results output they are ready for analysis with the desired methods normally used (preferably with the use of macros to further automate and accelerate the process).
3. If they are ROIs from the 1_all_rois_results output, remove unwanted segmentations from the ROI manager. Select them and press the Delete button on the manager window itself. Do not press the Delete key on the keyboard, as that deletes the contents of the selection in the image instead.
4. If software other than ImageJ is needed for the analysis, the ROI format will most likely not be compatible. In that case, import segmentations in the universally utilized form of binary masks, which are provided in the 1_complete_mask and 1_overlap_mask folders. Consult the instructions of the receiving software on how to perform that import.

Results

By following this protocol, researchers should be able to extract high-quality worm segmentations, even for the analysis of fluorescence images without visible worm borders. As discussed in section 6 of the protocol, these segmentations can be imported directly to ImageJ and used for the quick measurement of relevant properties such as the intensity of fluorescent reporters (commonly used for expression quantification) or the number, size, and morphology of fluorescently tagged areas (used frequently to study protein agg...

Discussion

The methodology presented here should allow users to analyze C. elegans microscopy experiments in a significantly faster timeframe without any loss in accuracy. Since it extracts segmentations from brightfield guide images independently of the actual fluorescence (as discussed in protocol section 1) it can be used with any strain and for any application that requires measuring phenotypes on a per-worm basis. These can include single-channel applications such as the quantification of the formation of abnormal pro...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Computer with internet access (example: Dell Vostro Desktop)	Several (Dell for example)	-	This will allow running SegElegans online via cloud computing as recommended and using image analysis software such as ImageJ.
CUDA compatible Graphics Processing Unit with at least 6 GB of VRAM (example: MSI Geforce RTX  RTX3050 Ventus 2X 6GB) (Optional)	NVIDIA + several manufacturers (Micro Star International for example)	https://developer.nvidia.com/cuda-gpus	This will allow running SegElegans offline on a local machine. CUDA compatible GPU chipsets can be found here: https://developer.nvidia.com/cuda-gpus The VRAM is dependent on the actual implementation of the chipset into a GPU by the manufacturer. WARNING: You will need an expensive and powerful CUDA card far past the minimum requirements (16+ GB VRAM) to match the performance achieved by the cloud computing version.
Epifluorescence Microscope (Invitrogen EVOS FL Auto 2)	ThermoFisher Scientific	AMAFD2000 for example	We recommend using a system that can automatically acquire images from both brightfield and one or more fluorescence channels almost simultaneously.
ImageJ (Optional)	NIH	https://imagej.net/software/fiji/downloads	We recommend the Fiji distribution, available here: https://imagej.net/software/fiji/downloads
PC with NVIDIA GPU compatibility (example: Dell Vostro Desktop) (Optional)	Several (Dell for example)	-	This is needed in conjunction with the GPU for running SegElegans offline on a local machine. A GPU will require Windows compatible hardware with the appropriate PCI slots on the motherboard, room to fit the card and a sufficiently powerful power supply.
SegElegans	-	https://github.com/KonstantinosKounakis/SegElegansOnline/tree/v1.0	See manuscript for full instructions.