A multi-spectral myelin annotation tool for machine learning based myelin quantification

Introduction

Myelin degeneration causes neurodegenerative disorders, such as multiple sclerosis (MS)^1,2. There are no remyelinating drugs. Myelin quantification is essential for drug discovery, which often involves screening thousands of compounds³. Currently, myelin quantification is manual, and labor-intensive. Automation of quantification using machine learning can facilitate drug discovery by reducing time and labor costs. However, myelin annotation suffers the same limitations as manual quantification. To assist researchers and bioimage analysts, we developed a workflow and a software for myelin ground truth extraction from multi-spectral fluorescent images.

Myelin is formed by oligodendrocytes wrapping the axons⁴. It is identified by continuous co-localization of cellular extensions that span multiple channels and z-sections (Figure 1). In our workflow, co-localizing pixels, candidate myelins, were determined using Computer-assisted Evaluation of Myelin (CEM) software that we previously developed⁵. In the current study, the 3D Myelin Marking (CEM3D) tool⁶ was developed to efficiently evaluate these candidate myelins and to extract myelin ground truths. Using CEM3D, an RGB-composite z-section image, corresponding CEM output image, and expert’s markings can be visualized simultaneously to decide whether to keep or remove candidate pixels (see Implementation). The user can move along x-y-z axes and show/hide channels, images and markings. Markings from the -1/+1 z-sections can be viewed simultaneously. Finally, CEM3D allows simultaneous visualization of myelin markings of two experts, which is important for inter-expert comparison.

Figure 1. An example of multi-spectral fluorescent image.

20× confocal microscopy image tiles were stitched together covering approximately 2 × 8 mm by 30–50 μm volume. Boxed area is enlarged to show myelin (brackets) and the false positive pixels (circles).

Using the described workflow, we annotated five images encompassing approximately 2 × 8 mm by 30–50 μm volume. The entire process, which would have taken several weeks, took approximately 5 days. More than 30,000 feature images were extracted from these five images and were used for testing various machine-learning methods^7–9. The annotated images, which are available with the manuscript, are a resource for the researchers working not only on myelin detection but also on segmenting multi-spectral images.

Methods

Implementation

In CEM3D, a new project is started by loading oligodendrocyte, axon, and nucleus images, red, green, and blue channels respectively in the example (Figure 2). Optionally, candidate myelin image, which is converted to vectors using the included module (see below), is loaded. Users can save and reopen projects. In CEM3D, users can zoom using the mouse wheel and can move in the x-y axes and z-axis using scroll bars and buttons respectively (Figure 2 and Figure 3).

Figure 2. Starting a new project in CEM3D.

Buttons for loading oligodendrocyte, axon, and nucleus images, and navigating the z-stack button to up and down are marked.

Figure 3. Myelin drawing and saving in CEM3D.

The relevant buttons and myelin vectors are marked.

Myelin pixels may be marked at various thickness values (Figure 3). CEM3D records myelin drawings as vectors in the “.iev” files. These vectors can be modified or deleted in CEM3D (Figure 3). Optionally, to facilitate myelin detection, the candidate myelins, can be loaded from CEM. Myelin identification using CEM is described in detail in 5. Output of CEM is a binary image, which is converted to vectors using the included module (Figure 4). Note that the conversion will overwrite your existing myelin vectors.

Figure 4. Loading CEM output image.

To load candidate myelin pixels, use “Convert Binary Image to Vector” button.

Additionally, CEM candidate myelins or two experts’ myelin vectors can be visualized. First, rename and copy the .iev file containing second myelin vectors to the same folder. Next, modify the .ini files as shown in Figure 5. After loading the modified .ini file using ‘Merge Edit’ button, myelin vectors will be shown in two different colors (Figure 6). These vectors can be modified as in Figure 6.

Figure 5. Visualizing two myelin vectors simultaneously.

Modify .ini file as in the lower panels and load it using “Merge Edit” button.

Figure 6. Modifying the myelin vectors.

CEM candidate myelins or two experts’ markings can be shortened, deleted or drawn over.

Once done with marking, users can convert the myelin vectors into an image using the “Save Myelin Mask Image” button. We implemented this strategy to extract gold standard myelin ground truths.

Results

In this study, myelin was marked by two experts on previously acquired oligodendrocyte and neuron co-culture images⁵ using the described workflow (see Implementation). A third expert evaluated their markings and extracted gold standard myelin ground truths. The ground truth images were saved as TIF on CEM3D⁶. All images are available (see below).

Because each image covered a large volume (approximately 2 × 8 mm by 30–50 μm), the entire process took approximately five work days. We estimated that it would have taken several weeks using conventional methods. Thus, CEM3D enabled collaboration of three experts for accelerated myelin ground truth extraction.

Next, we calculated experts’ performance. When compared to the gold standards that we extracted, two experts averaged 48.39% precision. The highest precision of an expert was 87.95% for one image. In comparison, our customized-CNN and Boosted Trees consistently reached precision values over 99%⁸. These results suggest that, machine learning methods can outperform human annotators once trained with accurately labeled data.

Conclusion

CEM3D⁶ accelerates annotation of multi-spectral images. As an example, we used it to annotate myelin, which can only be identified as co-localization of neuron and oligodendrocyte membranes within certain criteria. CEM3D’s visualization features simplified inter-expert collaboration and validation. Moreover, myelin ground truths accompanying this manuscript are a resource for the researchers working on segmenting myelin as well as other features in multi-spectral images.

Data availability

Software availability

CEM and CEM3D are available from: https://github.com/ArgenitTech/Neubias.

Archived source code as at the time of publication: https://doi.org/10.5281/zenodo.4108321⁶.

License: Non-Profit Open Software License 3.0 (NPOSL-3.0).