Dear Dr. Janjic,

Thank you very much for your approval and again noticing an important detail. We updated the manuscript to emphasize that this tool is for fluorescent images. Analysis of electron microscopy images of myelin present a different set of challenges.

We thank Predrag Janjic for his helpful comments. We believe that we address his concerns and the updated manuscript is easier to read and more satisfactory to the readers. Please see our point by point responses below.

I apologize in the time it took us to update the manuscript. I moved between institutions and it took me a long time to settle in part due to the pandemic restrictions.

T he manuscript introduces a 3D extension of myelin annotation tool reported in Ref[5], now applicable to fluorescent stacks. Although the main rationale to develop an automated tool for producing ground truth images is clear, and the importance has been laid out, methodological details and comparative data are missing for a researcher dealing with myelin imaging to get an idea of what the tool is really computationally doing in order to decide on its practical utility.

Please rework the Introduction and despite rather harsh length constraints add some minimal description of what algorithms in Ref[5] do.

Response 1: We updated the Introduction to include: “In this context, CEM software functions as a candidate myelin detection program because it simply identifies overlapping pixels. Briefly, CEM removes cell bodies, defined as the overlap of nuclei and cellular marker, and identifies overlapping pixels between remaining oligodendrocyte and neuron channels ⁶.”

Image acquisition:

Extend on the image and image processing details like the size of the captures, pixel size, deconvolving or not, depth corrections (you have some of it in the Results).

Response 2: The tool described in this publication is not an image processing tool per se. It is main function is annotation of myelin pixels for machine learning studies. Therefore, in this revision, to prevent any confusion, we renamed the tool as CEMotate.

Implementation:

Please extend on what has been done to integrate CEM tool, and parallel   visualization of myelin in subsequent planes, figures Fig.5 and Fig.6., elaborating the utility of this step which is the main procedural added value of the presented tool. Please move to additional material or remove Fig.1 - Fig.4., which are more of a user guide and are disruptive in the value presentation.

Response 3: In this study, the CEM tool is not integrated directly to CEMotate. We used images previously processed by CEM and annotated them using CEMotate. The details of processing by CEM were given in the reference Kerman et al. 2015. The one major goal of this manuscript is to introduce our myelin annotation tool and to describe how to utilize it. Therefore, we believe that Fig. 1 – Fig. 4. are useful for readers.

Comparative analysis:

Some more data is needed, new or from Ref[8] for a reader to be able to get the overall impression. Please consider adding a Benchmarking paragraph where you would extend a bit on some benchmarking given within the Results, with an estimate (table) of the computational time needed for the CEM scope per slice, and the total time to process a whole stack, all in order a potential end user to get an impression of the effort needed.

Response 4: The updated text below:

We added information on benchmarking and compared the length of the time it takes to use different approaches.

Results Section – Paragraph 2: Each image covered a large volume (approximately 2 x 8 mm by 30-50 μm). While CEM determined the candidate myelins on five images in approximately 43 minutes, ML approach took only 1.04 seconds for the same process ⁸ (Table 1).  Extracting the gold standard myelin ground truths from five images with candidate pixels that were determined by CEM took approximately another 35 hours for one expert.  This process involved determining FPs and FNs on ImageJ. The same process took approximately 20 hours for an expert using CEMotate. Thus, over 40% of time was saved (Table 2). Moreover, CEMotate enabled collaboration of three experts for accelerated myelin ground truth extraction. Because ImageJ does not have such a feature, we could not directly compare the times saved for this process.

Table 1. Time comparison to detect myelin in five images for CEM and ML Approach

CEM

ML Approach ⁹

Time ( ~ )

43 min

1.04 sec

Table 2. Time comparison for ImageJ and CEMotate annotation

ImageJ

CEMotate

Time ( ~ )

35 hours

20 hours

CEM identified 219032 candidate myelin pixels on five images. Two experts identified TP myelins. A third expert evaluated these results to obtain the gold standard myelin ground truths which covered 9550 pixels. To the best of our knowledge, this is the first time myelin ground truths are shared with the science community.

Table 3. Experts’ average precisions on candidate myelin pixels of five images

Expert 1

Expert 2

Average Precisions

36.23%

60.54%

Next, we calculated each expert’s performance. Two experts averaged 48.39% precision. The highest precision of an expert was 87.95% for one image. In comparison, our customized-CNN and Boosted Trees consistently reached precision values over 99% ⁸. These results suggest that machine learning methods can outperform human annotators once trained with accurately labeled data.

Results:

Please extend on limitations & issues of the CEM3D, and try to estimate if possible specific performance over a whole stack using this extension. (which is the actual improvement and a gain compared to relying only on CEM).  

 We updated the Results section to include the metrics as described above.