The Covid CT estimation tool is based on standard image processing techniques. Our interest is in volume, so the same voxel size is critical for good enough estimation. But it is also important to go through the different types of data we can encounter. In general, the Hounsfield Units (HU) make up the grayscale in medical CT imaging. It is a scale from black to white of 4096 values (12 bit) and ranges from -1024 HU to 3071 HU (zero is also a value). It is defined by the following:

-1024 HU is black and represents air (in the lungs). 0 HU represents water (since we consist mostly out of the water, there is a large peak here). 3071 HU is white and represents the densest tissue in a human body, such as tooth enamel. Materials with higher atomic numbers, such as bones, appear as brighter areas on CT images and are assigned higher HU values (typically between +700 and +3000). All other tissues are somewhere within this scale; fat is around -100 HU, muscle around 100 HU, and bone spans from 200 HU (trabecular/spongeous bone) to about 2000 HU (cortical bone).

DICOM files are usually saved in signed 16 bit, with original HU, usually with 3 mm slicing or 0.6 mm slicing CT images. TIFF, however, may have reshaped histogram values to cover the whole range and can preferably be in unsigned 16 bit or 8bit with some loss due to conversion. TIFF values usually lose Z voxel size metadata in conversion (resulting in Z voxel size value of 1), so it is essential to reset voxel values. The XY voxel size can be different with each data set, even from the same CT machine. The distribution of intensity values may change with different CT protocols, so some of the processing steps need to be done manually.

The workflow follows the Croney Ethical guidelines for the appropriate use and manipulation of scientific digital images. ¹⁴

The plugin tool is developed in ImageJ macro language. It needs Bio Format plugin to import DICOM files, which comes installed in FIJI. The macro language uses standard image processing techniques and morphological operations to estimate the volume ratio of lungs and pneumonia caused by COVID-19. It allows users to subsequently set up a threshold for pneumonia and lungs, and go through the whole data-set slice by slice and interactively tweak the threshold values. The tool was developed based on demand and with coordination from the Department of Radiology from the Faculty hospital Královské Vinohrady. It is challenging to do any kind of percentage estimate of pneumonia in the lungs just by visually inspecting CT scans stack by stack. The available hardware equipment and local account restrictions had to be taken into account for development tool selection. The ImageJ plugin is a compromise in accuracy and requirements. The scripts are published with the paper. The workflow for 8-bit script version is following: 1)

Input and pre-processing a)

Clear the log and close all open images.

Numeric result and composition image representation from step 3.a (original data, lung and pneumonia mask) is shown to the user (as illustrated in Figure 1).

Numeric results in percent are corrected by subtracting 3% (median of tissue present in healthy lungs, estimated from 10 patients) and CT is scored based on severity ranged (0:0%; 1, <5%; 2:5–25%; 3:26–50%; 4:51–75%; 5, >75%; range 0–5) defined by Ref. 13.

There are several steps during the tool runtime which require user inputs: 1.

Select the CT lung data ( Figure 2, TIFF or DICOM file based on script version) - CT sequence is opened and user can go through loaded stack in image sequence with a slider or as a video with a play button.

The tool has been tested both 3 mm slicing and 0.6 mm slicing CT images. The results were similar in percentage and the final CT score was the same.

In order to use the tool, the user needs to prepare CT images exported as DICOM or TIFF in the preferred view mode and preferably 16-bit representation. The CT images usually have a 12-bit gray-scale representation and an 8-bit conversion would lead to loss of potentially important information or shift of brightness values. The thickness of the CT slice can also contribute to numerical errors in the process, but there was no significant difference in results when processing the same data-set with 3 mm and 0.6 slicing.

The ImageJ software tool available from Zenodo or GitHub needs an ImageJ (ideally version 1.52v99 or newer) installed with Bio-Formats (preferably with version 6.8.0 which we tested) plugin (or FIJI which is a version of ImageJ with an already integrated Bio-Formats plugin).

The minimal requirements for both are Windows XP or later with Java installed, Mac OS X 10.8 or later with Java installed, Ubuntu Linux 12.04 LTS, or later with Java installed. Minimal RAM is based on the size of processed images. In this case, multiple images are opened at once.

The international standard for saving DICOM files defines 3 mm slicing of CT data as the default way. However resaving data as TIFF (losing voxel information) or using different slice thicknesses (like 0.6 mm slicing) may result in a different result. In theory, 0.6 slicing would provide 5 times more detailed sampling in the Z-axis. However, in practice it is different.

The same CT dataset exported with 0.6 and 3 mm slices (XZ view for comparison is in Figure 3) was analyzed with our tool with a lung threshold of 0-155 and a pneumonia threshold of 47-115. The results can be found in Table 1. The error from a comparison of 3 mm and 0.6 mm slicing is estimated at 0.58 %. The used CT is available in the attached published dataset as CT1_1 (0.6 mm slicing) and CT1_2 (3 mm slicing).

The biggest challenge in using this tool is an individual perception of images, as each person may see image data fundamentally the same - despite different appearances. Based on this a user can add the biggest bias even though the underlying data analysis is done correctly. The Table 2 contains a comparison of the results of the analysis in on 5 different CT datasets provided by the Faculty hospital of Královské Vinohrady. All CTs are analysed by users with different experience. The first CT exported with different slicing (also used in Table 1) is analysed by a radiologist (an expert user). The ANOVA test ( Table 5 Results for Score, Table 6 Results for Percentage) assesses the variance between datasets is statistically significant ( For Score F-value: 75.06 and p-value: 1.64e-21, For Percentage F-value: 89.85 and p-value: 3.53e-23), proving the selection of datasets is representative for comparisons.

The score aims to divide the percentage into groups based on previous research done, ¹³ and should be the deciding factor for future care for patients.

Ensuring the reliability and accuracy of results when working with user-input tools is crucial and requires careful consideration of inter- and intra-variability. This challenge can be addressed by using standardized procedures and guidelines, multiple raters for segmentation, and computer-aided methods. Our software tool addresses this issue by providing standardized procedures and guidelines, along with the ability to compare results through logs and promote reproducibility.

It is essential to take into account the level of training and experience of individuals performing the segmentation, as well as the time and resources available, as these factors can significantly impact the consistency and accuracy of the segmentation. The software tool provides a solution for addressing the challenges of inter and intra variability in CT data segmentation, helping to ensure careful planning and execution of a study and appropriate outputs to achieve repeatable and comparable results.

Using scoring will overcome some of the problems of comparing percentages directly. Figure (Comparison of CT scoring) shows that users rely on their experience and will choose parameters based on them. It shows the difficulties in ensuring consistency and accuracy of data segmentation when performed manually by multiple individuals. The possibility of comparing the results of the analysis of multiple users using a defined analysis process ( Figure 4a) leads to more reliable results. CT1_1 and CT1_2 is the same dataset with different slicing and percentage results of analysis from all users clearly show inter-variability ( Table 2, thresholds and scores in Figure 4). The overall scoring is the same as the result from the trained radiologists (CT1_1 - 31%, score 3; CT1_2 - 32%, score 3).

Intra variability ¹⁵ was established by 3 runs of set of randomized and blinded datasets. Each run consisted of 3 copies of the 6 published datasets, in random order. There was a at least 4 weeks in between of evaluation of each run. The resulting Statistical Analysis consists of the standard deviation and coefficient of variation for each metric (percentage and score) across repetition and an ANOVA ( Table 5 Results for Score and Table 6 Results for Percentage) and intraclass correlation coefficient (ICC) analysis to assess agreement and consistency. The results can be found in Table 3 Scoring results for 3 repetitions.

Tool performance based on descriptive stats

The single-user scoring system demonstrates excellent reliability (ICC1 = 0.91), showing that the user's scoring using proposed toll is consistent across datasets. This reliability is critical since all dataset evaluations depend on the judgment of one rater using this tool. When we average scores across multiple evaluations (by the same user), the reliability improves even further (ICC1k = 0.97). This result indicates that averaging can mitigate occasional inconsistencies and ensure stable, reproducible scoring. The high ICC values confirm that the scoring process is robust and reliable, even when performed by a single evaluator. This ensures the validity of the results and confidence in the conclusions drawn from the data analysis. The full ICC results are in Table 7 Results for Score and Table 8 Results for Percentage.

The tool still demonstrates varying precision across datasets ( Table 4 Standard Deviation and CV for Each Metric). Lower cv values for Percentage in CT4_TIFF and consistent Score in CT3_TIFF (cv: 0.00) suggest high reliability for these cases. Showing that previously designed Scoring parameters are well based, and preferred for use, while percentage can be valuable but not as reliable information. Because of this, user training, and using provided guidance will lead to stable results.

The biggest limitation of this approach is human error and inter and intra variation of manual selection. The percentage estimation might also be affected by other body cavities filled with air. There might also be a variance in results based on slice thickness, in worst case scenario 20%, but our experiment shows that there is only about 0.58% difference in result between 0.6 and 3 mm CT slice thickness. The scoring should also be improved so it is not dependent only on one value (volume percentage), but normalized SHU distribution in the pneumonia area should be also considered. When converting from 12-bit to 8-bit image representation, the reduced range of values results in a loss of information and detail, which can lower the quality of the output. However, for CT image segmentation, the use of Single Hounsfield Unit (SHU) values is adequate, as SHUs do not rely on single units and can provide good-quality segmentation.

For evaluating the agreement between repeated measurements, we used the intraclass correlation coefficient (ICC). ICC is a widely used indicator of reliability in medical research. ¹⁵ However, it is known that the ICC value is sensitive to the range of variability in the measured sample. ¹⁵ A narrow range of true values in the sample can lead to a low ICC even with good measurement agreement, and conversely, a wide range can artificially inflate the ICC value. Therefore, when interpreting our ICC results, it is necessary to consider the potential influence of data range. Our decision to use ICC was motivated by its common use for evaluating inter- and intra-rater reliability and the desire for comparability with existing literature in the field of image data analysis.

From the software point of view, there is a limitation in the version of ImageJ used. The new version of the code logs the ImageJ version and BioImage plugin version. There is a version of the code explicitly made for ImageJ version 1.52v99 and for other versions. The bind version helps reproducibility of any analysis based on logs, and it is advised to reproduce the analysis in the same version of ImageJ as indicated in logs.

Zenodo: CT scans of COVID-19 patients, https://doi.org/10.5281/zenodo.5805939. ¹⁷

Datasets contain CT scans of COVID-19 patients from Faculty hospital of Královké Vinohrady in DICOM (and TIFF), as per the folder name. Dataset CT1 is presented with 0.6 mm and 3 mm slicing.

This project contains the following underlying data: •

CT1_1

Data are available under the terms of the Creative Commons Attribution 4.0 International (CC-BY 4.0).