F1000ResearchF1000Research2046-1402F1000 Research LimitedLondon, UK10.12688/f1000research.129826.1Software Tool ArticleArticlesPyRadGUI: A GUI based radiomics extractor software
[version 1; peer review: 3 approved with reservations]
SherkhaneUmesh B.ConceptualizationData CurationFormal AnalysisSoftwareWriting – Original Draft PreparationWriting – Review & Editing12JhaAshish KumarConceptualizationData CurationFormal AnalysisSoftwareWriting – Original Draft PreparationWriting – Review & Editinghttps://orcid.org/0000-0001-5998-3206a123MithunSnehaData CurationFormal AnalysisWriting – Review & Editing123JaiswarVinayData CurationFormal AnalysisWriting – Review & Editing2TraversoAlbertoData CurationFormal AnalysisWriting – Review & Editing1WeeLeonardFormal AnalysisWriting – Review & Editing1RangarajanVenkateshConceptualizationWriting – Review & Editing23DekkerAndreConceptualizationWriting – Review & Editing1
Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, 6229 ET, Maastricht, 6229, The Netherlands
Department of Nuclear Medicine and Molecular Imaging, Tata Memorial Hospital, Mumbai, 400012, India
Homi Bhabha National Institute (HBNI), Deemed University, Mumbai, 400012, Indiaashish.kumar.jha.77@gmail.com
Radiomics is the method of extracting high throughput mathematical and statistical features from medical images. These features have the potential to characterize the underlying pathology of the disease that is inappreciable to a trained human eye. There are several open-source and licensed tools to extract radiomic features such as pyradiomics, LIFEx, TexRAD, and RaCat. Although pyradiomics is a widely used radiomics package by researchers, this software is not very user-friendly and can be run using a command line. We have developed and validated the GUI tool, PyRadGUI to make the radiomics software easy to operate. This software adheres to IBSI radiomic feature definition and implements the radiomic pipeline in batch processing to extract radiomic features from multiple patient’s data and stores it in a comma separated value (CSV). We validated PyRadGUI software with the existing pyradiomic pipeline.
Graphical User Interface (GUI)machine learningradiomicspyradiomicsplastimatch3DSlicer The Netherlands Enterprise Agency (RVO) and MeITyProject is funded by The Netherlands Enterprise Agency (RVO) and MeITy for the Indo-Dutch NWO/MeITy BIONIC and TRAIN project.The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Instruction
As per the World health organization (WHO), cancer is the second leading cause of death worldwide. Globally one out of six deaths are caused by cancer alone which amounts to 9.6 million deaths in 2018 (
WHO-Cancer, 2022;
Ferlay
et al., 2019). Treatment of cancer has always remained a challenging task for the oncology community. Although earlier diagnosis and treatment are often associated with a better outcome of treatment, at the same time selection of appropriate patients for appropriate treatment is important (
Stewart
et al., 2018). Considering the complexity of this disease and treatment, oncology is gradually moving towards personalized medicine (
Agyeman and Ofori-Asenso, 2015). A pathological test is considered a confirmatory test for cancer. Imaging tests like Computed Tomography (CT)/Positron Emission Tomography (PET)/Magnetic Resonance Imaging (MRI) also play an important role in diagnosis, treatment planning, and follow-up of the disease (
Caers
et al., 2014). In the last several years, the role of imaging and various advanced tests like immunohistochemistry (IHC), polymerase chain reaction (PCR), and gene sequencing has been increasing gradually in personalizing the treatment. Similarly, the role of artificial intelligence (AI) and radiomics has also witnessed a surge in oncology research over the last few years. Radiomics has been identified as an area of research to develop imaging biomarkers for the personalized treatment of cancer (
Lambin
et al., 2012;
Aerts
et al., 2014;
Goodwin
et al., 2017). Radiomics is a method to extract high throughput data from medical images. These features have the potential to uncover disease characteristics that are not appreciated by the expert radiologist or imaging personnel through visual interpretation (
Yip
et al., 2017). As radiomic features are extracted directly from medical images it provides a non-invasive method for tumor characterization as demonstrated by various researchers in the past (
Lambin
et al., 2012;
Goodwin
et al., 2017;
Aerts
et al., 2014). Researchers have shown the role of radiomics as a clinical predictor helping in advanced cancer care as personalized medicine in cancer (
Haider
et al., 2020). Apart from its role in precision diagnoses and characterization of a tumor, the role of radiomics has also been demonstrated in treatment planning (
Limkin
et al., 2017).
Several open-source and licensed software packages for radiomic extraction, like IBEX, RaCaT, CaPTK, LifeX or CGITA, Pyradiomics, and TexRad have been developed and used by several researchers in the past (
Pfaehler
et al.,2019;
Nioche
et al., 2018;
Zhang
et al., 2015;
Davatzikos
et al., 2018;
Johnson, 2015;
Haralick
et al., 1973).
The main challenges with these software packages are the complexity of radiomic extraction and the lack of standardized feature extraction from this software. The mathematical feature definitions have been provided image biomarker standardization initiative (IBSI) to standardize the radiomic extraction. It has also provided the phantom data sets with radiomic feature values (
Zwanenburg, 2017;
Zwanenburg
et al., 2016). Although IBSI standard is widely known, only a very few radiomic softwares have standardized the entire radiomic pipeline for feature extraction. Furthermore, the majority of these radiomic extractors have operability issues and not all defined features are extracted as defined by IBSI. Pyradiomics is a widely-used open source radiomics package and adheres to the IBSI standards, but it is not user-friendly. For instance, Pyradiomics is run on a command prompt and customization is technically demanding. Hence, the use of this software is cumbersome and technically demanding for clinical doctors or scientists.
In this study, we have developed and validated a graphical user interface (GUI) based radiomic extraction software using the pyradiomics package. This software adheres to all features defined by the IBSI.
Methods
This work is part of the Big Imaging data approach for Oncology in a Netherlands India Collaboration (BIONIC) and “personal health Train for radiation oncology in India and the Netherlands” (TRAIN) project is approved by the IEC of the hospital as a retrospective study. This software (PyRadGUI) was developed using Python open-source software on Windows systems.
Ethical considerations
This study was approved by the hospital Institutional Ethics Committee (Institutional Ethics Committee-I, Tata Memorial Centre [IEC, TMC], Mumbai, India Approval Number: 1905; dated: October 05, 2017) as a retrospective study, with waivers of informed consent from involved patients as per IEC policy of our hospital by the same Ethics Committee.
Software development
PyRadGUI front end was developed by using the open-source software Python 3.6.5
(Python, 2022) and the Python tk8.6 (
Python tk, 2022) module was used for the development of the graphical user interface (GUI). The open source Plastimatch package 1.8.0 (
Plastimatch, 2022) was used for Digital Imaging and Communications in Medicine (DICOM) to Nearly Raw Raster Data (NRRD) conversion of imaging data and the radiomic package in python; Pyradiomics 3.0 (
Griethuysen
et al., 2017) was used for radiomic feature extraction. This software loads DICOM images and region of interest (ROI) from a specific folder on the computer. First, it converts the image and ROI in NRRD format using Plastimatch 1.8.0, subsequently, it extracts radiomic features from the image and finally stores the output in comma separated values (CSV) format in an output folder on the computer. The details of the software are described below (
Figure 1).
The batch process of radiomics features extraction from our tool.
The user has to select the input folder containing the Digital Imaging and Communications in Medicine (DICOM) image and radiotherapy (RT) structure files. We can select the output folder and also customize the feature extraction process by changing values in the ‘.yaml’ file.
To start PyRadGUI, GUI_batch_radiomics.py is run on the command prompt. GUI, shown in
Figure 2, has been divided into two parts, i.e., left and right containers. The left container contains three tabs that are used for customization and radiomic extraction and the right container displays the process and error if any.
The figure shows the graphical user interface (GUI) of the radiomic extraction module (A) and customization window (B).
After starting the program, the user must select an input folder containing multiple patient images and a Radio therapy (RT) structure in DICOM format. Next, the user must select an output folder and then change the feature extraction settings from the settings tab.
Steps involved in customization settings
Selection of image type (Original, Laplace of Gaussian, and Wavelet): To extract features on the original image, we can select the image type as the original. If feature extraction has to be done on the transformed image, then we can select either LoG (Laplace of Gaussian) or Wavelet or both.
Selection of feature types: We have to specify the type of features we want to extract. The default includes all 1093 features from all image types.
Bin width selection: In the customization window one can select bin width as per the user requirement and the default bin width is 25. We have used 25 bin widths for CT and MRI and 0.5 bin width for PET radiomic extraction.
Sigma value for LoG features: In the customization window one can select the sigma value for LoG features. We have selected default 1-, 2-, and 3-mm sigma values for radiomic feature extraction.
Selection of resampled pixel spacing: In the customization window one can select pixel spacing as desired by the user for radiomic extraction. We have used the default 2×2×2 cubic mm pixel spacing.
After the settings have been customized, radiomics extraction can be started by clicking on the radiomics extraction button. The batch extraction starts by loading the DICOM folder and calls plastimatch. Plastimatch takes the reference CT folder and converts the input DICOM to NRRD (nearly raw raster data) format. It converts the image to image.nrrd and mask to mask.nrrd. Pyradiomics takes the converted nrrd image, nrrd mask, and the settings specified by the user and then extracts the radiomics features and writes the output in CSV format in the selected output folder. In this output, CSV file columns represent radiomic features and rows represent individual patients. The first column of the file is the patient identification number. This CSV file can be used for various analyses. Status of the running of the process, success, failures, and error reports are displayed in the right container in GUI, and processing reports are stored in the output folder as log files.
System requirements
We have developed and tested our software on two computer systems (machines) using various software packages. The details of system information and software packages are shown in
Table 1.
Computer configuration and packages used for this study.
In total 50 non-small cell lung carcinomas (NSCLC) patients’ PET/CT data with delineation and 20 chondrosarcoma patients’ MRI data with delineation who were imaged between 2014 to 2017 were used for validation and performance testing of this software. The patient’s demographic data is shown in
Table 2. Images from 100 NSLC patients were utilized to test the batch processing.
Demographic data of patients used in this study.
Disease
Sex
Total no.
Median age
Image type
NSCLC
Male
84
66
CT & WBPET
Female
16
53
CT & WBPET
Chondrosarcoma
Male
17
46
MRI regional
Female
3
42
MRI regional
Software validation
Software validation was performed by comparing Radiomic feature value extracted using PyRadGUI Workflow (PrGW) and the two reference workflows i.e., Reference Workflow1(RW1) (3DSlicer +Pyradiomics) and Workflow2 (RW2) (Plastimatch +Pyradiomics). The same version of the Pyradiomic package and Plastimatch was used for all the workflows. For the comparison, CT imaging data from 10 patients, PET imaging data from five patients, and MRI imaging data from five patients were all employed. PyRadGUI validation algorithm workflows are shown in
Figure 3.
The figure shows the PyRadGUI workflow (PrGW) validation using PyRadiomic Reference Workflow1(RW1) and Reference Workflow2 (RW2) for radiomic extraction.
Reference Workflow1 (RW1): Individual patient’s DICOM image and ROI were loaded in
3D slicer 3.0. The first appropriateness of tumor delineation was checked by an experienced imaging physicist (15 years of experience). Image and ROI were converted in NRRD format and the image was saved as an
image.nrrd and ROI as
label.nrrd in the patient's image folder. Again, the NRRD image and label were loaded in 3D Slicer, and appropriateness on tumor delineation was checked by the same physicist. Subsequently, the radiomic feature was extracted and saved in CSV format using the pyradiomic package on the command prompt. The same process was repeated for all 20 patients’ data.
Reference Workflow2 (RW2): DICOM image and RTStructure were converted in NRRD format using Plastimatch 1.8.0 and stored similarly as it is done in Workflow1. Subsequently, radiomic features were extracted and stored in CSV format similarly as it is done in workflow1. The algorithm used for the manual extraction of radiomic features is shown in
Figure 3. All the patient’s radiomic feature data was arranged similarly as it was arranged in the automated extraction of radiomic feature data as described in the automatic extraction section.
Statistical analysis
Our PyRadGUI workflow was compared with manual Reference Workflow1 and Reference Workflow2 as shown in
Figure 3. Interclass correlation (ICC) was calculated for all 1093 radiomic features to compare PyRadGUI workflow and Reference Workflow1 using python code. As we were expecting the same result in PyRadGUI Workflow and Reference Workflow2, we used the EXACT function of
Microsoft Excel 2007 software to compare both the data sets. The EXACT function compares two strings for all the characters, if all the characters are the same in both the strings it gives true as an output otherwise false.
Performance testing of software
The performance of the software was tested for batch processing on the two machines mentioned in
Table 1. CT and PET DICOM data were used in the batches of 10, 20, 50, and MRI DICOM data were used in the batches of 10, and 20. The total time required for individual batch processing was recorded.
Results
We successfully installed and ran the software on the two computers mentioned in
Table 1. All the customizations for radiomic extraction work well on multiple trials. With this software, we were able to perform radiomic extraction by batch processing of up to 100 patients’ CT data on both computers.
Software validation
The ICC value for the comparison of PrGW and RW1 was ICC =0.978 (range: 0.9612-1.0) across all the modalities. The ICC value for the comparison of PrGW and RW1 is shown in
Table 3. All the values (for 1093 radiomic features) in the validation steps for CT, PET, and MRI features for PrGW and RW2 were TRUE, which shows that we were able to extract the same values for all the features for the same data across all the modalities by using this software as confirmed by the manual process.
The table shows the interclass correlation (ICC) value between PrGW and RW1 for CT, PET, and MRI radiomic feature extraction.
Comparison
ICC value (Mean±SD)
1093 features
CT [10patients]
PET [5patients]
MRI [5patients]
PrGW vs. RW1
0.98±0.06
0.72±0.31
0.84±0.23
PrGW vs. RW2
0.94±0.08
0.74±0.52
0.82±0.74
Performance testing of software
On both machines the batch processing performance was found to be satisfactory. The details of run time are shown in
Table 4. Various imaging modalities were used to test the PyRadGUI tool.
Shows the processing time required by both Machines for batch processing of radiomic extraction.
Avg. Runtime per patient (sec)
Modality
Computer system
A batch of 10 patient
A batch of 20 patient
A batch of 50 patient
CT
Machine 1
81
53
75
Machine 2
58
30
31
PET
Machine 1
180
190
180
Machine 2
102
159
112
MRI
Machine 1
75
94
Machine 2
7
7
Discussion
Several licensed and open-source software are available for radiomic extraction, which can extract radiomic features from two- or three-dimensional medical images (
Gotz
et al., 2019;
Szczypinski
et al., 2009;
Zhang
et al., 2015;
Apte
et al., 2018).
Shi
et al. (2019) have developed an Ontology-guided radiomics analysis workflow (O-RAW) using the Pyradiomics package and SITK Python package, which is also able to extract radiomics features from DICOM images and RTSTRUCT in batch processing and store it in resource description framework (RDF) triple store. Another radiomic extraction software, TexRAD, is a licensed package GUI-based system. It has been used by many researchers for radiomic extraction. TexRAD is unable to do the batch processing and it can handle one patient’s data at a time (
Gotz
et al., 2019). This software has capabilities to display and review the image and delineate tumors manually, but another drawback of this software is its inability to use existing delineation (RTStructure) (
Ganeshan
et al., 2008;
Davnall
et al., 2012).
The RaCaT is radiomic software that implemented the IBSI defined feature but is not available as a GUI tool as well as unable to perform batch processing. The PyRadGUI software is a GUI-based tool and it can implement batch processing of DICOM images and RT structure for radiomic extraction. As this software is an extension of the pyradiomic package it inherently implements the IBSI feature definition. It can extract radiomic features from hundreds of patients’ images and RTStructure in batch processing mode and store the result in CSV format. Although we have not compared the delineation converted to NRRD using Plastimatch and 3D Slicer, radiomic feature values comparison show excellent agreement (ICC=0.998±0.012) between the two methods. As our results show, this software calculates radiomics features accurately and reliably. Radiomic extraction from PET and CT images takes a much longer time compared to MRI images, as PET and CT have whole-body images [contains more data] and MRI has regional images [contains less data]. We have used Pyradiomics, open-source software for radiomic extraction in our research infrastructure because this infrastructure can easily be replicated in other research centers. This software works as a plug-in and has no dependencies on the pyradiomic package version, it can be upgraded as and when a new pyradiomic package is available. Customization is the unique feature of this software, which provides flexibility to the user to customize the parameters in the ‘yaml’ file of the pyradiomics package. The ability of our software to customize and extract 1093 radiomic features from medical images in batch processing enables faster processing of radiomic extraction and storage of feature values in CSV format. During the customization, the user can also select a specific group of features to be extracted. The advantage of this software is its GUI and GUI-based customization of extraction, which allows performing the entire task from the GUI console by clicking the available buttons on the console. The CSV format in which this software stores data where each column represents radiomic features and rows represent individual patient’s data makes it easier to be utilized for machine learning. It can also be concatenated with clinical data if required. Log files can be used for identifying any error that occurs during the processing. We can identify the specific data that contains errors and then take corrective action by referring to the error log of each patient's data that is generated and stored in log file. In our existing project, we have also developed artificial intelligence (AI) infrastructure for AI-based research in oncology and PyRadGUI is also an integral part of it. The PyRadGUI can be implemented as standalone as well as part of AI infrastructure for radiomic based research. The portability and easy installation of this software will encourage the radiomic community to use this software and this software can be a valuable addition to radiomic based research infrastructure.
There are also a few limitations of this software like it is unable to display the image before or during the procedure. It requires a DICOM image as well as structure for radiomic processing. Future work will be to test this software on
Linux operating system, add a statistical and prediction analytics module and image segmentation and display module to this tool.
Conclusion
We successfully implemented and validated, PyRadGUI, a GUI-based easy-to-use Radiomic extraction software. This software can easily be implemented on Windows systems. The extracted features using this software are meeting the IBSI standards. We have found this software able to perform batch processing of up to 100 patients and extract radiomic features and store it in ready-to-use CSV format for machine learning. Documentation including the description of how to install and use this software can be found on GitHub (
https://github.com/Bionic-TMH/PyRadGUI).
Data availability
The corresponding author has access to the data created and used in this study for verification. Data sharing on a public repository is prohibited by IRB permission. Only the project partners are permitted to access the data due to confidentiality reasons regarding patient data.
Software availability
Source code available from:
https://github.com/Bionic-TMH/PyRadGUI
Archived source code at time of publication:
https://doi.org/10.5281/zenodo.7524822
License:
Apache License 2.0
Acknowledgement
The authors wish to thank The Netherlands Enterprise Agency (RVO) and MeITy for the Indo-Dutch NWO/MeITy BIONIC and TRAIN project.
ReferencesAertsHJVelazquezERLeijenaarRT:
Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.Nat. Commun.2014 Jun 3;5:4006. Erratum in: Nat Commun. 2014;5:4644. Cavalho, Sara [corrected to Carvalho, Sara].
2489240610.1038/ncomms5006PMC4059926AgyemanAAOfori-AsensoR:
Perspective: Does personalized medicine hold the future for medicine?J. Pharm. Bioallied Sci.2015 Jul-Sep;7(3):239–244.
2622936110.4103/0975-7406.160040PMC4517329ApteAPIyerACrispin-OrtuzarM:
Extension of CERR for computational radiomics: a comprehensive MATLAB platform for reproducible radiomics research.Med. Phys.2018;45:3713–3720.
2989689610.1002/mp.13046PMC6597320CaersJWithofsNHillengassJ:
The role of positron emission tomography-computed tomography and magnetic resonance imaging in diagnosis and follow up of multiple myeloma.Haematologica.2014 Apr;99(4):629–637.
2468811110.3324/haematol.2013.091918PMC3971072DavatzikosCRathoreSBakasS:
Cancer imaging phenomics toolkit: quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome.J. Med. Imaging.2018;5:1.
2934028610.1117/1.JMI.5.1.011018PMC5764116DavnallFYipCSLjungqvistG:
Assessment of tumor heterogeneity: an emerging imaging tool for clinical practice?Insights Imaging.2012 Dec;3(6):573–589. 2012 Oct 24.
2309348610.1007/s13244-012-0196-6.EpubPMC3505569FerlayJColombetMSoerjomataramI:
Estimating the global cancer incidence and mortality in 2018: GLOBOCAN sources and methods.Int. J. Cancer.2019 Apr 15;144(8):1941–1953.
3035031010.1002/ijc.31937GaneshanBMilesKAYoungRC:
Three-dimensional selective-scale texture analysis of computed tomography pulmonary angiograms.Investig. Radiol.2008 Jun;43(6):382–394.
1849604310.1097/RLI.0b013e3181690091GoodwinJNeugentMLLeeSY:
The distinct metabolic phenotype of lung squamous cell carcinoma defines selective vulnerability to glycolytic inhibition.Nat. Commun.2017 May 26;8:15503.
2854808710.1038/ncomms15503PMC5458561GotzMNoldenMMaier-HeinK:
MITK Phenotyping: an open-source toolchain for image-based personalized medicine with radiomics.Radiother. Oncol.2019;131:108–111.
3077317610.1016/j.radonc.2018.11.021GriethuysenJJMFedorovAParmarC:
Computational Radiomics System to Decode the Radiographic Phenotype.Cancer Res.2017;77(21):e104–e107.
2909295110.1158/0008-5472.CAN-17-0339PMC5672828HaiderSPBurtnessBYarbroughWG:
Applications of radiomics in precision diagnosis, prognostication and treatment planning of head and neck squamous cell carcinomas.Cancers Head Neck.2020;5:6.
3239117110.1186/s41199-020-00053-7PMC7197186HaralickRMShanmugamKDinsteinI:
Textural Features for Image Classification.IEEE Trans. Syst. Man Cybern.1973;SMC-3:610–621.
10.1109/TSMC.1973.4309314JohnsonMCI:
The ITK Software Guide: Introduction and Development Guidelines. 2015.2015.LambinPRios-VelazquezELeijenaarR:
Radiomics: extracting more information from medical images using advanced feature analysis.Eur. J. Cancer.2012 Mar;48(4):441–446.
2225779210.1016/j.ejca.2011.11.036PMC4533986LimkinEJSunRDercleL:
Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology.Ann. Oncol.2017;28:1191–1206.
2816827510.1093/annonc/mdx034NiocheCOrlhacFBoughdadS:
LIFEx: A Freeware for Radiomic Feature Calculation in Multimodality Imaging to Accelerate Advances in the Characterization of Tumor Heterogeneity.Cancer Res.2018;78:4786–4789.
2995914910.1158/0008-5472.CAN-18-0125PfaehlerEZwanenburgAJongJRde:
RaCaT: An open source and easy to use radiomics calculator tool.PLoS One.2019 Feb 20;14(2):e0212223.
3078593710.1371/journal.pone.0212223PMC6382170ShiZTraversoASoestJvan:
Technical Note: Ontology-guided radiomics analysis workflow (O-RAW).Med. Phys.2019 Dec;46(12):5677–5684.
3158048410.1002/mp.13844PMC6916323StewartCLWarnerSItoK:
Cytoreduction for colorectal metastases: liver, lung, peritoneum, lymph nodes, bone, brain. When does it palliate, prolong survival, and potentially cure?Curr. Probl. Surg.2018 Sep;55(9):330–379.
3052693010.1067/j.cpsurg.2018.08.004PMC6422355SzczypinskiPMStrzeleckiMMaterkaA:
MaZda—a software package for image texture analysis.Comput. Methods Prog. Biomed.2009;94:66–76.
1892259810.1016/j.cmpb.2008.08.005YipSSLiuYParmarC:
Associations between radiologist-defined semantic and automatically computed radiomic features in non-small cell lung cancer.Sci. Rep.June 2017;7(1):3519.
2861567710.1038/s41598-017-02425-5PMC5471260ZhangLFriedDVFaveXJ:
IBEX: an open infrastructure software platform to facilitate collaborative work in radiomics.Med. Phys.2015;42:1341–1353.
2573528910.1118/1.4908210PMC5148126ZwanenburgA:
EP-1677: Multicentre initiative for standardisation of image biomarkers.Radiother. Oncol.2017;123:S914–S915.
10.1016/S0167-8140(17)32209-0ZwanenburgALegerSVallièresM:
Initiative for the IBS. Image biomarker standardisation initiative.2016;295:328–338.
3215477310.1148/radiol.2020191145PMC719390610.5256/f1000research.142538.r219099Reviewer response for version 1NamdarKhashayar1Refereehttps://orcid.org/0000-0003-0497-6206
University of Toronto, Toronto, Ontario, Canada
Competing interests: No competing interests were disclosed.
The paper has been already reviewed and the current version is in an acceptable stage in terms of writing and structure. As a reviewer, I do not intend to add unnecessary comments. Anything I highlight is for the sake of improving the paper and helping the authors and thus it is open for discussion. Also, the authors are not expected to follow the inefficient tradition of starting every sentence with “we thank the reviewer for their helpful comment”.
Fundamental comments:
The core of the software is PyRadiomics. As it is mentioned in the manuscript, PyRadiomics is a reliable and popular library for radiomics extraction. Hence, choosing PyRadiomics for the back-end of the software is wise. However, the assumption of “PyRadiomics is overcomplicated therefore researchers will use our tool” may be superficial. A) this is not a new problem. Cisco routers have been known for their complex terminals and multiple GUI-enabled solutions have been proposed for them. However, still the terminal is the most popular interaction means. Therefore, you need to select your target users and develop the tool for them. B) with LLMs writing advanced scripts, radiomics extraction using PyRadiomics becomes more and more accessible. This is why I do not expect your paper will have many citations. C) As it is mentioned in the manuscript, there are multiple alternatives. The question is not ICC of the extracted features. Those check points are needed, but they are nothing more than sanity checks. The question is “why PyRadGUI and not Slicer+PyRadiomics extension?” You answered this question partially by highlighting batch processing. You would need more features D) installation of the libraries is always an issue, especially for researchers with limited technical background (which are your target users). Dockerizing the software and selecting a web interface could be a more versatile approach.
Radiomics-based ML pipelines are more comprehensive than radiomics extraction. Authors have tried to include more in the software by adding data-loading. However, this makes the solution project-specific. They should decouple feature extraction so that the software becomes compatible with different settings. What if the starting point is not DICOM files? What if there are no RT Structures? What if NifTi, NRRD, or even 2D JPEG/PNGs are provided?
Technical Considerations: The total number of extracted features can be more than 1093. You may want to read “Open-radiomics: A Collection of Standardized Datasets and a Technical Protocol for Reproducible Radiomics Machine Learning Pipelines” and check
https://openradiomics.org/. Please make sure trimesh is installed.
The quality of the figures is low. It seems in Figure 2, Bin Width is misspelled.
Re. “To start PyRadGUI, GUI_batch_radiomics.py is run on the command prompt.”: A) An executable file could be provided to make the solution more “software” than a tkinter python script. B) the specific file name is introduced out of the context and when the reader has no knowledge of the script structures. Please start the paragraph with introducing the github repository and the scripts and directory structures and functions.
PyRadiomics has numerous settings and hyperparameters, and you have provided access to several common ones. A) mention what they are why you have decided to choose them B) It would be nice to incorporate an “advanced” tab for users with more experience
Writing: I would improve the paper is the use of words such as “many” and “several” was more controlled. It becomes more important when these words are repeated in the same sentence (e.g. ” Several open-source and licensed software packages for radiomic extraction, like IBEX, RaCaT, CaPTK, LifeX or CGITA, Pyradiomics, and TexRad have been developed and used by several researchers in the past”).
Also, choosing “like” as a replacement for “such as” is informal in academic writing.
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?
Yes
Is the rationale for developing the new software tool clearly explained?
Yes
Is the description of the software tool technically sound?
Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?
Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
10.5256/f1000research.142538.r211383Reviewer response for version 1HosseiniSeyyed Ali1Refereehttps://orcid.org/0000-0001-7542-7541
McGill University, Montreal, Québec, Canada
Competing interests: No competing interests were disclosed.
The manuscript presents "PyRadGUI," a GUI-based software tool designed to facilitate radiomic feature extraction from medical images. The software aims to address the challenges posed by existing radiomics tools, particularly the lack of user-friendliness and standardization. The authors provide a detailed description of the software's development, features, and validation, along with potential use cases. The manuscript is well-structured, and the software appears promising. However, there are some areas that require further clarification and improvement.
General Comments:
Clarity and Organization: The manuscript is generally well-organized and clear. However, some sections could benefit from further clarification. For instance, the introduction is lengthy, and while it provides useful background information, it could be more concise. It's essential to maintain a balance between providing context and keeping the reader engaged.
Abstract: The abstract effectively summarizes the key points of the manuscript. However, it would be helpful to include specific results or findings from the study, as this would give readers a quick overview of the software's performance.
Keywords: The chosen keywords are appropriate and relevant to the topic.
Ethical Considerations: The authors mention ethical approval for their study but do not provide any information regarding patient consent or data privacy. It would be beneficial to clarify how patient data were handled, especially regarding privacy and consent, as this is an essential ethical aspect of medical research.
Software Availability: The manuscript mentions the availability of the software on GitHub and provides a DOI link. However, it would be helpful to include a brief section in the manuscript explaining how readers can access and use the software.
Specific Comments:
Introduction: The introduction provides a comprehensive overview of radiomics and its significance. However, it could be condensed to maintain reader engagement. Also, it would be useful to include a clear statement of the study's objectives and the problem the software aims to address.
Methods: The methods section is detailed and provides a good understanding of how the software was developed and tested. However, there are several grammatical errors and awkward sentences throughout this section that need editing. For example, "Although we have not compared the delineation converted to NRRD using Plastimatch and 3D Slicer, radiomic feature values comparison show excellent agreement (ICC=0.998±0.012) between the two methods." This sentence needs clarification.
Results: The results section provides valuable information regarding software validation and performance testing. However, it could be enhanced by presenting the results in a more organized and visually appealing manner, such as tables or figures.
Discussion: The discussion section effectively highlights the advantages of PyRadGUI and its potential applications. It would be beneficial to address the limitations and future improvements in more detail. For instance, the authors mention future work to add statistical and prediction analytics modules and image segmentation and display modules, but elaborating on these points would provide more insight.
Conclusion: The conclusion appropriately summarizes the key findings and contributions of the study. However, it would be helpful to reiterate the software's main advantages briefly.
Figures and Tables: Figures and tables are crucial for presenting results and data. Consider adding more visual elements to help readers grasp the key points more easily.
Language and Grammar: The manuscript contains numerous grammatical errors and awkward sentences. Careful editing and proofreading are necessary to improve readability.
Software Citation: Encourage authors to cite the software appropriately in their publications, as this is important for acknowledging the work of software developers.
Software Accessibility: Provide clear instructions on how to access and use the software. This should include information on system requirements, installation, and a brief tutorial if possible.
In summary, the manuscript presents a promising software tool for radiomics, but it requires editing for clarity, grammar, and organization. Additionally, providing more information on ethical considerations and access to the software would enhance the manuscript's completeness.
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?
Yes
Is the rationale for developing the new software tool clearly explained?
Yes
Is the description of the software tool technically sound?
Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?
Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
10.5256/f1000research.142538.r211376Reviewer response for version 1LeNguyen Quoc Khanh1Refereehttps://orcid.org/0000-0003-4896-7926
Taipei Medical University Digital Library Consortium, Taipei City, Taipei City, Taiwan
Competing interests: No competing interests were disclosed.
1. The authors compared radiomic feature values between different software tools. However, they lacked specific details on the metrics used for this comparison. The readers would benefit from knowing the statistical methods employed and the criteria for determining "excellent agreement" (e.g., ICC=0.998±0.012).
2. The software limitations mentioned, such as the inability to display images before or during the procedure, may be crucial for users. Elaborating on these limitations and their potential impact on the usability and applicability of the software is necessary.
3. The use of a DICOM image as well as structure for radiomic processing raises concerns about the software's flexibility. A discussion on the potential limitations imposed by this requirement and the implications for users dealing with non-DICOM data should be included.
4. The intention to test the software on a Linux operating system is mentioned as future work. It would be helpful to provide insights into the challenges and considerations involved in adapting the software to different operating systems, as this can significantly impact its accessibility.
5. The authors should provide more details on the specific functionalities and capabilities of statistical and prediction analytics module.
6. Similarly, the mention of adding an image segmentation and display module is a significant enhancement. The authors should discuss the rationale behind this addition and its potential impact on the software's utility.
7. While the software is mentioned as a valuable addition to radiomic-based research infrastructure and as part of an AI infrastructure, more details on how PyRadGUI integrates with AI models, its compatibility with different machine learning frameworks, and examples of successful integration with AI-based projects would strengthen the study.
8. The study could benefit from insights into the user experience, including user feedback, challenges faced during implementation, and any user training required.
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?
Partly
Is the rationale for developing the new software tool clearly explained?
Partly
Is the description of the software tool technically sound?
Partly
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?
Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?
Yes
Reviewer Expertise:
Radiomics; artificial intelligence
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.