Automated segmentation of endometriosis using transfer learning technique [version 1; peer review: awaiting peer review]

Background: This paper focuses on segmenting the exact location of endometriosis using the state-of-art technique known as U-Net. Endometriosis is a progressive disorder that has a significant impact on women. The lesion-like appearance that grows inside the uterus and sheds for every periodical cycle is known as endometriosis. If the lesion exists and is transferred to other locations in the women’s reproductive system, it may lead to a serious problem. Besides radiologists deep learning techniques exist for recognizing the presence and aggravation of endometriosis. Methods: The proposed method known as structural similarity analysis of endometriosis (SSAE) identifies the similarity between pathologically identified and annotated images obtained from standardized dataset known as GLENDA v1.5 by implementing two systematic approaches. The first approach is based on semantic segmentation and the second approach uses statistical analysis. Semantic segmentation is a cutting-edge technology for identifying exact locations by performing pixel-level classification. In semantic segmentation, U-Net is a transfer-learning architecture that works effectively for biomedical image classification. The SSAE implements the U-Net architecture for segmenting endometriosis based on the region of occurrence. The second approach proves the similarity between pathologically identified images and the corresponding annotated images using a statistical evaluation. Statistical analysis was performed using calculation of both the mean and standard deviation of all four regions by implementing systematic sampling procedure. Results: The SSAE obtains the intersection over union value of 0.72 and the F1 score of 0.74 for the trained dataset. Conclusions: The proposed SSAE approach identifies the affected region using U-Net architecture and systematic sampling procedure.


Introduction
Endometriosis is a common gynecological problem that occurs in women of aged 18 to 50 years. 1 The lesion-like structure that underlines the uterus and other surrounding regions is referred to as endometriosis. Endometriosis along with the uterus affects other regions, including ovaries, peritoneum, and multiple locations known as deep infiltrating endometriosis. 2 The most common practice for recognizing endometriosis is laparoscopy.
Deep infiltrating endometriosis (DIE) is a serious concern among women of reproductive age. The DIE affects multiple regions including the uterus, ovaries, gall bladder, liver, and other abdominal regions. DIE also penetrates approximately 4 to 5 mm into the tissues. 3,4 DIE is unpredictable at earlier stages and poses a great challenge for gynecologists.
The greatest problem with endometriosis is unbearable abdominal pain and infertility, which in turn leads to psychological depression and serious health issues includes dysmenorrhea, severe pelvic pain, dyspareunia, frequent urination. 5 The advanced stages of endometriosis may lead to endometrial cancer, leading to further complications. 6 Computerized diagnosis helps radiologists in identifying the exact location and also precisely recognize abnormalities. Various methods exist for identifying endometriosis including magnetic resonance imaging (MRI), transvaginal scanning (TVUS), and laparoscopic surgery. Among all laparoscopic surgery is considered the best practice to identify the exact location of endometriosis. 7 The staging depends on the location and aggravation of the lesion spread across multiple locations. According to the stages of endometriosis, endometriosis is classified as (a) minimal endometriosis, (b) mild endometriosis, (c) advanced or deep endometriosis.
The proposed work implements the segmentation process using predicted pathological images from earlier work and the corresponding annotated images from the dataset. The proposed work known as structural similarity analysis of endometriosis was validated using two approaches. The first approach was semantic segmentation using U-Net and the second approach used statistical evaluation.
Deep learning serves as a decision support system for radiologists. Deep learning is a state-of-the-art technique for recognizing the affected areas. Among the various deep learning networks, convolution neural networks (CNNs) play a vital role in processing biomedical images. CNNs perform various tasks including classification, prediction, localization, and segmentation. The CNN implements the segmentation process to recognize the pattern and identify the object. Segmentation analyzes the super pixel of each image and classifies them based on various criteria. Segmentation can be classified majorly into two types: a) instance segmentation, b) semantic segmentation. The combination of these two segmentations is known as panoptic Segmentation. 8 Instance segmentation considers multiple objects in the same class in various instances. Semantic segmentation treats multiple objects in a single class as a single instance. Medical images invoke a semantic segmentation. Image segmentation can be implemented effectively using transfer learning architectures including Mask R-CNN, fully convolution neural network (F-CNN), and U-Net architecture. 9 In this study, segmentation was used to predict the exact location of the endometriosis by implementing pixel classification.
Semantic segmentation using the U-Net architecture was identified as a prominent segmentation process for biomedical images. U-Net is a transfer learning architecture that invokes CNN for implementing pixel classification. The U-Net architecture applies down sampling to extract more features from an image. 10 Statistical analysis plays a predominant role in the validation and verification of medical datasets. The mean and standard deviation were used to analyze each value in the dataset and evaluate the difference in values.

Related studies
Literature studies used to analyze the similarity between two datasets based on deep learning techniques are discussed below.
Segmentation plays a vital role in recognizing abnormalities. Suggested gonadotrophin releasing hormone is used to improve pregnancy rates in women with and without endometriosis. 11 Endometriosis was predicted using the transferlearning approach. ResNet50 classifies pathological and non-pathological endometriosis with an accuracy rate of 91%. 12 Attribute description was developed through "pattern recognition and image processing techniques". Ultrasonic images are used for extracting features, segmentation of images and so on. 13 The segmentation of medical images was implemented using deep-learning techniques. A comparison was performed using supervised and weakly supervised learning techniques. 14 Semantic segmentation in biomedical images was analyzed. Traditional segmentation loses pixel quality, whereas semantic segmentation processes preserve pixel quality through down-sampling process. Semantic segmentation invokes a CNN to maintain the pixel quality. 15 The process known as automatic augmentation was implemented. The process involves the following steps: a) preprocessing of images b) detection of features c) mask generation d) mask processing and e) segmentation. 16 Introduced image segmentation with discriminant analysis of dental radiographs. Because annotation was performed manually, the similarity between the overlapped region and the corresponding samples is preserved. 17 The image segmentation was implemented on gastrointestinal images and later applied Machine learning algorithms for transparency. 18 Hybrid segmentation known as a 3D residual network was used for identifying tumors in the kidney and liver. The Squeeze-excitation block along with the 3D RN, was used for segmenting the tumors. 19 A convolution neural network (CNN) was used to segment skull regions from computed tomography (CT) images. The automated CNN outperforms well with a mean F1 score of 0.92 and a mean deviation of 1.2 mmAE1.75 mm. 20 Various architectures exist for semantic segmentation to identify abnormalities. Various learning architectures exist for implementing segmentation. The U-Net architecture plays a predominant role in medical imaging. The U-Net module was implemented for segmenting lung nodules. CT scan images of the lung were used for segmentation with an F1 score of 0.82 and an intersection over union (IOU) value of 0.752. 21 A novel method was introduced known as the U-NET transformer that encodes the sequence of input and captures global multiscale information. Performance was evaluated using brain tumor and spleen segmentation tasks. 22 The ensemble machine learning model was implemented for evaluating endometriosis. CA-CNN, DFKZnet, and 3D U-Net was adopted to validate the performance of the ensemble learning model. 23 The ovaries were classified automatically based on K-means clustering and an artificial neural network using texture features. Three features autocorrelation, average of sum, variance of sum were used for ovarian detection. 24 U-Net segmentation was implemented to identify uterine diseases using the MRI images. The mean F1 coefficient was 0.84 and the mean absolute distance was 18.5. 25 Mi U-Net is a state-of-art technique that helps segment kidney stones from medical images. The Mi U-Net outperformed well in terms of qualitative and quantitative metrics. 26 U-Net and DeeplabV3 segmentations were implemented to identify abnormalities in fetal echocardiograph. Among the two types of segmentation DeeplabV3 performs well and evaluation was performed based on the IOU, F1 Score. 27 Docker-based deep learning outperforms other methods for segmenting biomedical images. DDeep3M works effectively on smaller and larger datasets. 28 Mask RCNN segmentation was implemented for laparoscopic gynecological images. The model performed well with an accuracy of 95%. 29 From the literature review, various segmentation processes have been identified to recognize the aggravation of endometriosis. Segmentation was implemented by detecting features, K-means clustering, supervised machine learning algorithms, and neural network algorithms. Manually annotated images were also used to identify the overlapping regions, where pixel classification was not clear. A gap exists in the selection of annotated images and implementation of a suitable segmentation process. Hence the SSAE was implemented using two approaches to analyze the similarity between pathological images from earlier studies and their corresponding annotated images. Semantic segmentation along with U-Net plays a vital role in identifying abnormalities. SSAE implements a semantic segmentation process to recognize the location and aggravation of endometriosis. The SSAE uses both pathologically proven endometriosis images from an earlier work and the corresponding annotated images to perform the segmentation process. 30 The statistical method was adopted as yet another validation procedure for analyzing similarities.

Methods
This study took place in Jan 2022. Endometriosis was predicted in four regions of the reproductive system including ovary, uterus, peritoneum, deep infiltrating endometriosis (rectum, gall bladder). Laparoscopic images and annotated images of endometriosis were obtained from the standardized GLENDA v1.5 31 dataset. The dataset contains Laparoscopic images of both pathological and non-pathological identified endometriosis regions. Pathological lesions identified in laparoscopic images were used for segmentation process. In the proposed method 373 laparoscopic images and 628 corresponding annotated images were used for segmentation process.
A recognized pathological report of endometriosis was selected for segmentation. The images were classified as pathological images and their corresponding annotated images. Table 1 shows the number of endometriosis images by affected region.
Endometriosis is recognized at different location in a single pathological image. The pathologically affected images were identified from the dataset 12  The identified pathological and annotated image datasets were given as input for SSAE. SSAE was effectively implemented using semantic segmentation and statistical analysis. The performance of semantic segmentation was validated using Intersection over union (IOU) and F1 score. Similarly, the statistical performance was evaluated using the mean and standard deviation.

Semantic segmentation of endometriosis
The semantic segmentation process ascertains the spread of endometriosis at multiple locations identified from pathologically proven images and mapped with annotated images. U-Net is a state-of-art technique for semantic segmentation to identify the images based on pixel classification. 32 Semantic segmentation is a cutting-edge technology for recognizing the exact location of biomedical images. The steps involved in segmenting the laparoscopic images were as follows: 6) Performance validation of pixel classification using IOU, F1 score, IOU threshold, Jacard-Coefficient.
The steps are illustrated in Figure 2 as follows: Ovary 53 54 Deep infiltrating endometriosis 55 59 The training and test data was split as 70% and 30% for training and testing data. The segmentation process was implemented on the preprocessed images. The segmentation process implements pixel classification to ensure the aggravation of endometriosis at various locations. Various segmentation architecture processes have been proposed. The most effective U-Net architecture was implemented. U-Net is a convolution neural network architecture that was mainly developed to identify the precise location of the infected area.
The U-Net model was implemented with the following parameters: a) filter size as 64, b) Adam optimizer, c) loss function as binary cross entropy d) softmax as activation function. The training model was implemented with 20 epochs with 50 steps per epochs. From the targeted output, it was possible to identify the intensity of endometriosis in every region using pixel classification. 33 Table 2 lists the various hyper parameters identified for execution. The model is available from GitHub and is archived with Zenodo. 47 The performance of the proposed system was evaluated using various metrics as follows:

Intersection over union (IoU)
The IoU 34 is calculated as the ratio of the overlapped area between the predicted and ground truth to the overall area between the predicted and targeted areas.
Where ∝∩ indicates the overlapped area and ∝∪ indicates the overall area.

F1 Score
The F1 Score 35 is calculated as the ratio of the overlapped area multiplied by two to the total number of pixels in both images.
Where ∝∩ indicates the overlapped area and ρI denotes pixel of both the images.

Statistical analysis of endometriosis
The pathologically identified datasets and annotated datasets were used as inputs for statistical analysis. 46 The statistical analysis was performed in Excel 2013. Systematic random sampling was performed to validate the pathologically identified images with annotated images. The pixel intensity of endometriosis affected four regions namely the uterus, ovary, peritoneum, and deep infiltrating (rectum) were calculated as follows: To perform systematic sampling, population size, sample size, and starting point were calculated as follows: Where K is the systematic sampling value, N is the number of images and n is the sample size taken. The starting point (Ø) was calculated based on the systematic sample value.
The random weight value is calculated using starting point value as follows: where γ represents the random weight value calculated for both pathological and the annotated images.
By multiplying the random weight value with the corresponding identified pixel value (ρlÞ of laparoscopic images, the desired sampling value was obtained as follows.
Similarly, for annotated images, the sampling value was calculated as follows: where (ρAÞ represents the pixel value of annotated images. δL and δA represent the sampling values of pathologically identified and annotated images for the uterus, ovary, peritoneum, and deep infiltrating.
The mean and standard deviation of the sampling values 30 for both pathologically identified and annotated images that included all four regions were calculated as follows: Where λ Li and μ Li represent the mean and standard deviation for laparoscopic images of all four regions. Similarly for annotated images, Where λAi and μAi represent the mean and standard deviation for annotated images of all four regions.

Results
The pathologically identified images and their corresponding annotated images of endometriosis were considered as inputs for segmentation. The four regions including the ovary, uterus, peritoneum, and deep infiltrating endometriosis are involved in pixel classification. The pathologically identified images and annotated images were pre-processed. These preprocessing include rotation and shifting to increase the training size of the images. As a result, the pre-processed images were fed as input to the segmentation.
Segmentation analyzes the pre-processed images pixel-by-pixel level. Semantic segmentation involves the classification of each pixel of an image into all classes. The U-Net architecture 36 implements a down sampling technique to encode the input images to attribute representation at multiple levels.
The pathological 37 and annotated images were provided as input for semantic segmentation using U-Net architecture. As a result of the segmentation process involving various parameters the ground truth area was identified which predicts the region of occurrence as a segmented output as illustrated in Figure 3.
The hyper parameters as mentioned were executed in the colab environment and the total number of parameters executed was 31,055,492 with trainable parameters as 31,043,716 and non-trainable parameters as 11,776.
A careful investigation was performed by selecting the network parameters. Trials were carried out to identify the optimized parametric value. The filter size was identified based on trials with sizes as 16, 32 and 64 and it was found that a filter size 64 was the most optimized parameter values. The overlapping region was not sufficient for the lower bound values of filter sizes 38 16 and 32. Similarly, the optimizers used for U-Net architectures were RMSprop, SGD, and Adam. 39  The prediction area was evaluated using the following performance metrics: a) IOU b) F1 Score) with a filter size of 64, epochs of 20, and Adam optimizer was selected. Based on the execution with the identified hyper parameters the ground truth was predicted and the output image obtained was depicted in Figure 7.
A comparison was made between the training IOU and validation IOU along with the epochs. The best identified parameters were executed and graph was illustrated in Figure 8.
The proposed methodology for segmenting the endometriosis to identify the similarity of pixels between pathological and annotated images was compared with other architectures. The various architectures used for analyzing the pathological and annotated images were fully conventional network and Mask RCNN. These architectures were compared based on their performance using overlapping regions. Table 3 presents the comparison was based on various metrics and comparison was illustrated in Figure 9.
The proposed SSAE method was compared with other existing methods, where the SSAE method performs well in terms of Intersection over Union and F1 Score. The comparison is illustrated in Table 4.
In addition to the segmentation process, the intensity of endometriosis was identified using statistical analysis. 40,41 The pixel intensity of affected regions was identified for both pathological and annotated images. Random sampling was applied to both the pathological images and annotated image pixel values. From the obtained values, the mean and standard deviation were calculated for both the pathological and annotated images for all four regions was listed in Table 4.

Discussion
The empirical analysis was performed to identify the hyper parameters for segmenting the exact location. Based on analysis, the Adam optimizer performs well for overlapping regions based on the performance of intersection over union. The next parameter was the loss function. The loss functions used in U-Net architectures are cross-entropy loss, focal loss, and IoU Loss. 42 Cross-entropy performs well based on the overlapping region. The sizes of the epochs were 10, 20 and 30. It was identified that intersection over Union was obtained when the epoch size was 20. When the epoch size was 10, the performance of the IOU was not up to the level, whereas when the epoch size was 30, outliers were found to be detected.
The filter size was analyzed based on the performance of the SSAE method. When the filter size was 16, the obtained IOU was 0.48 and the F1 Score was 0.56. In addition, when the filter size was 32, the IOU was 0.65 and the F1 Score was 0.68. Finally, the best overlapping occurs when the filter size as 32 with IOU of 0.72 and F1 Score of 0.74 ( Figure 4).  Figure 6.
The F1 Score is another method used to evaluate the pixel classification performance. The training and validation F1 Score was compared with those of epochs and graphical illustrations are presented in Figure 8.  Among all other architecture, structural similarity between pathological and annotated images was implemented with higher performance using U-Net with an IOU of 0.72 and F1 score of 0.74, where the fully conventional network contains an IOU of 0.68 and an F1 score of 0.74. In addition, the performance of Mask RCNN obtains an IOU of 0.71 and an F1 score of 0.73. A graphical representation of performance analysis of the various architectures is illustrated in Figure 9.
The proposed SSAE method was compared with existing methods that invoke segmentation process for identifying various disorder. The first method was proposed by Leibetseder. 43 In this approach laparoscopic images of endometriosis was segmented using FCNN and Mask RCNN. This method obtained an IoU of 0.7 and F1 score of 0.73. Similarly, Giusti 44 uses Magnetic resonance images for segmenting deep infiltrating endometriosis. The method obtains an IoU of 0.68 and F1 score of 0.7. Next method Ma 45 uses Magnetic resonance images for segmenting gall bladder where the IoU obtained was 0.66 and F1 Score was 0.68. The SSAE method outperforms well as listed in Table 4.
The calculated values for the four regions are listed in Table 5. In the pathological images, the mean value for the uterus was 0.559 which was closer to the mean value of the uterus in the annotated images. In addition, the mean value of the peritoneum in the pathological images was 1.188 which was closer to the annotated image mean value of the peritoneum. The next region's ovary mean value in the pathological image is 0.861 which is closer to the mean value of Ovary in the annotated images. Finally, the DIE mean value is 0.85 was closer to the mean value of DIE in the annotated images.
Similarly, the standard deviation of the uterus in a pathological image is 0.040, which is closer to the standard deviation of uterus in annotated images is 0.056. In addition, the standard deviation of the peritoneum in the pathological image is 0.0770 which was closer to annotated image standard deviation of peritoneum is 0.077. The standard deviation of the next region of the ovary in the pathological image is 0.056 which is closer to the standard deviation of ovary in the annotated images as 0.080. Finally, the standard deviation of DIE is 0.0544 which was closer to the standard deviation of DIE in annotated image of 0.060. Among the four regions, peritoneum has the major impact that was identified from the mean value obtained.

Conclusion
Endometriosis is a disease that affects 1/15th of women in the reproductive age groups. The proposed SSAE system evaluates the aggravation of endometriosis at distinct locations namely the uterus, ovary, peritoneum, and rectum from pathologically proven and corresponding annotated images. The proposed work invokes the U-Net architecture for segmenting the endometriosis-affected regions for pixel-level classification. In addition to the segmentation process, system sampling was performed using the intensity of the pixel values from both the pathological and annotated images. Means and standard deviations were calculated as a result of the sampling process. The mean and standard deviation obtained for each region in the pathological images were similar to the mean and standard deviation of the annotated images. The statistical value obtained for the peritoneum was 1.188AE0.0773 for pathological images which was similar to the value obtained for annotated images 1.2142AE0.0770. Similarly, for the uterus the statistical value was 0.559AE0.0404 in the pathological images as 0.566AE0.0532 in the annotated images. Ovary the value in pathological images as 0.8613AE0.0566 was identical to the statistical value of annotated ovarian images as 1.0921AE0.0806. Finally, the peritoneum obtained the statistical value of 0.859AE0.054 in the pathological images which was similar to annotated image value of 1.0040AE0.0606. The proposed system obtains the IOU of 0.72 and an F1 score of 0.74.

Source data
The standardized Endometriosis dataset was obtained from Glenda V1.5. 31 The dataset holds around 25000 both pathological, non-pathological images, and annotated images. The dataset consists of four labels: Ovary, Peritoneum, Uterus and Deep Infiltrating Endometriosis. The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com