Classification of Human Sperms using ResNet-50 Deep Neural Network

—Infertility is a disease which scientists around the world are concerned with. The disease of infertility also is a worldwide health concern of many people in the community. The andrologists are continually searching for further developed techniques for any related problems. The intracytoplasmic sperm injection (ICSI) method is a widely recognized strategy for accomplishing pregnancy and considered as one of the best methods for infertility treatment worldwide. Choosing the best sperms are done using the vision through the specimen which is reliant on the abilities and the cleverness of the embryologists and as such inclined to human errors. Subsequently, a system that detects the normal sperms automatically is required for speedy and more precise outcomes. Deep learning approaches are usually effective for classification and detection purposes. This paper uses the Residential Energy Services Network (ResNet-50) deep learning architecture to recognize human sperms after classification of human sperm heads. The ResNet-50 proposed model achieved an accuracy of 96.66%. This proposed model demonstrated its efficiency at the detection of healthy sperms. The healthy sperms are used for the injection into eggs by the andrologists who always look for easier and more advanced methods in order to increase the success rate of ICSI process.


INTRODUCTION
Infertility is the failure of couples to accomplish pregnancy after twelve months despite having sex regularly [1]. The issue of the most couples around all countries is the infertility, around 30-40% of them associated with irregularities of male parameters as reported by WHO [2]. The main mission in the ICSI technique is the determination of the optimum sperms which can be used for fertilization of eggs. Embryologists select healthy sperms relying on the sperm shape by visual evaluation. This operation is difficult, tiring and prone to human errors. So, many researchers tend to AI approaches for finding better solutions for the detection of healthy sperms automatically.
The motivation of the paper comes from the need of laboratory technicians for automatic program to perform selection for healthy sperms that can be injected into oocytes. There are many precautions and conditions for the examination of sperm cells. The oversight processes about principles of quality, observation, testing, handling, protection, and storing of sperms are usually based on WHO guidelines [2]. Additionally, conditions in the space of assessment, capturing and the specifications of an equipment that influence the vision of semen sample by the laboratory technician for the assessment. There is an extra factor influencing the precision in visual assessment which is the cleverness of the andrologists regarding other parameters that affect the sperm selection. To decrease the time of examination of sperms, computer vision and the processing of images have been utilized. Some previous studies did not prove their efficiency because of staining factors, fundamentally it is useful for the research purposes with neural network architectures that are designed using reinforcement learning [18], [19]. Also, there are many algorithms that are used for this purpose and will be discussed in detail next in the literature review section.
The examination of semen sample mostly performed according to the morphology [24]. Some consultations from the andrologists are needed when the examination of morphology is completed successfully. Subsequently, the right decision is taken about the healthy sperms that are suitable for injection process. According to the morphology also, computer-aided sperm analysis is done using the artificial intelligence [20].
The morphology of sperms is the most important parameter for the assessment of the sperm which defines the range of integrity of the shape and size of sperms. There are many factors that affect the quality of human semen [23]. The morphology of sperms is assessed under microscope after the general test of semen which also indicate the percent of morphology abnormalities [21], [22].
New advancements in optics and robotized microscopy have made experts ready to view and record huge number of images. The WHO guidelines are important for the selection of sperms and for the male infertility recognition. It is essentially to inspect the sperms for checking their validity for the injection [25]. The analysis of sperms is usually done according to the morphology and the motility of sperms.
The assisted reproductive technology (ART) is needed for the recognition of the best ways for the finding the healthy sperms for raising the rates of fertilization, and live birth. This system proved its advantages for getting good results with better accuracy for obtaining the normal sperms, which are referring to the normal sperm heads, which can be utilized in ICSI process after the detection using this system. The healthy sperms are used for the injection into eggs by the andrologists and then the fertilized egg is transferred into the uterus. One of the advantages of this study is its ability to deal with nonstained images that is similar to the captured images from microscope. The studies that deal with stained images are not www.ijacsa.thesai.org applicable because the stain is harmful for the human sperms and make them not suitable for the injection process.
Examination of images from in-vitro tests are normally restricted for andrologists at labs and andrology centers where they are responsible for the sperm processing with special tools. Besides, their perceptions follow particular precautions and rules and the semen microbiome has an impact on the infertility and affect the function of sperms [3]. Currently, the manual selection for healthy sperms can cause problems and it cannot be as accurate as automatic systems for sperm selection. On the other side, the automatic systems are labour saver, and it is not necessary for laboratories to provide skilled technicians for sperm selection in the intracytoplasmic sperm injection process. These reasons motivate us to propose a method which provides automated detection for the best sperms that can be used for increasing the success rate of ICSI process and for getting higher pregnancy success rate.
The main parts of the paper are arranged as follows: Section II explains the literature review for this paper. Section III explains the material and methods, showing the dataset, and describes the stages of preprocessing used in this paper. Section IV describes the proposed model for healthy sperm detection. Section V shows the results of this model. Section VI presents the conclusions of this research and refers to the fields of future work.

II. LITERATURE REVIEW
Many studies about the analysis of sperms automatically using deep learning and machine learning that is important for the process of the intracytoplasmic sperm injection. Jason et al. presented a model for sperm classification of sperms by VGG16 architecture with high accuracy [4]. Soroush et al. introduced a model for the assessment of sperms with three parts [5]. Abolghasem et al. presented a method for sperm recognition with tail and neck with accuracy of 93.2% [6]. Miahi et al. improved an architecture for vacuole abnormality recognition with an accuracy of 91.66% [7]. Prabaharan et al. presented a CNN network for the detection of abnormal sperms with abnormal dimensions with 98.99% of accuracy and approved the efficiency of the program [8]. P., Zuhdi et al. [9] introduced DeepSperm, a profound brain network that utilizes a particular discovery layer to identify little items. The creators expanded the info goal of the organization, utilized a dropout layer and played out an information increase procedure in view of immersion and openness changes. The acquired outcomes beat the cutting-edge regarding accuracy, speed and computational calculations. S. Hicks et al. proved the importance of deep learning in the processing of videos of movable sperms. They used three videos in which labelled manually by the andrologists, the labels were bounding boxes for the egg, catheter and sperm. YOLOv5 model was used for tracking and detection of sperms. The three used videos were separated as two videos for training and one video for validation. The accuracy of eggs' tracking reaches 92% and for ICSI pipette reaches 94%. The author concludes the importance of deep learning for the detection of sperms for performing successful intracytoplasmic sperm injection process [10]. Ruth Marín et al. proposed the deep learning for and studied the effectiveness of transfer learning for the segmentation of sperms. They used public dataset (SCIAN-SpermSegGS) in which evaluated using Mask RCNN and Unet. The U-net model reaches 95% of segmentation for sperm head which consider a promising result for using the computer assisted systems for this purpose [11]. Lee et al. developed an algorithm for the detection of the human sperms with machine learning using microsurgical testicular sperm extraction utilizing bright-field microscopy in which the sperms were collected from healthy men. The algorithm used the CNN based on the U-Net architecture. In this study, the algorithm achieved 91% PPV and this make the rare sperms can be detected with biopsy samples using machine learning [12].
Raffael Golomingi et al. presented automated deep convolutional network the deep learning VGG19 was used for the implementation with accuracy reaches 93%. This method used labelling with bounding boxes for the detection of sperms. This study shows its success on images acquired by the optical microscope [13]. Aristoteles et al. used deep learning approaches for healthy sperms detection using video tracking using deep learning with result 90.31% average precision sperms and the system detects the non-sperm parts automatically rather than the manual methods that require more time and high costs [14]. Ahmad Mashaal et al. proposed a system for the recognition of healthy sperms using VGG16 deep learning model and used Otsu's thresholding method for the segmentation of sperm head and other methods for the enhancement of images with accuracy reached 97.92% [17].

III. MATERIAL AND METHODS
This section is divided into two parts, description of sperm head dataset, preprocessing methods of sperm head images with showing the impact of these methods on images. In the first part, all information about the dataset. In the second part, the main preprocessing methods for images will be explained.

A. Dataset
The Dataset is free for using, used by McCallum et al [15]. The dataset was processed before using, and then classified into normal and abnormal sperm heads with the aid of andrologists. Dataset is divided into test dataset, validation dataset and training dataset. The dataset contains 1200 images which divided into 240 images for testing, 240 images for validation, and 720 images for training, the dataset divided into 2 classes as healthy and unhealthy sperm as represented in Fig. 1.

B. Preprocessing
Image preprocessing through this research is explained as the following: 1) Image denoising: In this step, the median filter is used for removing the salt and pepper noise for all images for making the images more suitable for the proposed method.
2) Image normalization: This method is used for changing the range for values of pixel intensity in image for getting a better contrast image, this process is used for adjusting the contrast of sperm head images as shown in the image Fig. 2. The linear normalization of an image is represented as the following formula: (1) where O is the output channel and it is the output value which is calculated, I is the input channel, n is the lowest value for pixel intensity and m is the highest value for pixel intensity. 3) Image augmentation: Data augmentation is a method used usually for changing the shape of images and make them little different using versatile parameters that modify the images for increasing the number of training images. Data augmentation of images is done while the training of sperm images. This is performed by ImageDataGenerator class of deep learning Keras library with using different augmentation parameters. The data augmentation is so important for increasing the accuracy of validation and training curves and for decreasing the loss of validation and training curves while training through epochs. The data augmentation contribute to minimize the overfitting.

IV. THE PROPOSED RESNET-50 MODEL
The architecture ResNet-50 is one of the powerful deep neural networks which is used for classification tasks [16]. The ResNet-50 is used in this paper for classification of sperm heads for the best sperm detection that can be utilized in the injection into oocytes during performing the ICSI process. The convolution process is the method in which the two arrays of same or different sizes are multiplied to be merged giving third result, but it is necessary that the two arrays have the same dimensions. The mask or the matrix of convolution used may be called ‗Kernel' and it is used to move over the convolution getting the required information with fewer dimensions and help in the convolution operation.
We can manage the image using this operation and get results with many processing operations for images including image sharpening, image blurring, controlling the contrast of the image and decreasing and increasing the brightness of the image.
The resulted image of this process is resulted from the rolling of the mask over all pixels of image and the final result of the multiplications are combined together and a particular calculation operation is applied resulting new pixel value that takes place in the new image and these steps are repeated for all pixels.
In this study, the better sperms can be identified utilizing ResNet50. In the beginning, the images were collected, some operation methods are important before accessing by ResNet-50. The proposed ResNet-50 model was effective in this process and it is composed of layers of convolution followed by some layers as shown in Fig. 3. The salt and pepper noise was eliminated by median filter and the training and evaluation of model should be done. Therefore, any unknown image will go through preprocessing steps after insertion for the detection. The preprocessing stages contribute to increasing the rate of success of this model then the success of the detection method with high rates. The convolution algorithm is necessary to be implemented using the convolution equation as the following: (2) where C is the convolution kernel of size x, O is the feature map of the output, F is the feature map of the input and is the size of image.
Convolution steps can be described as:  Multiplying the filter with the input image using the convolution equation.
 Multiplying each element with the element in the position then the results will be summed producing one output value.
 These steps will be reiterated by sliding the filter over the image for obtaining the output.
Average pooling in this proposed model is used for down sampling feature maps by computing the average value in each patch of a feature map. Fully connected is a layer which has multiple neurons. Dropout layer and batch normalization used for avoiding the overfitting problems. The sigmoid function is the output function and it is used for sperm head detection. The sigmoid function can represented as the following, where is the sigmoid function of a variable x The rectified linear unit (ReLU) is an activation function that makes the output like the input directly if it is positive, otherwise, makes the output zero. The ReLU activation function can be given as the following: (4) where is the ReLU activation function of a variable x. www.ijacsa.thesai.org

V. RESULTS AND DISCUSSION
The results of this proposed model demonstrated their effectiveness for the detection of normal sperms that can be utilized for the ICSI process with high accuracy. The proposed ResNet-50 model is an efficient deep learning model for classification of images and proved that in this study. In this paper, we got 165 TP value, 5 FN value, 3 FP value and 67 TN value as shown in the confusion matrix in Table I. The proposed model proved its efficiency with high accuracy reaches 96.66%, sensitivity equals 97.06%, precision equals 98.21%, Specificity equals 95.71% and F1 Score equals 97.63% as shown in Table II according to the formulas [5]- [9]. The accuracy considered as the fraction of total results that is predicted correctly to the overall number of samples. The precision is known as the proportion of values of true positive to overall predicted positive values, the sensitivity is the proportion of true positive values to total actual positive values, and the specificity is the proportion of values of true negative to the overall negative values. The detection results of this model are Normal or Abnormal as shown in Fig. 4 for the inserted unknown images. The performance of the proposed model is shown in Fig. 5 with training and validation accuracy respectively reached 97.82% and 98.41% after training with 120 epochs. The training and validation loss respectively reached 1.4026 and 1.5254. The ResNet-50 proposed model in this study is more suitable for sperm detection than other models with simpler preprocessing stages. The advantage of this proposed model is the possibility of getting accurate results quickly for making the ICSI process much easier for the andrologists.

VI. CONCLUSION AND FUTURE WORK
In this proposed model, the normal human sperms needed for the intracytoplasmic sperm injection process are recognized automatically. Andrologists are interested in easier tools for the optimum selection of healthy sperms for injection process into eggs. This study proved a technique using ResNet-50 and its outputs have been accepted and approved by the embryologists. In this paper, ResNet-50 proved the efficiency of the recognition of normal sperm and that is better than machine learning. The proposed deep learning model ResNet-50 resulting in high accuracy of 96.66% and this is one of the advantages of this study rather than the other model results in relative studies part. The ResNet-50 model has given its advantages for the detection of healthy sperms with automatic system in a quick and accurate way. The technology in methods of sperm selection is needed for the recognition of the best sperms that are required for raising success rate of ICSI operations. This system proved its distinction for obtaining good results with better accuracy for obtaining the normal sperms which referring to the normal sperm heads that can be utilized through ICSI process. One of the advantages of this study is its ability to deal with non-stained images that is similar to the captured images from microscope. Most of studies deal with stained images that are not applicable because the stain is harmful for the human sperms and make them not suitable for the injection process.
In the future, various models of deep learning can be utilized for achieving more accurate results. Developing algorithms for the detection of sperms using videos which can be built in microscope for real time detection of live sperms.