Cylinder Liner Defect Detection and Classification based on Deep Learning

—The machine vision-based defect detection for cylinder liner is a challenging task due to irregular shape, various and small defects on the cylinder liner surface. To improve the accuracy of defect detection by machine vision a deep learning-based defect detection method for cylinder liner was explored in this paper. First, a machine vision system was designed based on the analysis of the causes and types of defects to obtain the field images for establishing an original dataset. Then the dataset was augmented by a modified augmentation method which combines the region of interest automatic extraction method with the traditional augmentation methods. Except for introduction of the anchor configuration optimization method, an XML file-based method of highlighting defect area was proposed to address the problem of tiny defect detection. The optimal model was experimentally determined by considering the network model, the training strategy and the sample size. Finally, the detection system was developed and the network model was deployed. Experiments are carried out and the results of the proposed method compared with those of the traditional methods. The results show that the detection accuracies of sand, scratch and wear defects are 77.5%, 70% and 66.3% which are improved by at least 26.3% compared with the traditional methods. The proposal can be used for field defect detection of cylinder liner.


INTRODUCTION
Cylinder liner is one of the most important parts of engine. Its surface quality will directly affect the working performance and service life of an engine. The surface quality will inevitably deteriorate due to the comprehensive effects of friction, high temperature and corrosion. If there are cracks, sand holes, air holes and other manufacturing defects in the cylinder liner itself, the degradation process will be greatly accelerated. Therefore, defect detection is of great significance in the production process of cylinder liner.
Machine vision inspection technology has the advantages of non-contact, easy to realize automation, and easy to analyze and process the detection results by computer which has been widely used in metal surface defect detection. At present, machine vision based defect detection methods mainly include traditional methods and deep learning methods [1][2][3][4][5][6]. The traditional machine vision defect detection method constructs the feature descriptors for different defects through image segmentation, feature extraction and other image processing algorithms. Through the descriptors, the surface defects are located, identified, graded, counted, stored and inquired. However, due to the influence of image quality, the complexity of industrial scene, the difference of defect shape and size, the traditional methods still often fails.
In recent years, deep learning based methods have been applied to defect detection of metal surfaces. However, there is no relevant research on how to apply deep learning to detect the surface defects of cylinder liners, and systematically describe the design and implementation of the detection system so far. Therefore, this paper will explore the deep learning based method for the cylinder liner surface defect detection and give the design process and its implementation of the defect detection system.
The main structure of this paper is as follows: the related works in this field are introduced in the second section; the defects types are analyzed and the design method of machine vision system is discussed in the third section; the optimal deep learning model are experimentally determined for cylinder liner defect detection in the fourth section; the design of detection system and the field experiments are given in the fifth section; the last section will summarize this paper.

II. RELATED WORK
A cylinder liner defect detection system based on X-ray and linear array camera was built by Han Yueping of North China University and its key technologies were studied, such as sequence image filtering, threshold segmentation and morphological processing and calculation of defect parameter chain code tracking method [7,8]. Considering that the probability of defects is low and the defect area is relatively small compared with the whole image, so the cylinder liner Xray image is highly sparse. The group also proposed the use of compressed sensing algorithm for defect detection [9].
In other metal surface defect detection, the traditional machine vision method is more widely used. A multi-scale defect recognition method was proposed by Yu Jiahui et al. which has good accuracy and detection speed [10]. Tian Hongzhi et al. designed a micro defect detection system for grinding surface by combining plane illumination mode with multi angle illumination mode [11]. Mentouri zoheir et al. employed an improved dual cross algorithm to online monitoring of steel surface quality [12]. Aiming at the problems of complex defect pattern and low contrast between defect and background in steel strip surface defect detection, Liu Kun proposed a total variation image decomposition algorithm based on self-reference template and improved index gradient similarity [13]. Cao Binfang et al. proposed a defect detection method based on spatial-frequency multi-scale block local binary pattern to solve the problem of complex geometry and texture distribution of the nickel foam surface defect images [14]. Sun qianlai uses singular value decomposition to identify and locate surface defects of strip steel without image segmentation [15]. Based on the research on pseudo defect elimination, patch texture description and adaptive threshold segmentation, Liu Kun et al. proposed a new unsupervised steel surface defect detection model based on Haar-Weibullvariance [16]. Jeon Yong Ju et al. proposed the dual-light switching lighting technology to solve the problems of uneven brightness and various defects on the steel surface [17].
The core of traditional methods is to design and use feature descriptors, which include local binary pattern (LBP), histogram of oriented gradient (HOG), gray-level cooccurrence matrix (GLCM) and other statistical features. Feature descriptors are sensitive to lighting, background and other environmental factors. So it is very important to collect high-quality images. Through the optimal design of the imaging system, the difficulty of algorithm development can be reduced and the robustness of algorithm can be improved, but the cost of detection system must increase. Moreover, due to the complexity of industrial sites and the difference of defect shape and size, the failure of traditional algorithms still often occurs.
In recent years, with the successful application of deep learning in many fields of machine vision, more and more researchers are committed to using deep learning for defect detection in industrial field, aiming at improving the accuracy, efficiency, stability and reliability of the detection system. RetinaNet with difference channel attention and adaptively spatial feature fusion was proposed for steel surface defect detection by Cheng and Yu [18]. Zhang Jiaqiao et al. employed a CP-YOLOv3-dense neural network in the steel strip surface defect detection [19]. Xiao Ling et al. [20] proposed a surface defect detection method based on image pyramid convolution neural network model. Wei Rubo et al. [21] proposed a method for steel defect detection based on the fast regional convolution neural network. A steel surface defect detection model based on deformable convolution enhanced backbone network and pyramid feature fusion was proposed by Hao Ruiyang et al. [22].
These above deep learning based methods mainly focus on deep neural network (DNN) model. However, the dataset with support samples is more important than the DNN model. It is often difficult to obtain enough support samples for surface defect detection in industrial field. Therefore, how to improve the accuracy of deep learning-based defect detection method has become a research hotspot under the condition of a small number of samples.  [23]. To address the problem that the existing defect datasets are generally unavailable for on-site deployment due to the limitation of data scale and defect types, Lv Xiaoming et al established a dataset named GC10-DET using a linear array image acquisition system to collect images [24]. To meet the challenge of detection the micro defect from high resolution images, a novel machine vision method was proposed for automatically identifying micro defects by Lian Jian et al. [25] and the main contributions of the proposal can be summarized as follows: 1) a defect exaggeration approach based on regularization, 2) a defect sample production method based on a generative adversarial network (GAN) and a convolutional neural network (CNN), and 3) a data augmentation method based on GAN. A novel approach for data augmentation was proposed by Jain Saksham et al. using GANs to create synthetic images to address the problem of time-consuming and high cost of onsite image acquisition. According to the comparative experiment, the performance of CNN architecture is significantly improved with GANs-based augmentation data and the sensitivity and specificity of the synthetically augmented CNN are 5.59% and 1.12% higher than those of the classical enhanced CNN, respectively [26].
The deep learning method has been employed in metal surface defect detection, but it has not been applied in cylinder liner surface defect detection. Therefore, this paper will take the lead in exploring the deep learning-based defect detection method for cylinder liner, designing a cylinder liner surface defect detection system, proposing a defect detection process, and giving a design case of cylinder liner surface defect detection system. 151 | P a g e www.iijacsa.thesai.org

A. Defects Types of Cylinder Liner Surface
As shown in Fig. 1, the common surface defects of the cylinder liners are the sand defect, the crack defect, the wear defect, the oil defect, the scratch defect and the collision defect.
When casting cylinder liner, gas and non-metallic impurities can not be discharged before solidification of liquid metal, resulting in the formation of sand defects on the surface of cylinder liner after machining. The size of the sand defect is small, its contour is usually elliptical and its edge is smooth. Sand defect is one of the main defects of cylinder liner which may appear in any part of the cylinder liner. The existence of sand defects may greatly reduce the impact and fatigue resistances of cylinder liner which is easy to cause cylinder collapse, water leakage and other faults. In the process of machining, the cylinder liners were deformed under the combined action of various stresses. When the deformation exceeds the plastic limit, the slender flocculent or snowflake like cracks may appear on the surface of the cylinder liner. Most of the cracks occur on the inner and outer surface of the cylinder liner which may affect the reliability and replacement cycle of the cylinder liner.
Wear defect usually refers to drag, block or furrow deformation on the surface of cylinder liner during production and transportation. Compared with the normal area, the wear area is generally silvery white. Wear defects usually occur on the end face or outer surface of the cylinder liner which may reduce the sealing, corrosion resistance or wear resistance of the cylinder liner, resulting in the decrease of engine power.
Oil defect is a kind of pseudo defect which is formed by the air evaporation of the cleaning oil or antirust oil left on the outer or inner wall of the cylinder liner. Oil defects are charred black and long drop-shaped in appearance which are prone to false inspection.
Scratch defect is a kind of non-uniform and strip-shaped ravine defect which is caused by the friction between impurities and cylinder liner in the process of processing or transmission. Scratch defects often appear on the inner or outer wall of the cylinder liner which may lead to unreasonable fit clearance improper assembly and other problems, thus reducing the wear resistance and mechanical properties of the cylinder liner.
Collision defect are the falling off or blocky defects caused by the collision between the cylinder liner and the cylinder liner or the fence in the process of transportation. Most of them appear on the upper end face and skirt. Collision defects may reduce the cylinder liner sealing and engine efficiency, shorten the replacement cycle of cylinder liner, and lead to engine damage accidents.
In the field of cylinder liner surface quality inspection, there is no clear standard to identify the types and severity of the above six kinds of defects. However, the sand defect, scratch defect and wear defect are common in most enterprises. Therefore, the deep learning defect detection method with a small number of samples was studied to detect the above three types of defects.

B. Design of Machine Vision System
As shown in Fig. 2, the machine vision system for the cylinder liner defect detection consists of three area array cameras and a linear array camera. Camera 1, camera 2 and camera 3 are area array cameras. Camera 1 was employed to capture the image of top face and its object distance is about 270 mm. Camera 2 was employed to capture the image of the inner wall and its object distance is about 255 mm. The angle between its optical axis and the axis of camera 1 is about 62.5° ± 5°. Camera 3 was employed to capture the image of the skirt and its object distance is about 247 mm. The angle between its optical axis and the cylinder liner axis ranges from 26.5° to 36.5°. Camera 4 which is a linear array camera was used to capture the image of the outer wall and it object distance is between 338 mm to 358 mm. The angle between the light 1 and the optical axis of the camera 4 is about 45°. The light 2 is a ring light source, the light 3 is a circular backlight and the light 4 is a ring light source of which the inner diameter is slightly larger than the outer diameter of the cylinder liner. Furthermore, the four light sources were installed at the determined positions.
With the above machine vision system, the images of the top face, skirt, inner wall and outer wall of the cylinder liner were collected, as shown in Fig. 3. There is no image set for cylinder liner surface defect detection. Therefore, 7500 images of cylinder liner were collected by using the above machine vision system with the guidance of field engineers and the image sizes are 2048 pixels × 2448 pixels. Among these images, 586 were defective. The sand defect images of the top face, the skirt, the inner wall and the outer wall of the cylinder liner are shown in Fig. 4, respectively. • Imageset Augmentation The imbalanced datasets will make the results of convolutional neural network over biased to the classification of abnormal targets. In order to alleviate the over fitting problem and enhance the robustness of the network, considering that the cylinder liner image is gray image and the surface defects are small, we use upsampling method to expand the defect sample data to meet the requirements of neural network training data.
The common data augmentation methods mainly include rotation, offset, clipping and scaling. The rotation enhancements are accomplished by rotating the image to the right or left by 30°, 45°, 60° and 90°. The offset enhancements are to move the image around to change the original defect position. When the original image is converted in one direction, the remaining space can be filled with 0. The clipping enhancements are to cut the original image at 30°, 45°, 60° and 90° and fill the remaining space of the image with 0. The scaling enhancement makes the whole image scale in different ratios. In the convolution neural network training, more image invariant features can be learned to improve the detection accuracy.
The augmented dataset contains 5000 images which is basically balanced with the normal sample. The set was divided into the training set, the verification set and the test set by 8:1:1.
• Labelling Defects Some regions of interest were extracted to reduce the search time of target region and the training time of neural network by the bi-dimensional maximum conditional entropy based threshold segmentation method of which a detailed derivation was carried out in our previous work. The mathematical model of grey entropy is as follows.
where m represents gradient levels of gradient image of gray image, n represents gray levels of gray image, ij p represents the probability that a pixel with higher gradient but lower gray belongs to an edge and ij p  represents the probability that a pixel with higher gradient and gray belongs to an edge.
(s*, t*) which makes the objective function H (s, t) take the maximum value is the optimal threshold for segmentation of a grayscale image and its gradient image. t* was used to segment the cylinder liner image and extract the regions of interest such as the skirt and the top face. Labelimg which is a digital image labeling tool was employed to label the cylinder liner image. The label information is stored in an XML file which contains the image name, the image resolution, the defect size, the defect location and the defect names including the wear defect, the scratch defect and the sand defect and which is shown in Fig. 5.

B. Improvement of Anchor
A differential evolution search algorithm was employed to optimize the aspect ratios and scales of anchors to address the problem that the default anchor configuration turns out to be ineffective for detecting lesions of small size and large ratios [27].

C. Determination of Optimal Deep Learning Model
Usually, the deep learning models were evaluated by the indicators of recall, precision and accuracy. AP value was employed to represent the recognition rate of the deep learning model on a certain type of defect, which is equal to the area of the trapezoid enclosed by the P-R curve formed by recall and precision and the coordinate axes. mAP represents the average recognition rate of the model on all types of defects and its value ranges from 0 to 1. The larger the value of AP or mAP, the higher the defect recognition rate. The mAP will be adopted to evaluate and determine the optimal target detection model. Three groups of experiments were carried out to analyse the influence of factors of the detection models, the number of datasets and the training strategy on convolutional neural network. The random gradient descent method was used to optimize the parameters, the loss function was calculated by the softmax layer, the initial learning rate is 0.00001 and the decay rate of the learning rate is 0.1. These configurations remained unchanged in the experiments to ensure comparability.

TP
As shown in Fig. 6, the mAPs of SSD, Faster-RCNN and Retinanet with transfer learning strategy are higher than those of detection networks without transfer learning strategy. Therefore, the transfer learning strategy can shorten the training time and contribute to achieving higher detection accuracy which is suitable for a small number of samples. The performance of the three networks with transfer learning strategy were compared and the comparison results are tabulated in Table I. According to the table, the mAPs of SSD, Faster-RCNN and RetinaNet are 0.597, 0.604 and 0.620, respectively. RetinaNet has the highest detection accuracy. In terms of time, the SSD is the fastest which takes 0.976 s and the RetinaNet is 0.212 s slower than the SSD. The RetinaNet will be employed as the main network for cylinder liner surface defect detection to achieve a good compromise in accuracy and speed. Furthermore, the effect of sample size on the performance of Retinanet was compared experimentally. Using 25%, 50%, 75% and 100% samples to train Retinanet, the mAP is 0.11, 0.40, 0.53 and 0.62, respectively. It can be seen that the more samples, the higher the accuracy of the network. Therefore, the data-driven deep learning model needs a lot of data to train its deep network, so as to obtain accurate feature extraction.
According to the above experiments, the Retinanet with the transfer learning strategy will be used for the cylinder liner defect detection.

D. Highlight Defect Areas of Support Samples
RetinaNet with transfer learning strategy achieves the highest mAP, but only 62%. By analyzing the dataset, it was found that the defect area only accounts for 0.25% of an image. However, the Retinanet uses the feature pyramid network (FPN) to produce feature maps with rich semantic. Most of the candidate image windows are background (negative classes) and only a few areas contain defects (positive classes). A large number of background cover up the defects which makes it impossible to fully extract the feature information of small defects in the process of the deep neural network training and finally leads to the low accuracy.
To highlight the defect to improve the detection accuracy, according to the location and area of the defect from the XML file, 100 pixels were extended to the top, bottom, left and right of the defect to form an image window to surround the defect. Then the defect area was updated by the formed window. Compared with the original image, the proportion of defect area was increased significantly and the detection accuracy is expected to be improved, as shown in Fig. 7.
The proportion of the redefined defect area in an image is significantly increased. Therefore, the accuracy of defect detection is effectively improved. Through the above method, the accuracy of Retinanet was improved from 0.62 to 0.71, which was increased by 14.52%. For the sand defect, scratch defect and wear defect, the AP value is 0.78, 0.69 and 0.66 respectively.

A. Configuration of Detection System
The general layout of the cylinder liner surface defect detection system is shown in Fig. 8, the field oriented cylinder liner defect detection system is shown in Fig. 9, and the feeding system and image acquisition system are shown in Fig.  10.
According to Fig. 8 and Fig. 9, the workflow of the field oriented cylinder liner defect detection system is as follows: the cleaning subsystem cleans the cylinder liner; as shown in Fig. 10 (a), the feeding subsystem transports the cylinder liner to the end of the belt to trigger the position sensor; the lifting subsystem grabs and lifts the cylinder liner to the rotary platform accordingly; as shown in Fig. 10 (b), the rotary platform and the cylinder liner are sent to the detection room to complete the whole process of surface defect detection; they are sent out of the detection room and the sorting subsystem pushes the genuine and defective products to the corresponding conveyor belt according to the detection results.   A workstation was used for network model training of which the memory is 256 GB, the model of the graphics card is NVIDIA Tesla P100, the video memory is 16 GB, and the operating system is Ubuntu 16.04. The framework of deep learning is tensorflow. The network model was offline trained by using the above dataset and it was deployed into an industrial-grade server successively.

B. Modular Design of Inspection System
After oil washed, the cylinder liner is easy to adhere to impurities such as lint and dust on the inner and outer walls which may easily lead to misjudgment during automatic optical inspection (AOI). Therefore, a cleaning subsystem was designed which mainly includes the air cylinder, air knife, oil buffer and other parts, as shown in Fig. 11 (a). The air knife forms an angle of 30 degrees with the axis of the cylinder liner to spray high-pressure gas to the surface of the cylinder liner to clean the surface of the cylinder liner and the air knife is pushed by the air cylinder to move up and down with the guidance of the guide rod to achieve the entire surface. The above cleaning process usually needs to be repeated twice. In addition, a shock absorber was employed at the joint between the cleaning device and the cylinder liner to reduce the impact of the start and stop impact of the air pump on the system. The grasping device is the end executive device of the screw module of which the main design requirements include: (1) the clamping force should be large enough to ensure reliable clamping and avoid displacement or vibration during handling. However, it should not be too large to prevent the cylinder liner surface from being damaged. (2) The central line of the gripper coincides with the central line of the cylinder liner to ensure that the cylinder liner will not collapse during clamping to avoid secondary damage to the cylinder liner. (3) The gripper should be suitable for both D123 and D130 cylinder liners. Therefore, a gripper driven by air cylinder was designed. Oil resistant rubber was pasted on the inside of the claw to increase friction to prevent the cylinder from sliding, as shown in Fig. 11 (b). The design of detection room includes the design of rotating platform, as shown in Fig. 12 (a), and internal structure of the detection room. The cylinder liner is clamped on the rotating platform through the central positioning mode and the platform and the cylinder liner is driven to rotate to collect images of the inner and outer cylindrical surface to achieve the defect detection of the inner and outer walls. The higher the center positioning accuracy is, the greater the clamping force will be. If the clamping force is too large, it is easy to scratch texture on the inner surface, causing damage to the inner wall. If the positioning accuracy is too low, the cylinder liner cannot rotate reliably with the platform which affects the image acquisition. Therefore, the diameter of the rotating platform is 4mm smaller than the inner diameter of the cylinder liner and the motor drives the platform to rotate by the PLC. The internal structure design mainly includes the support and mechanical interface design for cameras, lights and rotating platform, as shown in Fig. 12 (b). 156 | P a g e www.iijacsa.thesai.org

VI. EXPERIMENTS AND RESULTS
The field experiments were carried out using the network model trained in the third section and the dataset containing three types of defects: the sand, the scratch and the wear defect with 80 images for each type of defect. The experimental results were tabulated in Table II. The detection accuracy of sand defects is 77.5%, that of scratch defects is 70%, that of wear defects is 66.3% and the average accuracy is 71.26%. The comparison results of some traditional non-dedicated defect detection methods and the proposal were tabulated in Table III. The average accuracy of feature point registrationbased method is 36.0% and that of morphology-based method is only 27.3%, but that of the method based on deep learning proposed in this paper is 71.3%. Compared with the feature point registration-based method, the proposed method improves the detection accuracies of sand, scratch and wear defect by 51.5%, 28% and 26.3%, respectively. Compared with the morphology-based method, the proposed method improves the detection accuracies of sand, scratch and wear defect by 51.5%, 44% and 36.3%, respectively. The deep learning-based method is more effective for cylinder liner defect detection compared with some traditional non-dedicated methods. The effect of proposed deep learning-based method was shown in Fig. 13. Our proposal can detect tiny defects such as the sand, the scratch and the wear defects, identify the types of defects and locate the defects in the very large cylinder liner images. The detection method can basically meet the actual cylinder liner surface detection requirements of the enterprise to continuously improve the product quality. 157 | P a g e www.iijacsa.thesai.org However, the detection accuracy still needs to be improved. We can continue our research work from several aspects, such as the updated data sets [28], the motion platform control algorithms [29], the automatic labeling method [30], and the deep network model [31], which is expected to reduce the false detection rate and missing detection rate and further improve the accuracy of our method.
1) The network model can be further trained to improve its fitting accuracy by continuously adding the field data to the dataset to increase the amount of sample data.
2) The accuracy and reliability of the motion platform and its controller need to be further improved to ensure the acquisition of highly reliable images.
3) For the online defect detection, an automatic labeling method needs to be developed to avoid the missing labeling problem of manual labeling and improve the efficiency of defect labeling.
4) The network model can be improved and optimized to further improve its learning ability with a few samples and its detection ability for small defects.

VII. CONCLUSION
To address the actual needs of the cooperative enterprise, this paper developed a method and its system for cylinder liner surface defect detection based on deep learning. First, a machine vision defect detection system based on the causes and types of cylinder liner defects was built. Then, a dataset augmentation method based on the automatic extraction of region of interest was proposed which effectively increases the number of samples. Next, an automatic extension method of defect region was developed with the XML file which improves the detection ability of our proposal for small defects. After this, the network model and training strategy were experimentally determined and the influence of sample size on detection accuracy was discussed. Lastly, the scheme of implementing cylinder liner defect detection system in industrial field was given and the experiments were carried out. The results show that the detection accuracies of sand, scratch and wear defects are 77.5%, 70% and 66.3% which are improved by at least 26.3% compared with the traditional methods and that our method has achieved preliminary results and effects.