Bridge Pillar Defect Detection using Close Range Thermography Imagery

—Currently, radiometric thermography image has been explored adequately as alternative advance Non-Destructive Testing (NDT) especially for early detection analysis in various applications. Systematic image calibration, higher spatial resolution and high degree order image processing, thermography imagery potential to be used in concrete structure defect detection. Therefore, this study is carried out to examine the defect on bridge pillar surface concrete using drone-based thermography sensor (7–13 µm). Close range remote sensing NDT based on drone platform and imagery segmentation analysis have been applied to interpret the crack line on two pillars at North-South Expressway Central Link (ELITE) Highway. As a result, thermography imagery segmentation and support by multispectral radiometric imagery (RGB) successfully to delineate the micro crack line on the bridge pillar concrete using K-means clustering method. Overall, this study successfully shows the higher order optional platform using drone and thermography sensor that potentially to be applied in forensic concrete structure defect detection for tall structure building.


I. INTRODUCTION
Thermography that uses a mid-infrared band is the part of remote sensing that manages the acquisition, processing and interpretation of information procured essentially in the thermal infrared (TIR) area of the electromagnetic (EM) range. In thermography remote sensing, the radiations emitted from the target surface will be measured which differ from optical remote sensing where the reflected radiation from target surface is the one that will be measured and these measures will determine a body's radiant temperature that were determined by two variables, kinetic temperature and emissivity [1]. Current technology has allowed the concept of thermography remote sensing being adapted by the use of drone or Unmanned Aerial Vehicle (UAV) as many compact thermal sensors were available. Nowadays. the thermal sensors are extensively utilized in different surface temperature and thermal emission measurements. The oldstyle issue on kinetic temperature and emissivity assurance through the force and its circulation over the wavelength region of close-range remote sensing can be explained not similar concept as the airborne or spaceborne thermal sensors. Concerning low altitude or drone-based sensors, the atmospheric impacts are insignificant, lab-level adjustment are progressively available, and theoretically the measurement of temperature are more accurate [2]. Close-range remote sensing imagery technology detection are believed to be a much more practical and more relevance approach to the work of detecting a surface that are not accessible by the remote sensing sensor.
Concrete structure surface defect come in quite few types with cracking and scaling being one of the most common defects. In order to detect defect of a certain structure or infrastructure, the methods that are considered as the most suitable are usually a Non-Destructive Testing (NDT) method because this method allows to avoid interfering with the asset's current state. The majority of NDT for structural elements, such as ultrasonic, magnetic field, and eddy current methods, are best suited for detecting flaws at depths of 5 to 100 cm and have two major drawbacks: they require physical contact with the tested object and scan images slowly [3]. Other method that would be very useful to the concrete structure defect detection is a LiDAR measurement method. LiDAR is a method for determining ranges by targeting an object with a laser and measuring the time for the reflected light that return to the receiver. As a result, LiDAR provides point cloud data, which is basically 3D geometric information that may be used to measure surface anomalies, by populating a surface with hundreds of laser points [4]. As it was mention *Corresponding Author www.ijacsa.thesai.org earlier, this study will incorporate a thermography imaging method which is also consider as part of NDT method. Thermography, commonly known as thermal scanning or infrared imaging, is a rapid and remote technology that is increasingly being utilized in conjunction with NDT methods [5,3]. Usually, defects produce an excessive heat during operation due to the excess current flow at the fault point results in power dissipation [6].
Nowadays the concept of infrared thermography (IRT) imagery was being utilized in many ways. Firefighters use them to see through smoke, find people and localize hotspots of fires. Law enforcement uses the technology to manage surveillance activities, locate and apprehend suspects, investigate crime scenes and conduct search and rescue operations. Power line maintenance technicians locate overheating joints and parts eliminate potential failures. Where thermal insulation becomes faulty, building construction technicians can see heat leaks to improve the efficiencies of cooling or heating. Physiological activities, such as fever, in human beings and other warm-blooded animals can also be monitored with thermographic imaging. IRT imagery have great thermal resolution capabilities, but they have a basic spatial resolution constraint [6]. This indicating that thermography imaging concept has proven its usefulness and other ways that this concept can be used is for the detection of defect on a concrete surface such as bridge pillar.
Surface temperature data can be collected using IRT method on which its data can detect the internal faults in concrete [7]. The work defect detection on a concrete surface can be carried out done by differentiating the temperature on its surface as thermography create an illusion of differences temperature for human to detect the difference in thermography imagery or in this case a defect on a concrete surface. The concept of thermography imagery in detecting the defect structure is when the thermographic camera detects the radiation emitted by an object as it was heated by sun or any sourced of heat. The thermography NDT image is mainly judged by analyzing the abnormal conditions of the temperature change law at the crack manifested that when the ambient temperature is lower than the object temperature, the cracks are shown as hot spots or regions in the thermal camera. While cracks would manifest themselves as cold spots when the region of interest is colder than the surrounding environment [8].
This project utilized a drone-based platform in which the thermal image acquired vertically to reconstruct a complete pillar structure. A thermal sensor mounted to the drone to capture the thermography image of the pillar surface. The acquired imagery then underwent in a vertical flight plan as in order to completely cover all the area of the pillar. Therefore, when this concept was applied on the maintenance work of the bridge pillar, a thermal sensor will be attached on a drone and thermography image of the pillar surface will be collected. From those images, the temperature of the pillar surface will be identified and the differences in temperature will produced an assessment that indicate there is something thermal hot spot with the concrete surface of the pillar hence, a defect is detected. This defect might result as the presence of cracked on the surface of the concrete which needed to be inspected and immediate action can be performed to avoid any unwanted event.

II. STUDY AREA
The research areas that explored are located at part of North-South Expressway Central Link (ELITE) Highways' bridge pillar, which is also known as Asian Highway 2 (AH2), situated near to Management and Science University (MSU) Shah Alam, Selangor, Malaysia (03 o 04'41.3‖N,101 o 33'06‖E) as shown in Fig. 1. This study focused on the crack detection of two pillars or piers of the highway bridge. The data acquisition that has been collected then processed and analyzed using the specific software to detect crack on bridge surface. Concrete Pillar.

III. AIM AND OBJECTIVE
The aim of this research is to study the performance of thermal imaging technique for detecting the defect of vertical concrete surface. In pursuit of the aim of this research, this study specifically addresses two objectives as follows:  To demonstrate the vertical concrete surface defect detection technique using drone-based close-range thermography imagery.
 To detect the defect on bridge surface using segmentation method by utilizing thermal group of UAV remote sensing. www.ijacsa.thesai.org

A. Close Range Thermography Imagery
Close range photogrammetry also known as terrestrial photogrammetry is a procedure in which photos are collected with relatively high convergent camera orientations around an entity, often facing towards the entity's center [9]. The concept of it refers to the imagery collected in a range less than 300 meters away from a study object. In this project, the closerange drone approach was utilized where the thermography imagery was acquired using a sensor carried by DJI Inspire 1. The drone is a commercial off-the-shelf (COTS) photography platform that is ready to maneuver right out the box. The drone is featured with an onboard color camera equipped with a lens and a stabilized gimbal, also a retractable landing gear that can be pulled up out of the view, giving the camera 360 degrees of view towards the world below without obstruction.
The reason to acquire and process the thermography imagery photogrammetrically was to produce thermal orthomosaic with ground resolution distance (GSD) below 10mm. Meanwhile, thermography lacks the image's features. The recorded details are based on temperature, limiting the success of image matching among overlapped photographs for orientation in photogrammetric software. For that reason, the normal color image was also acquired to assist the photogrammetric image orientation process.
The DJI FC350 camera and the FLIR Zenmuse XT (Table I) were used to capture the color and thermographic images, respectively. The former is an onboard camera for the DJI Inspire 1 drone system. It features a 12.4-megapixel effective pixel and a 20-millimeter lens. Additionally, the camera is detachable from the drone, allowing for the attachment of the former camera for thermographic imaging. The FLIR Zenmuse XT is designed to be mounted on the DJI Inspire 1, capturing images between 7.5 and 13.5 meters and storing them in 640 x 512 pixels image.
In a typical photogrammetric mapping project, the flight mission configures autonomous data collection in the open space area prior to the flight mission. The benefits of an autonomous flight mission include the drone maneuvering autonomously as a result of the GPS satellite signal assisting with flight direction and hovering. Additionally, the image can be captured automatically based on the coordinates of the configured camera station. The coordinates are then geotagged to the corresponding captured photo station to facilitate the point cloud construction process in photogrammetric software, as the exterior orientation parameter is available.
However, in this project, the drone was piloted manually because the data acquisition took place the beneath of bridge, where a tick concrete structure interfered with the GPS signal. Manually controlling the drone without locking the GPS signal was difficult in areas where the drone was uncontrollable due to turbulence. Additionally, the image was captured manually, as it was not necessary to preconfigure the flight planning. The image captured from a photogrammetric image block is made up of individual strips. The overlap of images was set to 80%. The camera-to-object distance was set to 1.5m in order to obtain thermography images with a 10mm GSD.

B. Establishment of Control Point
Photogrammetric processing enables the estimation of an object's scale and orientation based on the image's exterior orientation parameter. However, because the work is being conducted in a tick structure environment, accurate geotagging of GPS signals on the image is compromised. As a result, the image's geotagged parameter did not accurately reflect the image's actual location on the ground.
Alternatively, the use of GCP can be used to resolve the determination of the constructed object's coordinates and scale. Additionally, GCPs are used to determine the exterior orientation parameter of an image by resecting the coordinate to the image point via a mathematical equation. With a precise exterior orientation for the image, light rays are projected to precisely and accurately model space from images. As a result, the 3D model of the object constructed has an accurate scale and orientation similar to that of the ground.
On the other hand, the thermal image's nature is to capture the object's heat/temperature. Thus, the thermal image does not contain a common GCP marker used in photogrammetry. The use of a thermal patch may be appropriate. However, the patch's coordinates cannot be precisely determined. For this project, the standard photogrammetric control point marker was designed based on the focal lens, sensor resolution, and camera-to-object distance of the respective camera. Ten markers were printed on A4-size paper and adhered to the surface of each pillar. In this study, the location of markers on two pillars was set to the upper area as the focus point for thermography analysis and shown in Table II. Each marker's position was determined using the tacheometry technique using an arbitrary coordinate system consisting of a single baseline. The geometric control point coordinates of each pillar are shown in Table III and  Table IV, respectively. The information from the GCPs was then used to create the thermography orthomosaic, as explained in the following section.

C. Orthophoto Construction
After constructing the tie point cloud, a dense point cloud was generated. The distinction between a tie and a dense point cloud is that the former is used to determine the exterior orientation parameter by resecting the tie point. Meanwhile, the latter is used to transform each pixel in the image into three-dimensional space. From the dense point cloud, a variety of photogrammetric products such as a 3D model and orthomosaic can be generated. The thermographic orthomosaic in this work was created using the Pix4D mapper (academic version).
The process began with aligning the color images in Pix4D in order to create a tie point cloud and estimate the exterior orientation parameter for each image. The GCPs are then used to reorient the image via bundle adjustment and to scale the pillar's 3D point cloud for metric measurement. Following that, a dense 3D point cloud was generated and the coordinates of pillar features such as edges and contrast spots were extracted in order to align the thermography imagery. Due to high resolution of color image, the common features on overlapped images were seamlessly matched for tie point construction.
The thermal dataset was then sequentially inserted onto color photographs that had previously been aligned. The edges and contrast spots were marked as control points for the thermal point cloud. The coordinates for the control point were extracted from the object coordinates previously determined using the color image. This procedure enables the transformation of the constructed pillar's coordinates to the features visible on the thermographic image. When the thermal imagery was successfully aligned, the thermal image's dense point cloud was also generated. Following that, the thermal orthomosaic was created using the orthoplane function included with Pix4D Mapper. Fig. 2 to Fig. 5 has shown the process using Pix4D Mapper. www.ijacsa.thesai.org   FLIR Tools+'s image stitching for thermal images is fairly simple, beginning with importing the images into the FLIR Tools+ workspace. FLIR Tool+ stitched together several smaller images into a larger one using panorama mode by analyzing each image to detect pixel patterns that match pixel patterns in other images. Fig. 6 shows the results of image stitching from a thermal image using the FLIR Tool+ software.

D. Image Enhancement and Segmentation
According to [10], there are many researchers has used the image processing technique to detect a crack or defect in a certain structure, for example [11] has proposed preprocessing, image segmentation and feature extraction, [12] have presented a detection method based on the use of acoustic emission and digital image correlation and [13] used Gabor filtering to presented a method for automatically detecting cracks in digital photographs. In this project the method used was the image enhancement and image segmentation. www.ijacsa.thesai.org For Image enhancement is the process of adjusting digital images so that the results are more suitable for display or further image analysis. Furthermore, Image enhancement is used to improve the perception of visual quality in an image as well as to provide better input for other computerised image processing processes [14]. This process allowed the user to remove noise, sharpen, or brighten an image thus, making it easier to identify key features on the image and for this case, is to detect the defect area. Using FLIR Tool+ software, the image enhancement process can simply be achieved by adjusting the image contrast between temperature. Fig. 7 below shows the result of image enhancement.
Image segmentation is the process of dividing an image into several segments in order to transform the image's interpretation into something more meaningful and easier to work with or being analyzed [15]. As for this project, the segmentation was used to identify a weak region on the pillar. For image segmentation, the k-means clustering method was implemented in both ENVI and Matlab software. When working with small datasets, K-means clustering performs well because it can separate items in photographs and produce superior results. The ENVI software image segmentation process was carried out using the software's existing k-means clustering function, whereas the Matlab software required the implementation of a simple code, as shown below. To begin with, the enhanced image in Fig. 8 below shows that the quality of the thermal image has been improved as it has become brighter. These findings will make the image segmentation process more visible. The K-means clustering method of image segmentation divides the image into several predefined clusters and groups the different clusters with the same color to create the segmentation region. The segmented image in Fig. 8 shows that the results are quite similar, but the ENVI result appears to produce a much more detailed result. The segmentation process produced three distinct clusters, with the hottest area assuming to be the most affected area of defect. Subsurface deficiencies in concrete heat up at faster rate than sound surface which means the hotter region on the concrete surface appear to be the weak region that could possibly be a crack. The algorithm cluster thermal image by extracting the hottest pixel is brightest blob for great contrast between defect and non-defect area. The output from the segmentation process on the enhance thermal image it can be seen that the hotter region falls in to the crack area but in the segmented image there are also other hotter region on the pillar that fall out of the crack area and this is indicating that area might also have defect or just another weaker spot.

VI. DISCUSSION AND CONCLUSION
This study was successful in identifying a new method for detecting high concrete structure defects that was integrated with advances in aerial data acquisition and imagery processing. As is well known, the drone platform is very useful for acquiring low altitude aerial information using a camera or sensor payload. While the image processing system can produce good segmentation for various types of surfaces. It is a difficult task to create the 3D thermal image enhancement orthoplane where there is difficulty in stitching the homogenous structure surface such as to produce 3D bridge pillar. Another critical step is to use image processing system to determine the micro level of structure cracks from thermography image. As a result of this breakthrough, another advanced NDT process method for structure inspection is now available.
Furthermore, the capability of thermography imagery in dealing with the problem of defect detection on a concrete surface can be seen. This new method is quite useful and effective in that thermal imagery can be interpreted into other forms of imagery through the process of segmentation, providing a better initial view to solve the problem. Thermography imagery will always be an interesting NDT method for monitoring the health or condition of structures, not just concrete structures, but also other types of structures in a cost-effective manner. Sirca Jr and Adeli (2018) [16] state that this is due to the unique environmental constraints and the massive amount of data that must be processed, applying the thermography method of defect detection to large scale threedimensional concrete structures such as power plant cooling towers and hydroelectric dams will remain be a challenge. This statement has surely been a reason to keep on studying and explore the potential of the thermography imagery in solving the structure defect problem.
In conclusion, the aim of this study in which to study the performance of thermal imaging technique for detecting the defect of bridge surface has been achieved as the final result of the study has shown a positive outcome. Furthermore. The objective which is to demonstrate concrete surface defect detection technique using UAV thermography imagery and to detect the defect on bridge pillar surface using segmentation method by utilizing thermal group of UAV remote sensing has also been achieved. This study has profoundly used the method of close-range UAV in collecting thermal which then the data were processed into a better form of imagery in order to conduct the segmentation process that produce a clear vision to the defect on the concrete surface.