Multi Focus Image Fusion using Image Enhancement Techniques with Wavelet Transformation

Multi-focus image fusion produces a unification of multiple images having different areas in focus, which contain necessary and detailed information in the individual image. The paper is proposing a novel idea in a pre-processing step in the image fusion environment in which sharpening techniques applied before fusion in the pre-processing step. This article is proposing multi-focus hybrid techniques for fusion, based on image enhancement, which helps to identify the key features and minor details and then fusion performed on the enhanced images. In image enhancement, we introduced a new hybrid sharpening method that combines Laplacian Filter (LF) with a Discrete Fourier Transform (DFT) and also performs sharpening using the Unsharp sharpen approach. Then fusion is performed using Stationary Wavelet Transformation (SWT) technique to fused the enhanced images and obtaining more detail of the resultant image. The proposed approach is applied to two image sets, i.e., the “planes” and “clocks” image sets. The quality of the output image evaluated using both qualitative and quantitative approaches. Four will know quantitative metrics used to assess the performance of the novel technique. The experimental results of the novel methods showed efficient, improved outcomes and better for multi-focused image fusion. The SWT (LF+DFT) and SWT (Unsharp Mask) are 2.6 %, 1.8%, and 0.62%, 0.61% better than the best baseline measure, i.e., SWT, considering RMSE (Root Mean Square Error) for both image sets. Keywords—Multi focus image fusion; image enhancement; unsharp masking; Laplacian Filter (LF); Stationary Wavelet Transforms (SWT); frequency domain technique


I. INTRODUCTION
The perfect image should contains the complete elements of the view that are totally transparent and required all the necessary information for the particular application. Due to the intrinsic limitations of the capturing system, an image may cannot comprise all the essential information and the objects description in the scene. For example, the restraint of the limited depth of the focus of optical lenses, that is Complementary Metal-Oxide Semiconductor (CMOS) / Charge-Coupled Device (CCD), digital cameras prompt to prepare the refined image from various focused (multi-focused) of the same scene using the image fusion method, which combines all the focus information from the source images to produced well-informative image [16].
In image fusion, produced the resultant image which captured the complete necessity information in the source images. The fused image is more accurate and informative than any of the individual input image in image fusion. The primary goal of the image fusion is to construct an image from various images that are more appropriate for the specific application or scenario and more understandable, which also reduce the size of image [1]. The image fusion approaches are involved in many important applications such as object detection, image analysis, monitoring, robotics, remote sensing, hyperspectral image fusion [8], [14], military and medical [16].
The multi focus image fusion is important research field from last couple decades, and the researchers are continuously developing methods that can generate improved results for combining images into the fused image. Basically the image fusion is depends upon two domains, frequency domain and spatial domain, and it's also known as spectral domain and time domain respectively. In spatial domain includes Minimum/Maximum Selection [22], Averaging [15], Principal Component Analysis (PCA) [20] and Intensity Hue Saturation (IHS) [11] methods. All these methods generate poor results because of spectral distortions in resultant image, and generate image with low contrast, which contain less information comparatively [23]. On the other side, the methods such as Discrete Cosine Transform (DCT) [6], Stationary Wavelet Transform (SWT), and Discrete Wavelet Transform (DWT) [7] and the most common frequency domain methods used in multi-focused image fusion. In image fusion, DWT is advantageous method in wavelet transformation [1] but with the following drawbacks: • It keeps the vertical and horizontal characteristics only • It suffers through ringing artefacts and reduces the quality of the fused image • Lack of shifting invariance • Lack of shifting dimensionality • Not good for edge places due to missing the edges during in fusion The method of discrete wavelet transform is not time-invariant transformation method, which means that "with periodic signal extension, the DWT of a translated version of a signal X is not, in general, the translated version of the DWT of X." The typical DWT method lost to restore the translation invariance, which is marginally covered-up with SWT method by averaging slightly different DWTs, also called ε-decimated DWT [24].
From last many year the scientists performed the fusion with simple multi-focused images such the objects that are only located in the special depth of focus are clear, and the others are blurred. In this Paper, we are introducing the new concept as a pre-processed step before fusion. The preprocessed step is based on image enhancement, which helps to identify the key features and minor details. The Laplacian filter (LF) and Discrete Fourier transform (DFT) is developed as the new hybrid method for sharpening the images (preprocessed) before fusion. In the novel hybrid method, initially, the images are enhanced with LF + DFT sharpen method and then combined the enhanced image with the SWT fusion method. Similarly, the Unsharp sharpening method is also introduced as a new approach for pre-processing. The Unsharp method enhanced the images and then fused by the SWT method. The pre-processing step is firstly introduced in the image fusion environment. The new approaches produced encouraging results using both qualitatively and quantitatively evaluation approaches and compared the results with traditional techniques using two datasets.
The paper is organized as follows: Section 2 describes the novel approach, LF+DFT and Unsharp sharpening methods, and SWT fusion method in detail. Section 3 describes the motivations of the porposed sharpening techinque, Section 4 describes the performance metrics, Section 5 providing the experimental results and its comparison with exiting techniques, The conclusion is drawn in Section 6.

II. PROPOSED APPROACH
In this work, we are introducing a new multi-focus hybrid approach for fusion based on image enhancement, which helps to identify the key features, minor details and then fusion is performed on the enhanced images. The novel framework is presented in Fig. 1, and both the sharpening method i.e., LF + DFT and Unsharp masking with SWT method, are described as follows:

A. Laplacian Filter (LF)
LF is a spatial filtering method often applied to the images and used to identify the meaningful discontinuities in image, i.e., grey level or colour images, by detecting edges. The edges are formed among two different parts, i.e., having different intensities by calculating the Laplacian using second derivatives and convoluting it with the image [13], [3]. The calculation of the Laplacian equation as follows; The zero-crossings of the second derivative in Fig. 2, corresponding to the edges of the objects [18].

B. Discrete Fourier Transform (DFT)
The DFT [20] is the equivalent of the continuous Fourier Transform for signals known only at N instants classified by sample times T (i.e., a finite sequence of data). The Fourier Transform of the original signal,f (t) , as follows; The inverse discrete fourier transform is used as;

C. Unsharp Mask
An "unsharp mask" is a simple image operator, contrary to what its name might lead you to believe. The name is derived from the fact that it sharpens edges through a process that subtracts an unsharp mask of an image from the reference image, and then detects the presence of edges [4]. Sharpening can demonstrate the texture and details of the images. This is probably the common type of sharpening and can be executed with nearly any image. In a sharpened image, the resolution of the image doesn't change. In the unsharp mask method, the sharpen image a(x, y) will be produced from the input image b(x, y) as Whereas is the correction signals are calculated as the output of a high pass filter and is a positive scaling element that control the level of contrast and an enhanced image achieved at the output [19].

D. Stationary Wavelet Transform (SWT)
The SWT is a wavelet transform developed to get the better of the lack of translation invariance of the DWT method. The stationary wavelet transform is the whole shift-invariant transformation and overcome the down sampling step of the decimated tecchnique and alternative of up sampling the filter by putting zeros among the filters coefficients [17]. The design is simple and provides better time-frequency localization. Appropriate high pass and low pass filters are applied to the data at each level, and it generate two sequences at the next level. In the decimated algorithm, the filters are used for the rows at the first and second for the columns [5], [12]. The benefit of SWT are: No sub-sampling of input, Translation invariant, providing better time-frequency localization, providing the freedom to carry out a design [10]. The detail of stationary wavelet transform is in Reference [17]. The SWT filter bank structure is shown in Fig. 3.

III. MOTIVATION OF USING SHARPENING TECHNIQUE
In sharpening technique, the apparent sharpness of an image is increased, which is the merger pair of factors, that is, resolution and acutance. Resolution is straightforward and not subjective which means the size of the images in terms of the number of pixels. With all other factors remain equal, the higher the resolution of the image -the more pixels it has -the sharper it can be. Acutance, which is a measure of the contrast at an edge, is subjective and a little complicated comparatively. There's no unit for acutance -you either think an edge has contrast or think it doesn't. Edges that have more contrast appear to have more defined edge to the human visual system. Sharpness comes down to how defined the details in an image are especially the small details.

IV. PERFORMANCE MEASURES
To properly evaluate the performance of the novel hybrid approaches, To considered four known and common performance measures, i.e., Mean Absolute Error (MAE), Percentage Fit Error (PFE), Root Mean Square Error (RMSE) and Entropy (E) as briefly discussed below; Percentage Fit Error (PFE) It is computed as the norm of the difference between the corresponding pixels of the true image and resultant image to the norm of the true image [9]. The smallest values are showing good results. PFE as defined in eq. (6) where the norm operator is calculate the highest singular value.
Root mean square error (RMSE) is generally applied to compare the difference among the true image and resultant image by instantly calculating the variations in pixel values [21]. The resultant image is close to the true image when the RMSE value is near zero or zero. RMSE is indicating the spectral quality of the resultant image.
Entropy (E) is an significant metric applied to measure the information content of the resultant image [21]. Entorpy as define in eq.(8) Where 'L' is the number of grey levels of the fused image. "P i " is given by the ratio of the number of pixels.

V. EXPERIMENTS
In this section, we are discussing the experiments which are performed by the proposed hybrid approach on two image sets, such as "Clocks" and "Planes". These image sets are used as testing multi-focus images for the experimental evaluation of the proposed techniques. The size of the image set (test images) is 512 × 512. The performance of both the proposed approaches such as SWT + Unsharp and SWT + (DFT + LF) methods are compared with the will performed traditional and advanced techniques, which include the average method (AM), minimum method (MM), DWT and SWT methods. The algorithms are implemented using MATLAB 2016b application software tool, and the simulations are performed using a computer of Intel (R) Core(TM) i7-6700K CPU at 4.00 GHz machine with 8GB of RAM to carry out the experiments. The resultant images are evaluated in two ways, i.e., quantitatively and qualitatively. The qualitative analysis is a significant evaluation metric in multi-focus image fusion, which is used to visually observed the changes or improvement in the fused images after applying a techniques. Similarly, quantitative analysis techniques are used to evaluate the effectiveness of a technique statistically. Here, we are using four well know performance matrices for evaluation, such as  Table I. To easily observed, the best results of the proposed technique against the known techniques are bold. The smallest values indicate good performance for three performance metrics, i.e., RMSE, PFE, and MAE, which can be observed for both the proposed techniques. While the largest value for entropy performance metric and demonstrated impressive results by the SWT with LF+DFT technique.
In Fig. 6, image (a)    The four known performance matrices are used, and the results are shown in Table II

VI. CONCLUSION AND FUTURE WORK
Image fusion techniques are essential to get a more informative image from multi-focused images. To fused to fuse multi-focused images and get a more informative resultant image, we proposed hybrid approaches. In which the source images are sharpened in the pre-processing step and then applied two new techniques, i.e., SWT (Unsharp Mask) or SWT (LF+DFT). The results of the novel techniques are compared against four know baseline techniques, i.e., RMSE, PFE, MAF, and Entropy, to assess the proposed techniques. The proposed techniques show good results comparatively by applying both qualitative matric and quantitative matrics to two image sets. The accuracy is keenly analyzed using the RMSE performance matric from Table III. The SWT(LF+DFT), and SWT (Unsharp Mask) shows improved results and outperformed all the comparative techniques, i.e., SWT (Unsharp Mask), SWT, DFT, MM, and Average) by 2.18%, 2.6%, 2.84%, 7.21%, and 37.56% for Plane image set, and 1.23%, 1.84%, 2.04%, 5.85%, and 22.74% for Clock image set.
Currently, we are working to assess the effectiveness of the proposed techniques for other greyscale image sets, and color image sets. In the future, the proposed methods will be extended and improved by other advanced fusion methods such as DWT or DCT. The different performance metrics will validate the new approaches because each metric has its own situational properties. A third evaluation technique will be introduced beside both qualitative and quantitative measures in the future.