Efficient DWT based Fusion Algorithm for Improving Contrast and Edge Preservation

The main principle of infrared (IR) image is that it captures thermal radiation of light. The objects that are captured in low light, fog, and snow conditions can be detected clearly in IR image. But the major drawback of IR image is that it provides poor resolution and low texture information. Due to that humans are unable to understand overall scene information present in IR image. Nowadays for the detection of objects in poor weather conditions with improved texture information, the result of visible (VI) and IR image fusion is used. It is mostly used in military, surveillance, and remote sensing applications. The efficient DWT based fusion algorithm for improving contrast and edge preservation is presented in this paper. First morphology hat transform is applied on source images for improving contrast. DWT on decomposition produces low frequency and high frequency sub-bands. A novel mean weighted fusion rule is introduced in this paper for fusing low frequency sub-bands. Its aim is to improve the visual quality of final fused image. The max fusion rule has used for fusing high frequency sub-bands to improve edge information. The final fused image is reconstructed by using IDWT. In this paper, the proposed fusion algorithm has produced improved results both subjectively and as well as objectively when compared to existing fusion methods. Keywords—Visible image; infrared image; Discrete Wavelet Transform (DWT); Inverse Discrete Wavelet Transform (IDWT); novel mean-weighted fusion rule; max fusion rule


I. INTRODUCTION
Digital image processing is a significant and very useful research area, which plays a vital role in day today life of humans. The usage of it has been increased exponentially in the last decades. It is applied in wide range of applications such as medical [1], [2], [3], security [4], etc. This paper presents latest research area work of digital image processing.
Nowadays improving scene information for better human observation that is captured during bad weather conditions is playing a significant role in many vital applications like remote sensing [5], military [6], and surveillance [7]. This can be done by using VI and IR image fusion. Presently a lot of research is going on VI and IR image fusion [8]. The main motive of researchers is to improve the result of VI and IR image fusion. The IR camera sensor is used for capturing IR images. The basic principle of the IR camera sensor is that it captures thermal radiation of light. The VI camera sensor is used for capturing VI images. The basic principle of VI camera sensor is that it captures reflection of light. The object information details are better provided in the IR image irrespective of weather conditions. But the drawback of IR image is that it provides poor spatial resolution. The background details such as textures are provided well in VI image due to high spatial resolution. The disadvantage of VI image is that object details are not seen properly in bad weather conditions such as low light, fog, and snow. The fusion of these both images will produce a single image. It provides better scene information such as good object details and as well as better background information in single image.
Traditionally fusion process can be implemented in two ways:  Fusion in spatial domain.
 Fusion in transform domain.
The fusion methods in spatial domain are weighted average [9] and PCA [10] fusion. These methods don't require any transforms. They are applied directly to pixels. Advantages of these methods are, fusion can be done in a fast manner and computational complexity of fusion algorithm is less. The main drawbacks of these methods are:  They produce blurred results after fusion.
 Spatial distortions and spectral degradations are produced in the final fusion result.
The fusion in transform domain mainly comprises of three steps:  First transform is applied on source images. On decomposition it produces low frequency and high frequency sub-images.
 Fusion rules are applied.
 Inverse transform is applied for reconstructing fused image.
The main advantage of fusion methods in transform domain is that high quality fused images are produced compared to fusion methods in spatial domain. Nowadays so, many transform domain fusion methods have been developed. The transforms that are used in these methods are Discrete Wavelet Transform [11], Laplacian pyramid transform [12], Contour-let transform [13], and Dual-tree complex wavelet transform [14].
An efficient DWT based fusion algorithm for improving contrast and edge information is presented in this paper. Its motive is to achieve both better contrast and rich texture information resultant fused image. *Corresponding Author www.ijacsa.thesai.org II. RELATED WORKS Habeeb et al. [15] have developed a DWT based fusion algorithm. Here source images used are VI and IR images. In this algorithm sharpen filter has been proposed. Here sharpen filter has been used for improving the contrast and edge enhancement of the IR image. DWT on decomposition produces both detailed and approximation coefficients. For merging both detailed and approximation coefficients weighted averaging rule has been used. The fused image is reconstructed using IDWT. Mainly this algorithm has been developed for highlighting edges and fine details of output resultant fused image. The limitation of this method is that the resultant fused image shows less visible feature information.
Habeeb et al. [16] have introduced the fusion algorithm for combining multi-modal images in the spatial domain. The modalities chosen are VI and IR images. First PCA is applied to both source images for reducing dimensionality. Enhancement of infrared image is done by using a 3 × 3 sharpen filter. Similarly, for enhancing the visible image histogram equalization method is used. Finally, weighted average rule is used for fusing the output of sharpen filter and histogram equalization. This algorithm achieved better results than traditional methods results. But the main drawback of this fusion algorithm is that it produced a blurred result.
Xu and Su [17] have proposed a DWT based enhanced fusion method. On DWT decomposition detailed and approximation coefficients are produced. The approximation coefficients of the IR image are modified using a stretching mechanism for improving contrast. The simple average scheme has been used for combining approximation coefficients. Max-fusion rule was used to combine detailedcoefficients. The fused image is reconstructed using IDWT. This method had produced a better result when compared to similar methods results. The limitation of this method is complexity of fusion method is more. More number of decomposition levels was used to get satisfactory results.
S. Panguluri and L. Mohan [18] have developed VI and IR image fusion algorithm using DWT and unsharp masking. On DWT decomposition detailed and approximation coefficients are produced. The unsharp masking technique was applied to the approximation-coefficients. Later average fusion rule was used for fusing output coefficients generated after unsharp masking. Max fusion rule was used for fusing detailedcoefficients. The resultant fused image had been reconstructed with IDWT. This algorithm has produced high contrast and edge informative resultant fused image. The drawback of the algorithm is that edge improvement of the fused image have done partially only.
Zhan et al. [19] have developed a fusion scheme based on DWT for fusing IR and VI images. Mainly DWT is used for producing both detailed and approximation coefficients. Regional energy-based rule was used for fusing approximation coefficients. Weighted sum and difference of neighboring coefficients based fused strategy was used to integrate detailed coefficients. This fusion technique had produced better target information in the resultant fused image compared to similar research techniques. The limitation of this fusion scheme is that it produced poor background textures in the resultant fused image.
Han et al. [20] have presented VI and IR image fusion method using discrete wavelet transform. For fusing detailed coefficients weighted average rule based on feature selection has been used. Absolute selection value rule was used for fusing approximation-coefficients. This method is producing superior results compared with traditional methods. The problem of this method is that it produced poor illumination fused image.
Shah et al. [21] have introduced an IR & VI image fusion mechanism based on DWT. Here detailed and approximation coefficients are produced by combining curvelet and DWT. The fusion strategy used for combining detailed and approximation coefficients is the mean-max rule. The proposed method fused image obtained better contrast and good sharpness when compared to similar work. The drawback of this method is that for producing a fused image it has used two multi-scale transforms.
Zhan and Zhuang [22] have proposed an IR & VI image fusion algorithm using three stages of the discrete wavelet transform. Mainly different fusion rules were in three different stages. The resultant fused image has achieved more contrast and good sharpness when compared to similar work mechanisms. The limitation of this method is the complexity of the fusion algorithm is more. Since more number of fusion strategies have been used in the fusion algorithm for getting a good result.
The motive of all the above methods is to improve contrast and enhance the edge information of the resultant fused image. If the contrast and texture information of the fused image is more, then the detection of objects becomes easy and also overall scene information can be better understood by humans. To achieve both better contrast and rich texture information of the resultant fused image, the above methods face many problems.
Most of the problems faced are:  In the fusion method, edge improvement of the fused image is done partially only.
 An obtained fused image contains less visible feature information.
 More number of decomposition levels are required for getting satisfactory results. It increases the complexity of the fusion algorithm. It also increases computational time.
 Two different multi-scale transforms are used in the fusion algorithm. This increases the complexity of the fusion algorithm.
 Multiple stages of discrete wavelet transform and more number of different fusion rules has been used in the fusion algorithm. This increases the complexity of the fusion algorithm.
The aim of this paper is to achieve both better contrast and rich texture information resultant fused image. At the same www.ijacsa.thesai.org time reducing the above-mentioned problems. This paper introduces an efficient DWT based fusion algorithm. Mainly in this method morphology hat transform has used for improving contrast. A new fusion rule "Novel mean-weighted fusion rule" is introduced for improving quality of final fused image. Single level decomposition of DWT is used in this method in order to reduce the complexity problem.

III. PROPOSED METHOD
An efficient DWT based fusion algorithm for improving contrast and edge preservation has been presented. First source images are resized into 256 × 256. Morphology hat transform is applied to both resized images to improve contrast. DWT on decomposition will produce both detailed and approximation coefficients. A novel mean-weighted fusion rule is introduced for integrating approximation coefficients. Detailed-coefficients are integrated using max fusion rule. The resultant fused image is reconstructed using IDWT. The proposed algorithm block diagram is shown in Fig. 1.

A. Morphology Hat Transform
Morphology hat transform is a contrast enhancement technique. As shown in Fig. 1, morphology hat transform is applied to both resized images of IR and VI images to improve contrast. It is a combination of two transforms. They are top-hat transform and bottom-hat transform. The purpose of the top-hat transform is to highlight brightness information and bottom-hat transform purpose is to highlight darkness information.
Top-hat transform can be defined in such a way that difference between the original image and its opening image. The expression for top-hat transform is given by OP TH x y f x y f x y  (1) ( , ) OP f x y f s  (2) Where "TH(x,y)" indicates top-hat transform image, "f(x,y)" indicates original image, "f OP (x,y)" indicates opening of "f(x,y)" by structuring element "s".
Bottom-hat transform can be defined in such a way that subtracting the original image from its closing image. The expression for bottom-hat transform is given by Where "BH(x,y)" indicates bottom-hat transform image, "f(x,y)" indicates original image, "f CL (x,y)" indicates closing of "f(x,y)" by structuring element "s".
The final expression for the morphology hat transform is given by Where "MH(x,y)" indicates morphology hat transform image, "f(x,y)" indicates the original image, "TH(x,y)" indicates top hat transform image and "BH(x,y)" indicates bottom hat transform image.

B. DWT
DWT is an efficient multi-scale transform. As shown in Fig. 1, DWT is applied to the output of morphology-hat transform images of both source images. On decomposition, it produces four sub-band images. They are "LL, LH, HL and HH" sub-band images. "LL" sub-band image represents approximation coefficients. It provides average information. The "LH, HL, and HH" sub-band images represent detailed coefficients and they provide edge information. The main advantage of using DWT is that it removes spatial distortions and spectral degradations in the final fused image.

C. Fusion Strategy for Low-Frequency Coefficients
As shown in Fig. 1, "Novel mean-weighted fusion rule" is used to integrate approximation coefficients. Generally, approximation coefficients contain average or smoothening information. Let "F L1 (U,V)" represents low-frequency coefficients produced after morphological hat transform of IR image. "F L2 (U,V)" represents low-frequency coefficients produced after morphological hat transform of VI image. www.ijacsa.thesai.org The formula for novel mean-weighted fusion rule is given by. 12 ( , ) 1* ( , ) 2* ( , ) Here W1, W2 represents weights.
The process for the generation of weights for novel meanweighted fusion rule is shown in Fig. 2. First IR and VI images are resized into 256 × 256. Let "X" represents the resized IR image and "Y" represents the resized VI image. Sharpen filter is applied to the output of the resized image of the IR image.
The 3×3 sharpen filter used here is shown in Fig. 3. The output of the sharpen filter is indicated by "S(X)". The use of the sharpen filter is to highlight the fine details and edges of the IR image.
Histogram equalization is applied to the output of the resized image of VI images. The output of histogram equalization is indicated by "H(Y)". The main principle of histogram equalization is to make all the grey levels participate in image information. So, that high contrast image is produced at the output of histogram equalization.
The formula for calculation of weight "W1" is given by  The formula for calculation of weight "W2" is given by The novel mean-weighted fusion rule is mainly used to enhance brightness and improve the sharpness of edges of original images. So, that clarity of resultant fused image increases.

D. Fusion Strategy for High-frequency Coefficients
As shown in Fig. 1, "Max fusion rule" is used for integrating detailed-coefficients. Mainly detailed-coefficients purpose is represent edge information. Edge information reflects textures of original images. Let "F H1 (U,V)" represents detailed coefficients produced after morphological hat transform of IR image and "F H2 (U,V)" represents detailed coefficients produced after morphological hat transform of VI image.
The formula for max fusion rule is given by 12 The purpose of max fusion rule is to highlight edge information of fused image. So, that it helps to improve textures of the final fused image.

A. Software and Hardware Details
The Mat-lab 2019b software has used for developing fusion algorithm. The features of the computer that is used for developing fusion algorithm are corei5 processor, 4GB RAM and 1TB hard disk.

B. Comparison Methods for Evaluating the Proposed Method
In this paper proposed method fused image is compared with four existing similar work methods fused images.
The above four methods are implemented in this paper.

1) Experimental process of sharpen filter method:
First source images (VI, IR) are resized into 256 × 256. For enhancing edge information, a sharpened filter is applied to the IR image in the spatial domain. The 3x3 sharpen filter that has used in this method is shown in Fig. 4. Applying DWT produces both detailed and approximation coefficients. "Weighted average strategy" has used for integrating both detailed coefficients and approximation coefficients. Finally on applying IDWT produces the final fused image.
2) Experimental process of PCA method: First source images (VI, IR) are resized into 256 × 256. PCA is applied to both the source images for reducing dimensionality reduction. Sharpen filter has applied to the output of PCA of IR image to highlight edge information. The 3x3 sharpen filter that has used in this method is shown in Fig. 5. For improving contrast histogram-equalization has applied to the output of PCA of VI image. "Weighted average strategy" has used for integrating the output of sharpening filter and histogram equalization output. The output of weighted average fusion produces a resultant fused image.

3) Experimental process of enhanced contrast method:
First source images (VI, IR) are resized into 256 × 256. Applying DWT produces both detailed and approximation coefficients. "Simple average strategy" has used for integrating approximation coefficients. "Max fusion strategy" has used for integrating detailed coefficients. Finally on applying IDWT produces resultant fused image.

4) Experimental process of unsharp masking method:
First source images (VI, IR) are resized into 256 × 256. Applying DWT produces both detailed and approximation coefficients. Here unsharp masking has applied to approximation coefficients produced after DWT decomposition. "Average fusion rule" has used for integrating the result of unsharp masking of both approximation coefficients. "Max fusion strategy" has used for integrating detailed coefficients. Finally on applying IDWT produces the final fused image.

C. Subjective Analysis of Results
Subjective analysis of results is mainly used for evaluating the proposed method resultant fused image with comparison methods resultant fused images. TNO image fusion dataset [24] source images (VI, IR) are used in this paper for verifying the working of proposed fusion algorithm. Mainly four sets of source (VI, IR) images are used in this paper. And they are shown in Fig. 6 to 9.
Here Fig. 6 shows the fused results of "soldier image". It gives clear information that the military man and trees are not properly identified in sharpen filter method fused image, PCA method fused image, enhanced contrast method fused image and unsharp masking method fused image due to low contrast and poor texture information. Whereas in the proposed method fused image the military man and trees details are clearly identified. Mainly because the proposed method resultant fused image is providing more contrast and rich texture information. Here Fig. 7 shows the fused results of "traffic image". Due to poor brightness letters on the name board are not visualized clearly in other methods resultant fused images. Whereas in the proposed method fused image letters on the name board is clearly identified due to more brightness.
Here Fig. 8 shows the fused results of the "two-person image". House, two persons and board are observed clearly in the proposed method resultant fused image when compared to other methods resultant fused images. It is because of two reasons. The first one is that the proposed method resultant fused image is having high contrast when compared to other methods resultant fused images. The second one is that texture information is highly improved in the proposed method resultant fused image.
Here Fig. 9 shows the fused results of "soldier in trench image". Solider and trench details are identified clearly in the proposed method resultant fused image when compared to other methods resultant fused images. It is because the contrast is high and also texture information is more in the proposed method resultant fused image.

D. Objective Analysis of Results
Objective analysis of results gives accurate information for evaluating the proposed method resultant fused image with comparison methods resultant fused images. Performance metrics used for experimentation are mean, standard deviation, entropy, mean gradient, spatial frequency, QAB/F and visual information fidelity.
2) Standard deviation: Standard deviation metric gives contrast details of the fused image. The value of standard deviation is high means, it represents more contrast. More contrast means, better visualization of fused image. The formula for standard deviation is given by 3) Entropy: Entropy metric represents the amount of information and texture richness present in the fused image. The value of entropy is more means, the fused image contains more information and textures. It is calculated by.
4) Mean gradient: The mean gradient metric represents the amount of edge information preserved in the fused image. The value of the mean gradient is more means, the fused image contains more edge preservation information. It is calculated by.

  
5) Spatial frequency: The spatial frequency metric gives the activity level information in the fused image. The value of spatial frequency is more means, the fused image provides better overall scene information. It is calculated by. 2 2

6) Visual information fidelity:
The visual information fidelity (VIF) metric is mainly used for assessing image quality [23]. If its value is more means, the fused image provides better visual quality. 7) Q AB/F : It measures edge information of fused image which is transferred from source images [8]. If QAB/F value is more means, the fused image provides better visual quality. It is calculated by.
( , ) ( , ) ( , ) ( , ) Tables I to IV, shows a comparison of fused results for different images. Table I, shows a comparison of fused results  for "soldier image". Table II, shows a comparison of fused results for "traffic image". Table III, shows a comparison of fused results for "two-person image". And Table IV shows a comparison of fused results for "soldier in trench image". From Tables I to IV, it can be observed that the proposed method resultant fused image got better values with respect to mean, standard deviation, entropy, mean gradient, spatial frequency, QAB/F and visual information fidelity when compared with sharpen filter, PCA, enhanced contrast and unsharp masking method fused images. Better values are highlighted in bold. The above results indicate clearly that the proposed method resultant fused image has more contrast, abundant information, and rich texture information.

V. CONCLUSIONS AND FUTURE WORK
This paper introduces an efficient DWT based fusion algorithm for improving contrast and edge preservation. Mainly in this algorithm morphology hat transform is used for improving contrast. For improving brightness and edge information of fused image a new fusion rule "Novel meanweighted fusion rule" is used for fusing approximation coefficients. Single level decomposition of DWT is used in algorithm in order to reduce the complexity problem.
Here proposed method fused image is compared with four existing similar work methods: The methods are 1) Sharpen Filter method.
The above four methods are implemented in this paper. Here four sets of source (VI, IR) images are used for verifying the performance of proposed method. The experimental results clearly shows that our proposed method has produced improved results when compared to similar existing techniques both subjectively as well as objectively.
In future the above work can be extended by introducing deep learning concepts for improving the quality of fused image.