Denoising in Wavelet Domain Using Probabilistic Graphical Models

Denoising of real world images that are degraded by Gaussian noise is a long established problem in statistical signal processing. The existing models in time-frequency domain typically model the wavelet coefficients as either independent or jointly Gaussian. However, in the compression arena, techniques like denoising and detection, states the need for models to be nonGaussian in nature. Probabilistic Graphical Models designed in time-frequency domain, serves the purpose for achieving denoising and compression with an improved performance. In this work, Hidden Markov Model (HMM) designed with 2D Discrete Wavelet Transform (DWT) is proposed. A comparative analysis of proposed method with different existing techniques: Wavelet based and curvelet based methods in Bayesian Network domain and Empirical Bayesian Approach using Hidden Markov Tree model for denoising has been presented. Results are compared in terms of PSNR and visual quality. Keywords—Guassian Mixture Models (GMM); Hidden Markov Model (HMM); Discrete Wacelet Transform (DWT); Hidden Markov Tree (HMT)


INTRODUCTION
Removing noise from an image with less loss of information is referred as Image Denoising.Even the simplest acquisition, processing and transmission of image can subject it to unwanted noise.Denoising is required at every level whenever there is a question about dealing with images [1].It finds its applications ranging from image enhancement in terms of highlighting some useful aspects of information from an image to achieving better quality of medical images (CT scan, Ultrasound etc).
There have been a number of techniques and algorithms designed in wavelet domain.All of them worked successfully because of primary properties that wavelet transform has that is; locality, multi-resolution and compression.In [2], a framework for signal denoising in time frequency analysis domain using Hidden Markov Model (HMM) has been proposed.A neighbouring coefficients thresholding method is applied in multi-wavelet framework for denoising in [3].A non-parametric tree based model for joint statistics of wavelet coefficients has been discussed in [4].To realize neighbouring dependency between wavelet coefficients across scales, a generalized Multivariate Gaussian distribution has been proposed in [5].In [6], another tree model using hidden markov tree structure has been developed that refers to local parameterization for image denoising.In [7], wavelet shrinkage function is used to check neighbouring coefficients dependencies for image denoising.In [8], a denoising scheme has been realized that incorporates dependency of wavelet coefficients with three scale dependency.In [9], another image denoising technique based on neighbouring wavelet coefficients has been proposed.
Although there exists a number of denoising algorithms but there is still a need for improvement.Further research and study can result in an improved performance and better image quality.In this work, we have developed HMM based image denoising algorithm which is used in the context of 2D Gaussian Mixture Models (GMM) and 2D DWT.EM algorithm iteratively finds the maximum likelihood of a fundamental distribution from a given data set when the data is said to have some missing values.It is best suited in HMMs where the hidden states are not observed and the data is said to be missing.This modelling framework summarizes the nature of wavelet coefficients in a probabilistic way.This model finds its flexibility to the two main features; one is the Mixture Densities for dealing with the non-Gaussian nature of coefficients of wavelet transform by modelling them with hidden states as Mixture Distributions.The second feature is that of Probabilistic Graphs which models coefficients interdependencies and represent them in the form of a tree structure.Several local and standard images are put to test to check the performance parameters comparable to different other existing techniques for denoising.Results are shown both in terms of Peak Signal to Noise Ratio (PSNR) and the quality of denoised image.

II. STATISTICAL IMAGE MODELLING USING HIDDEN MARKOV TREE MODEL
Wavelet transform is known to be favorable because of its properties such as non-Gaussianity, Clustering and Persistence [5], [7], [8].Fig. 1, shows 2D discrete wavelet transform www.ijacsa.thesai.org(DWT).In this figure the decomposition upto 3 levels is shown.HMM combined with these properties provides an attractive model for capturing the non-Gaussian parameters of the wavelet coefficients that are persistent across scales [2], [10].The details of wavelet transform and wavelet domain HMM along with applications in statistical image processing can be found in detail in [1], [2], [11], [12], [13] and [14]., the pdf will be Gaussian with mean and variance as and respectively.The overall pmf will be; where ( ) is the probability mass function (pmf) and is the conditional pmf given by the following equation; Owing to persistence across scale, state transition matrices shows the parent  child state-to-state links between hidden states given as; [ ] where shows that given being the parent coefficient state, the child coefficient is in state [15].HMT model on the whole is specified by pmf of the node , a state transition matrix and , means and variances, of the wavelet coefficient given being in the state k.All these parameters are combined together in a vector .The state transitions and variances are generally different for each wavelet coefficient.This can lead to a more complex HMT model.To reduce the computational complexity, a method referred to as tying within scale is implemented as discussed in [2].
H. Chipman has shown that GMM is capable of well approximating this non-Gaussian density in [16].GMM has a generative model with a random variable Z and marginal distribution as ∑ ( | where represents the co-variance matrix of coefficients.A multidimensional GMM is referred to as HMT.To capture the non-Gaussianity of the wavelet coefficients which is referred to the Clustering property, it associates each of the wavelet coefficients magnitude | | with a hidden variable ( ) that is said to be unobserved.We find few of the coefficients with large magnitudes and they tend to contain the maximum amount of information regarding the image.On the other hand, the coefficients with smaller magnitudes have the lesser amount of relevant information about the image but they tend to exist in large numbers.This leads to the simplest model with only two states; "high" and "low" referring to large and small magnitudes of wavelet coefficients respectively.Several results have proved this model to be simple in nature but effective in estimation point of view.
To capture the inter-scale dependencies between the wavelet coefficients, referring to the Persistence property [15], [17], GMM performs Markovian Chain across scales that are tree-structured as the magnitude of the coefficients is said to be only depending on the size of their respective parents alone.This means that the probability of a coefficient to be "large" or "small" is determined only by the magnitude of its parent.HMT in time-frequency domain was developed in [2] that connect the state variables vertically.For images it forms a structure in a quad-tree fashion that associates each hidden state variable as parent to its four children.This model satisfies both secondary properties that wavelet transform has such as Clustering and Persistence.HMT is applied to image processing in [11], [13] and [15].

III. MODEL TRAINING VIA EM ALGORITHM
In this section a denoising model has been discussed that uses EM algorithm to determine noise-free coefficients from noisy elements.Translating the problem in image denoising, there is a need to determine noise-free coefficient c from the noisy coefficients y.Estimate such that = + , where .Generalizing it, this model refers to determining the vector denoted by .The following relation of expectation describes the sufficient statistics of the model for variable X and the hidden state variable Z; The conditional pmf of hidden states and its maximization is given by following expressions;

∑
The noise-free coefficients can be obtained from the following expression;

[ | ∑
Fig. 2 shows the denoising scheme used in this research work.The proposed scheme for image denoising can be summarized as follows; www.ijacsa.thesai.org4) Train the HMT model using EM algorithm with reference to tying within scales method 5) Apply inverse wavelet transform to estimate the noisefree coefficients to obtain the denoised image

IV. EXPERIMENTAL RESULTS
We have tested standard as well as local gray scale images in order to compare this scheme with several other denoising methods that are used.Each image is of 256x256 dimensions.We have assumed that each image is corrupted with Gaussian noise with three different known variances 10, 20 and 30.The image corrupted by noise is decomposed into three levels of wavelet transform by using Daubechies-8 Wavelet.Each sub-band is then applied by the proposed denoising scheme.

TABLE I. PSNR VALUES OF DENOISED IMAGES USING DIFFERENT METHODS
Standard images include Lena, Camera Man, Peppers and Barbara.These images are tested using the proposed method comparable to other techniques including Wavelet Transform using Bayesian Network Approach [18], Curvelet Transform using Bayesian Network Approach [18] and Empirical Bayesian Approach using HMT [15].Also, some local images are also used that are tested based on the PSNR of the noisy image compared with the denoised one.
It can be noticed from the table I that our proposed scheme gives a better performance in terms of PSNR values.We have compared our method in terms of PSNR at different noise variances.Our technique shows comparable results to that of other techniques.It can be noticed that Empirical Bayesian approach performs well when used in HMT domain.Comparable results are found in proposed work when the HMT is initialized by EM algorithm.Also, the standard image "Lenna" is tested for visual quality.Using our proposed method, it has been shown in fig. 3 that image quality is comparable to other techniques with less blurring.The different denoising techniques have been compared graphically as well.Fig. 4, shows the graphical representation of different techniques at 3 different noise variances.
From many years MOS (Mean Opinion Score) method has been used for subjective analysis of an image, video or voice quality.This technique refers to the averaged value of the opinions taken from the users [19].This method scores the quality of the image from 1 (worse) to 5 (excellent).Table II shows different classes of MOS ranging from 1 to 5.
We have performed MOS analysis on three different images that are tested using different techniques in order to have an idea about visual quality of the images.Fig. 5, shows the results taken from a group of people and then averaged over the total number of observations.The graph shows that visual quality of images using our proposed technique has the comparable score to other techniques.The denoising algorithm proposed is also applied on several images taken from the institute as well.These images are tested at different noise variances and noisy image is compared with the denoised image in terms of Fig. 6, shows the results taken after applying the algorithm.

IV. CONCLUSION
We have proposed HMM based image denoising method in Time-Frequency analysis domain.It has been shown that the proposed algorithm outperforms the existing few techniques based on PSNR values.The proposed framework concisely models the non-Gaussian statistics of the individual wavelet www.ijacsa.thesai.orgcoefficients.The EM algorithm used for training the HMT model captures the statistical dependencies between the coefficients that are tested using some standard and local images comparable to other techniques.This technique gives comparable results as compared to HMT that uses Bayesian approach to model the coefficients.Our method shows improvement of the denoised images as compared to other techniques based on both the PSNR values and in terms of visual quality of the images.

Fig. 1 .
Fig. 1.Three level decomposition using 2D Discrete Wavelet Transform For each wavelet coefficient magnitude | |, we can associate a set of Hidden states ( ) with it.Given, the pdf will be Gaussian with mean and variance as and respectively.The overall pmf will be;

Fig. 2 .
Fig. 2. Proposed Denoising Scheme 1) Add AWGN noise in the real-world image (NxN dimensions) after converting it into gray scale 2) Apply Daubechies-8 DWT on the image to divide it into three levels of DWT3) Estimate the initial GMM parameters of each sub-band from the noisy coefficients.4)Train the HMT model using EM algorithm with reference to tying within scales method 5) Apply inverse wavelet transform to estimate the noisefree coefficients to obtain the denoised image