SDME Quality Measure based Stopping Criteria for Iterative Deblurring Algorithms

Deblurring from motion problem with or without noise is ill-posed inverse problem and almost all inverse problem require some sort of parameter selection. Quality of restored image in iterative motion deblurring is dependent on optimal stopping point or regularization parameter selection. At optimal point reconstructed image is best matched to original image and for other points either data mismatch occurs and over smoothing is resulted. The methods used for optimal parameter selection are formulated based on correct estimation of noise variance or with restrictive assumption on noise. Some methods involved heavy computation and produce delay in final output. In this paper we propose the method which calculate visual image quality of reconstructed image with the help of Second derivative like measure of enhancement (SDME) and helps to efficiently decide optimal stopping condition which has been checked for leading image deblurring algorithm. It do not require any estimation of noise variance or no heavy computation are needed. Simulation has been done for various images including standard images for different degradation and noise condition. For test leading deblurring algoritham of Blind and Semi-Blind deblurring of Natural Images using Alternate direction method of minimizer (ADMM) is considered. The obtained results for synthetically blurred images are good even under noisy condition with Δ ISNR average values 0.2914 dB. The proposed whiteness measures seek powerful solution to iterative deblurring algorithms in deciding automatic stopping criteria. Keywords—Image deblurring; stopping point; Point Spread Function; Second derivative like measure of enhancement


INTRODUCTION
Blind image deblurring is ill-posed inverse problem solved with iterative techniques using regularization methods with reduction of solution space.In regularization methods solutions accuracy and smoothness tradeoff is controlled by regularization parameter selection or by deciding optimal stopping condition.The reconstructed image quality is highly depended on the iteration number and above the optimal iteration point each extra iteration result in amplification of noise and computational cost.If the sharp image is available than one can compute full reference image metrics such as mean-squared error (MSE) , peak signal to noise ratio (PSNR) [1], or structural similarity (SSIM) [2] and can decide optimal stopping point when MSE is minimized or PSNR and SSIM is maximized.However, in most practical situations the sharp image is not available and this metrics cannot be used for decision of automatic stopping point determination.
The most commonly used methods for choosing a regularization parameter or for decision of automatic stopping point are as follows.
Visual Inspection: If the prior knowledge of the scene is available one can select the stopping point on the basis of visual inspection of the results.Clearly, as prior information about the scene is not obtainable all the time, the method has very limited application.
L-Curve Method: In any regularization method there are two terms, the data fidelity (residual) and prior information fidelity.The optimal stopping point is determined by plotting the data fidelity term against the prior fidelity term.Graphical behaviour of these two terms forms L-Curve and the optimal stopping point (regularization parameter) is the corner of L-Curve.It requires correct curvature evaluation and can be computationally expensive [3][4][5][6][7].
Discrepancy Principle: If the noise power is known, then the residual norm value can be matched to the noise variance value and on this basis optimal stopping point is selected.If correct noise power is not known it should be estimated and can lead to over smoothing in any inaccurate estimation [8][9][10].Its improved versions are based on residual moments [11].
Generalized Cross-Validation (GCV) Method: GCV is an estimator that minimizes the predictive risk.The underlying idea is that the solution that is obtained using all but one observation should predict that left-out observation well if the regularization parameter is a good choice.The total error for a particular choice of the parameter is calculated by summing up the prediction errors over all observations.The optimal parameter value is the one that minimizes the total error.A search technique or an optimization method could be used to determine the optimal value.it does not require knowledge of noise variance and is known to yield regularization parameters for linear algorithms that asymptotically minimize the true MSE [12][13][14].SURE: Stein's unbiased risk estimate (SURE) is MSE estimation based method and proved to be good alternative to GCV as almost all algorithm work by minimizing MSE criteria.For blind deconvolution it can't be used as it requires knowledge of noise statics [15][16].
Statistical Approach: In reconstruction of sharp image using the statistical methods one can estimate regularization parameter or indirectly optimal stopping point.The methods can be solved using the expectation maximization (EM) technique [17] which alternately restores the image and a new estimate of the parameter is calculated [9].
Whiteness-based method: It measures the whiteness properties such as covariance and power spectral density of the residual image.When residual image is spectrally white, residuals match to the noise having least structure content and it is considered as optimal stopping point [18].All the above summarized methods decide the optimal stopping point without concerning the visual appearance of the reconstructed images.They use the strategy of approximate computation of the metric MSE which is not the idea indicator of the visual quality.So the method is proposed in this paper, which uses SDME measure to decide the optimal stopping point.SDME measure is properly correlated with the noise level and intensity contrast (which indicates the "visibility" [19][20][21]) of the structured regions of an image.SDME measure is modified such that its value drops if the variance of noise rises or if the blur increases in the image.The contribution of the work lies in the Calculation of SDME measure at every iteration and deciding automatically stopping point.This measure exhibits a clear maximum point and helps to select iteration number.As per indication in Table I the proposed method do not require any input from user and no other parameter tuning is needed.It takes care of visual quality of image with no need of excessive computation.So it seeks powerful solution for determining optimal stopping point in regularized iterative algorithm.
The rest of the paper is organized as follows: Mathematical preliminaries described in section II and the proposed technique is given in section III.Results and discussion are given in section IV.Conclusion is summed up in Section V.

A. Linear and Shift-invariant (LSI) Motion Blur Model
In almost all imaging application observed image is degraded version of original image as it is blurred by some function h which is known as blur or point spread function (PSF).Image deblurring means to solve an inverse problem with the aim of restoring an image which suffered a motion blur with additive white gaussian noise.The image deblurring methods can be divided into two groups: non-blind deblurring, where the degradation operator of an image is known, and blind deblurring, where the degradation operator is not known.Normally, Degradation process is nonlinear and space varying process but most of the problems can be addressed with a linear and shift-invariant (LSI) model [22].Output of an LSI system is the convolution of the true image with the impulse response of the system and can be written as, (, ) = ℎ(, ) • (, ) + (, ) (1) where  (, )is the original image that we want to recover from the degraded measurement (, ),where (, ) indicates special coordinates.Here, ' • 'is the convolution operator and η is additive white Gaussian noise involved.

B. Regularized Least Squares Estimation
For any ill posed problem it is difficult to satisfy uniqueness, existence and stability criteria all together.The standard methodology to solve such a problem is to use least squares solution with regularization term. .The least squares estimator minimizes the sum of squared differences between the observed image  (, ) and the predicted image ℎ(, ) * (, ) [22].Regularization term uses prior information about the true image and helps to obtain a solution with desired properties.The cost function to be minimized in regularized least square estimation can be formulated as: Here, h is the PSF kernel to be found, β is the scaling factor and )] ( [ g f r is the regularization function where ) (g f is the edge response.To test the problem we considered deblurring with modification in total variation based solution [23].Most images have sparse leading edge structure and edges are less sparse for its blurred version as area of the blurred edge is larger.The preferred solution should have sparse edges representation.The scaling parameter is at first a large value and then decreased over iterations.The edge responses of the blurred image are defined by the function ) (g f given by [23]: Here d φ is directional filter with φ values of 0, 45, 90 and 135 to find the edge response.The sparse prior, given the edge intensity for a pixel j represented by ( ) k adjusts for the scale of edge intensities and q controls the prior's sparsity and ∈ is a small parameter.Taking the noise into consideration as Gaussian likelihood [23] is given by: The log-likelihood maximization is similar to having the cost function minimized Here, The regularization parameter functions over the edge response and regularizer was chosen which favors the sharp edges or the priors are selected that reach sparser edge response.

Algorithm for getting deblurred image:
1. Initialization 2. Initially set PSF (h) to identity matrix keeping dimension of h bigger than that of the actual PSF matrix.3. Initially set deblurred image (g ) equal to blurred image (z).
4. Set λ ,q and ∈ to initial values Loop 5. Find x =argmin( ( , ) s C g h for given , , g z h λ ,q and ∈ 6. Find h =argmin( ( , ) s C g h )for given , , g z h λ ,q and ∈ .Use the updated x, but keep other parameters constant.
8. Repeat from step 4 to step 6 for each iteration and calculate SDME sum at each iteration Here g is the Deblurred image which has to be found, h is the PSF kernel which has to be found, z is the input normalized blurred image.λ is the scaling parameter which changes over iterations.

C. Second Derivative-like Measure of Enhancement
Second-Derivative-like Measure of Enhancement (SDME) is a visibility operator [19] and a metric for quantitatively assessing image quality [20][21].This visibility operator can be viewed as a second derivative analogue of the Michelson contrast measure.
Suppose the image I is divided into As per the definition SDME works as local contrast descriptor and it is strongly sensitive to the degradation.Intensity variations are resulted in neighboring pixels of sharp image when blurring takes place.Instead of using direct comparison between the center pixel and its neighborhood pixels, SDME uses order statistics such as B max and B min to show sufficient amount of variations.For the digital second derivative spatial differentiation filters the weights are [1 -2 1] and replacement of this spatial weighting is done in the numerator of the SDME contrast with the weighting of order statistics and the central image pixel (1 for B max , -2 for B cen , and 1 for B min ) [25].It is observed that the SDME is quite intimately related with the sharpness and contrast of the local region.
To decide the stopping condition we selected only those blocks which are having dominant orientation.The noisy and blurred patches are less structured and do not have dominant orientation so they are removed in calculating global SDME sum over the image.The patches to be selected are decided on the basis of singular value decomposition of gradient vector of the patch.SDME sum is modified in this proposed method and given by; 2 Where m(i,j) is given by; And r(i,j) is given by; The threshold T is empirically selected as 0.3781 for the patch size of 5 and checked over multiple database images.The above expression contains maximization of sum of SDME value with dominantly oriented patches and provides a reasonable solution to the optimal stopping point problem.If X(t) represent reconstructed images over iterations than at the maximum of SDME(X(t)) the reconstructed image is best matched to original sharp image, hence further iterations will not enhance the image quality, but may add more noise.
To verify the usefulness of SDME sum in the presence of blur and noise we applied motion blur with different length and theta parameter and recorded resultant SDME values.its performance is plotted in Fig. 1.Next, we take noise into account.As shown in the graph the visibility operator SDME sum decrease with increase in both blur and random noise.Same experiment is repeated for the image is corrupted by white Gaussian noise and results are noted for increasing values of noise variance.As per the results obtained the metric has well-behaved characteristics in the presence of both noise and blur and SDME sum values decreases with noise and blur.Thus, SDME has the potential for quality assessment of the blurred image.The proposed stopping criteria are based on measures of the fitness of the estimated data based on SDME .The blurred image was generated by convolution of the original image with a motion blur degradation function with different values of Length and theta and then corrupted by AWGN with different BSNR values dB.We applied the state of art iterative deburring method to restore the blurred version of the image [23].Reconstructed image generated in all iteration is divided into 5*5 image patches.For each patch we computed horizontal and vertical gradients.Local gradient is used to give us structural information of the patch.In order to extract global information of contrast, the patches with dominant orientation is used .The patch is less structured and will not have dominant orientation when it is more noisy so that patches are neglected in computing global image metric.An accumulation of local contrast SDME information of structured patches of whole image is computed, which can be declared as SDME sum.Iteration are continued till SDME sum value become 40 and after SDME value reaches 40 at each iteration SDME value is compared with its previous value and when it reaches maximum (before it start decreasing) we stop the iteration.Any impulsive value of SDME that is above than 300 is considered as outliers (empirically setted based on result of database) and SDME sum value is assigned previous value to remove outliers.The flow chart of the algorithm is given in Fig. 2

A. Dataset Description, Experimental Set Up and Evaluation Metrics
The dataset consists of various standard images such as Cameraman and other images obtained from public databases.The proposed stopping criteria is tested on a wide dataset of grayscale and color images.Table II gives the sample images.We tested a wide range of white Gaussian noise with noise levels BSNR=25, 30 or 35dB was added to the blurred images.The PSF is a motion blur with different values of length and theta.
For the purpose of comparison of stopping criteria we used state-of-the-art discrepancy principle (DP) method.DP method requires knowledge of the noise variance so its two types DPσ and DP MAD [24] are considered.DP σ uses the true value of the added noise and DP MAD which estimate noise variance by MAD (Median absolute derivative) rule.DP σ is considered to be ideal as the true value of the noise variance is unavailable in practice.Numbers of experiments are conducted on numerous test images to validate the use of the proposed algorithm for deciding stopping condition.
The proposed technique is implemented in MATLAB on a system having 8 GB RAM and 2.5 GHz Intel i-7 processor.The evaluation metrics used are: (1) Δ Improve signal-to-noise ratio (ΔISNR) -the difference of best ISNR and ISNR achieved by manual stopping (low value means good quality reconstruction),(2) Structural Similarity (SSIM) index [2] (lies between 0 and 1,closer to 0 means poor quality and closer to 1 means good quality),(3) ΔMSE is used (low value means good quality reconstruction).Performance of the proposed method is compared with the state-of-the-art methods qualitatively (visual aspects) as well as quantitatively using this metrics.

B. Simulation Results
Experiment 1-This experiment is conducted on all images in Table II keeping blurring parameters constant.The image is blurred with PSF parameters, L = 12 pixels and θ = 230 • with BSNR of 30 dB.This blurred image is given as an input to algorithms and the stopping point is decided.The ΔISNR, difference between best ISNR value (with minimum MSE) and ISNR at the iteration where the algorithm stops automatically are recorded.Table III shows iteration stopped with each of automatic criteria and best iteration number with minimum MSE for Length=12, theta=230, BSNR=30 dB and Length=8, theta=120, BSNR=35 dB.As shown in Table VII and Table IV, the reconstructed image quality of proposed method outperforms other algorithms in terms of both visual quality and quantitative evaluation and is close to the best restored image which can be obtained by using minimum mean square error criteria.Table VII shows the simulation results obtained for the proposed technique for three images (robot, cameraman, boat) with degradation parameters length-12, theta-230, noise level-30 dB.All latent images, their blurred versions and final restored image for different automatic stopping criteria are given in Table VII.It is clear that we got better results compared to the state-of-the-art methods.

DPσ DP(MAD)
Fig. 4 gives comparison of all the methods for various degradation applied to all the images in the table.As per the chart, one can see that iteration number selected with SDME sum method is matched to best ISNR iteration in majority cases.DP σ method also seems to be matched with best iteration but the method is based on true value of variance which is unavailable in practice.Results obtained shows nearer or superior results obtained compare to DPσ method which is based on true value of noise variance which is unavailable for blind image deblurring problems.8 shows that visual quality of the reconstructed images for the proposed technique is best compared to other methods.
• The Δ ISNR average values came about 0.2914 dB and 0.32 dB losses with respect to best ISNR.The obtained evaluation matric values confirm the effectiveness of the proposed technique.
• Table IX, X, and XI gives the performance analysis by varying the length, theta and noise level.For robot image the Δ ISNR average values came about 0.3850 dB.

V. CONCLUSION
In this work visibility operator SDME sum is effectively used to decide the optimal stopping point in regularized iterative reconstruction methods which involves intensity contrast of the structured regions of an image.The results are obtained for different images degraded with uniform motion blur.The method calculates the stopping point with less loss of ISNR (0.29dB, 0.32dB) compare to state of art methods and the maximum of SDME sum is located close to the best restored image where MSE is minimum.A large number of experiments are carried out to prove that the SDME measure is highly correlated with noise and contrast of image.The proposed approach is quite general and does not require knowledge about the noise variance.It does not involve any extra parameter tuning and do not allow any visual quality degradation of image.So it seeks powerful solution for determining optimal stopping point in iterative IBD problems.
the maximum and minimum values of the pixels in each block separately, andi j cen B , ,is the intensity of the center pixel in each block, then SDME is defined by the equation:

Fig. 1 .
Fig. 1.SDME sum Vs Noise deviation (b) SDME sum Vs degradation III.PROPOSED METHOD FOR AUTOMATIC STOPPING POINT DETECTION

Fig. 3
Fig.3Shows that initially SDME sum value is increasing, reaches to maximum and then decreases because of noise.The results of ISNR graph, sum of SDME graph and MSE curves are plotted for the robot image of size (240 × 240) are shown in Fig.3.It can be noted that the quality of the estimated image improves as the iteration increases in the beginning, and corresponding to the change of the estimated images, the SDME sum value become higher, too.Then the distortion caused by noise amplification becomes much stronger, and both the estimated images and the SDME values are affected by random noise.SDME sum graph follows ISNR graph and optimal stopping point is the point with maximal value of SDME sum.Maximum ISNR obtained is for iteration number 22 which is shown in graph of ISNR Vs iteration and iteration number selected on the basis of proposed criteria is 21 as shown.

TABLE I .
COMPARISON OF DIFFERENT METHODS .

TABLE II .
SAMPLE IMAGES FROM DATABASE

TABLE III .
TERATION NUMBER SELECTED FOR DIFFERENT METHODS

TABLE IV .
EXPERIMENTAL RESULTS (IN DECIBEL FOR TWO DIFFERENT DEGRADATION PARAMETERS).FIRST COLUMN: BEST ISNR OBTAINED DURING THE ITERATIONS.SECOND, THIRD, AND FOURTH COLUMNS: Δ ISNR FOR SDME SUM , DPSIGMA AND DP(MAD)

and
Table VIII shows MSE values and SSIM values.Table VI gives graphical comparison of all the methods in terms of Δ ISNR and Δ MSE.As per the graph SDME sum is best suited to decide the optimal stopping point.

TABLE V .
EXPERIMENTAL RESULTS (FOR TWO DIFFERENT DEGRADATION PARAMETERS).FIRST COLUMN: BEST MSE OBTAINED DURING THE ITERATIONS.SECOND, THIRD, AND FOURTH COLUMNS: Δ MSE FOR SDME SUM , DPSIGMA AND DP(MAD)

TABLE VII .
SIMULATION RESULTS

TABLE VIII .
SSIM VALUES FOR DIFFERENT METHODS

TABLE IX .
ITERATION NUMBER FOR ROBOT IMAGE BY VARYING PARAMETERS

TABLE X .
SSIM VALUES FOR ROBOT IMAGE BY VARYING PARAMETERS

TABLE XI .
ISNR VALUES FOR ROBOT IMAGE BY VARYING PARAMETERS • The Tables give the performance evaluation of the proposed technique.Table IV gives the Δ ISNR values Table V gives the MSE values and Table VII gives SSIM values.The values are obtained for nine images.Table