Noise Reduction on Bracketed Images for High Dynamic Range Imaging

The quality of a high dynamic range (HDR) image produced from bracketed images taken at different camera exposure times can be degraded by noise contained in bracketed images. In this paper, we propose a noise reduction method on bracketed images based on exposure time ratio. First, for each pixel pair of a same scene point lying on two different images, the ratio of their intensity values is compared with the ratio of exposure times of the images on which the pixels are lying. If the compared ratios are close, these two pixels are included in noisefree pixels set. The complement of noise-free pixels set is defined as noisy pixels set. Then, the intensity value of each pixel in noisy pixels set is corrected by its expected value computed from noisefree pixel of the same scene point lying on another image. Experimental results show that all the noisy intensity values can be correctly restored from noise-free pixels except the saturated pixels. Denoising results by PSNR show that the proposed method outperforms the other recent denoising methods such as based-on pixel density filter (BPDF), noise adaptive fuzzy switching median filter (NAFSMF), and adaptive Riesz mean filter (ARmF). Keywords—Image denoising; high dynamic range imaging; noise detection; noise removal; image restoration


I. INTRODUCTION
The human visual system (HVS) can perceive higher dynamic range (i.e., the ratio between the maximum and minimum intensity values in an image) than most of the cameras. This causes the gap between the visual quality of image captured from standard camera and the quality of actual scene perceived by human eye. One way to reduce this gap is using exposure bracketing technique [1]- [3] which is the process to achieve high dynamic range (HDR) image from merging multiple low dynamic range (LDR) images, which is called bracketed images, acquired by gradually increasing the camera exposure settings.
A typical problem in constructing HDR image is that if there are moving objects while capturing a series of bracketed images, they appear blended in merged HDR image which is called as ghosting artefacts. Thus, a lot of research has been conducted to remove ghosting artefacts in HDR imaging [4]- [9]. Like the standard images, the images acquired in bracketing process suffer from noise that usually occurs during acquisition. And this noise can degrade the quality of the final constructed HDR image. Therefore, the denoising of bracketed images is one of the most important processes to obtain high quality HDR image. Since each bracketed image can be denoised using existing image denoising methods [10]- [14], more attention of research in HDR imaging has been focused on removing ghosting artefacts than image denoising. In this article, we propose a method to exploit information during exposure bracketing process which can be used to reduce noise in bracketed images.
Most of the exposure bracketing techniques use three images: slightly underexposed image, correctly exposed image and slightly overexposed image. Thus, we used three bracketed images in this work. It is found that the exposure time ratio between two differently exposed images is constant and the value is equivalent to the ratio between intensity values of the two corresponding pixels [15]. Therefore, if the ratio of the two corresponding pixels' intensity values is close to the exposure time ratio, these two pixels can be assumed to be noise-free. In this way, all the pixels in the bracketed images are checked whether they are noise-free, and the classified noise-free pixels are grouped into the noise-free pixels set. The noisy pixels set is made by taking complement of the noise-free pixels set. Then, the intensity values of the pixels in noisy pixels set are corrected by their expected values computed from noise-free pixels of the same scene point lying on another image. Experimental results on several datasets illustrate that the proposed method can correctly restore intensity value of any noisy pixel when there exists corresponding noisy-free pixel of the same scene point in another image. However, noisy intensity values of the saturated pixels that have maximum intensity values cannot be correctly restored because once an intensity value reaches a maximum value, it does not increase as the sensor exposure of a pixel increases.
The rest of this paper is organized as follows. Section II presents the proposed algorithm of based-on gamma-corrected exposure time ratio filter (BGEF). In Section III, we present experimental analysis on different datasets. Conclusions are given in Section IV.

II. PROPOSED ALGORITHM
Through the article, we let = � � × be an image where is the measured pixel intensity value at location ( , ), r is the number of pixels in a row of , c is the number of pixels in a column of . The total quantity of light accumulated at camera sensor location ( , ) for units of time produces a sensor exposure ⋅ , where is sensor irradiation at ( , ).
The measured pixel intensity value at location ( , ) is proportional to the sensor exposure ⋅ scaled by the camera response function as ( ⋅ ) . Among various methods to estimate P [16], [17], gamma curve approximation is widely used by many camera manufacturers [18]. Assuming the camera response function to be a gamma curve, the pixel intensity at position ( , ) can be expressed as For each pixel location ( , ) , the gamma-corrected exposure time ratio ( , ) between differently exposed images and is defined in [15] as where and are pixel intensity values at ( , ) in image and respectively.
It can be observed in (2) that the gamma-corrected exposure time ratio is constant for all pixel location ( , ). From (2), we can estimate the pixel intensity value ̂ at ( , ) in image from the observed intensity value in image as ̂= • . Thus, if the estimated pixel intensity value ̂ is different from the observed value , one or both of and can be thought to be corrupted by noise.

A. Selection of Noisy Pixels
We obtain three images 1 , 2 , 3 of the same scene by gradually increasing the camera exposure time ( 1 < 2 < 3 ) and compute 1 2 , 2 3 and 3 1 from (2). For each pixel location ( , ) in each image frame, we compute the estimated pixel intensity value from another image frame as Let be pixel at location ( , ) in image , is noisy pixels set, is noise-free pixels set and is collection of all the pixels. and are initialized as empty set. Gammacorrected exposure time ratios 1 2 , 2 3 and 3 1 cannot be corrupted values since they are computed from pre-determined camera parameters (i.e., exposure time and gamma value) as shown in (2). However, pixel intensity values 1 , 2 and 3 can be corrupted by various types of noises. Thus, for each pixel , we guess from (3) whether the pixel belongs to or as where is predefined tolerance and it is set to 0.1 in this work. When the difference between the observed pixel intensity value and the estimated pixel intensity value is small, both of the two pixels (i.e., the pixel whose intensity value is estimated and the pixel from which the other pixel's intensity value is estimated) can be assumed to be noise-free pixels. Finally, the noisy pixels set is constructed by subtracting noisefree pixels from the collection of all pixels.

B. Correction of Noisy Pixels
The intensity values of pixels classified as noisy in previous section can be corrected from the intensity values of noise-free pixels in another image lying on the same location. Equation (4) indicates that each pixel location ( , ), there are three cases: all three pixels are noise-free, two pixels are noise-free and the remaining one pixel is noisy, all three pixels are noisy. The case that two pixels are noisy and the remaining one pixel is noise-free cannot occur in (4). In this method, the noisy pixel in the second case can be corrected. For each pixel location ( , ), if only one value among 1 , 2 and 3 is noisy, the noisy intensity value can be corrected as In (5), the noisy pixel intensity value is corrected by its estimated value in (3) since the pixel, from which the value is estimated, is noise-free. However, if all three pixels lying on same location are noisy, the noisy intensity values cannot be corrected in this method.

III. EXPERIMENTAL RESULTS
The proposed method was applied on three sample datasets in [6] -'Yard', 'Shop' and 'Cars' scenes. Each dataset has three images of a scene captured by doubling the camera exposure time. The sample datasets are shown in Fig. 1.
Out of several types of image noise that occurs during image acquisition and transmission, the impulse noise is one of common types of noise which includes salt-and-pepper noise (SPN) [19]- [21] and random-valued impulse noise [22], [23]. SPN is caused by sudden disturbances in the signal and the pixels affected by SPN hold a maximum or a minimum gray value. In experiments, we imposed SPN noises on the bracketed images with various ratios.   Fig. 1(a). In these saturated regions, the proposed filter does not work well since the method rely on intensity ratio between images and the intensity ratio on saturated region does not reflect correct exposure time ratio. However, for all three images are corrupted, the denoising quality decreases as the noise level increases. For a certain location ( , ), the proposed method can correct the noisy intensity value when only one pixel is noisy out of three pixels as in (5). It means, when only one image is corrupted by noise, theoretically 100% of noisy pixels except saturated pixels can be corrected by corresponding two other noise-free pixels lying on the same location in other two images. However, when all the three images are corrupted by % noise level, the probability of a noisy pixel can be corrected is equivalent to the probability that the corresponding two other pixels are noise-free which is (1 − /100) 2 × 100 %; thus, only some portion of noisy pixels can be corrected. The same experiments were conducted on 'Shop' and 'Cars' scenes and their denoising results are shown in Fig. 3 and Fig. 4, respectively. In both experiments, we can see the expected results as in Fig. 2. In 'Cars' scene in Fig. 1, the saturated parts around sun in the sky occupy large areas in the image. Thus, the denoising failure around sun is notable in this scene. 154 | P a g e www.ijacsa.thesai.org For quantitative assessment of the effectiveness of image denoising, we use the common error metric peak signal-tonoise ratio (PSNR) which is defined as where peakval is the maximum intensity value in image datatype and MSE stands for mean square error defined as 155 | P a g e www.ijacsa.thesai.org (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 8, 2020 where = � � is a ground truth and = � � is an evaluated (denoised) image. The higher value of PSNR indicates a better image quality.
In Table I, PSNR results of the proposed BGEF method on  middle images are compared with recent SPN filtering techniques − based-on pixel density filter (BPDF) [24], noise adaptive fuzzy switching median filter (NAFSMF) [25], adaptive Riesz mean filter (ARmF) [26]. PSNR results show that BGEF performs better than the other filtering methods in all three scenes. Since BGEF reduces noise in bracketed images using exposure time ratio computed from a priori knowledge, it can be considered as preprocessing technique before applying existing denoising filters. Thus, we also tried to apply BPDF, NAFSMF and ARmF after applying the proposed BGEF, each of which is termed as BGEF-BPDF, BGEF-NAFMSMF and BGEF-ARmF respectively. We can see these combined filters have further improved the results when only one image is corrupted. However, in some cases when all three images are corrupted, the combined filters showed worse performance than single BGEF filter as the noise level increases.

IV. CONCLUSIONS
A noise reduction method based on exposure time ratio on three bracketed images has been proposed. For each location ( , ), the ratio of intensity values of the corresponding pixels between two images is equivalent to gamma-corrected exposure time ratio which can be computed from a priori knowledge of camera parameters. Thus, when a ratio of two corresponding pixels' intensity values is different from gamma-corrected exposure time ratio, one or both of them are classified as noisy pixels. Then, the classified noisy pixels are corrected from noise-free pixels on the same location lying in another image using gamma-corrected exposure time ratio as a cue. Experimental results show that if only one image out of three is corrupted by noise, most of the noisy pixels' intensity values except saturated pixels can be correctly restored by the proposed method regardless of noise level imposed on the image. However, if all the three images are corrupted by noise, the denoising performance decreases as the noise level increases. Experimental results on several datasets using PSNR show that the denoising performance of the proposed method is better than the recent denoising filters BPDF, NAFSMF and ARmF. When the proposed method was used as a preprocessing filter and the existing noise specific denoising filters were applied on the processed images, the denoising performance has further improved.