An Example-based Super-Resolution Algorithm for Multi-Spectral Remote Sensing Images

This paper proposes an example-based superresolution algorithm for multi-spectral remote sensing images. The underlying idea of this algorithm is to learn a matrix-based implicit prior from a set of high-resolution training examples to model the relation between LR and HR images. The matrixbased implicit prior is learned as a regression operator using conjugate decent method. The direct relation between LR and HR image is obtained from the regression operator and it is used to super-resolve low-resolution multi-spectral remote sensing images. A detailed performance evaluation is carried out to validate the strength of the proposed algorithm. Keywords—Remote sensing Super-resolution; Image-pair analysis; Regression operators


INTRODUCTION
Remote sensing is vital for various application such as decision support for disaster management, weather monitoring and surveillance of land [1]. The data provided by Geo Information System (GIS) is significant for remote sensing. In recent years, sophisticated imaging devices and state-of-the-art technologies have been continuously deployed in the Earth observation system to provide precise data for remote sensing. The need to have a high spatial resolution for satellite images is twofold: apart from the improvement in visual inspection of a larger dataset precisely, it also plays a vital role in the postprocessing steps such as feature extraction and segmentation of objects from the image. However, despite using sophisticated high-resolution (HR) imaging devices for satellite imagery, the captured images will inherit a poor spatial resolution due to the larger distance between the sensors and the sensed object. cartosat series of satellites are used to regularly monitor earth for disaster management.. These satellites have evolved over past few decades and currently the Cartosat-3 with the advanced imaging device is in existence. The panchromatic (PAN) and multi-spectral (MS) imaging devices are deployed in these satellites to provide progressive imaging. The PAN images have high spatial and low spectral resolution whereas MS images have high spectral but low spatial resolution. The spatial resolution of an MS image captured by cartosat-2 series satellite will be approximately 2.5 meters/pixel.
The modern image sensor element used in MS imaging device is typically a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) activepixel sensor. The image signals are captured by the sensor elements that are typically arranged in a two-dimensional array. The size of sensor element or the number of sensor element present in a unit area determines the spatial resolution of an image. The spatial resolution of MS image is significantly less due to the limited dynamic range of CCD sensors. An imaging device with deficient sensor elements will generate low-resolution (LR) images with blocky and displeasing visual artifacts due to aliasing effect. However, deploying more sensor elements to increase spatial resolution will incur additional cost.
Moreover, the limitation to deploy high precision optics in imaging device is diploid. In addition to the cost incurred due to the increase in sensor elements, the ceaseless demand to improve the spatial resolution cannot be catered by the state-ofthe-art camera technologies. For instance, reducing the pixel will increase the spatial resolution but will introduce shot noise [2]. Similarly increasing the number of pixels in a unit area by increasing the chip size can increase the spatial resolution. However, increasing the chip size will increase the capacitance which results in undesired artifacts [3].
Due to the inherited limitations, the spatial resolution of MS images will be poor. Anyhow, in many applications including disaster management, rescue operations, resource surveying, etc. precise geo spatial information is required. Henceforth, it is significant to use an effective post-process technique such as image super-resolution (SR) approach to improve the spatial resolution of MS images. The need to improve the spatial resolution of remote sensing imagery have garnered special interest by researchers and have witnessed diverse SR algorithms [4][5][6][7][8][9][10][11][12].
Spurred by the need to improve the spatial resolution of Landsat images, Tsai et al. presented a conventional multiimage SR algorithm in frequency domain [4]. Classical SR algorithm requires multiple frames of the same scene with exact registration to super-resolve a LR image [4][5][6][7]. To overcome this, a wide variety of learning-based single image SR algorithm have been proposed [8][9][10][11][12]. SR from single LR satellite image is a challenging task as the problem is severely ill-posed. However, learning-based SR algorithms can effectively handle the ill-posed problem by learning an efficient prior to model the relation between low and HR training image patches. The prior required to handle the illposed problem can be either explicit or implicit. Explicit priors use a mathematical energy functional of an image class such as primal sketches, Field of Experts (FoE) [13], Gradient profile [14] etc. to model the relation between LR images with its HR counterpart. In contrary, implicit priors are learned from the training image pairs and give rise to a family of SR algorithms called as Example-based SR algorithms [15]. It requires a collection high-quality example images and synthetically generated LR images to learn the image-pair prior information. The correspondence between LR images with its corresponding HR image is learned as an implicit prior.
The implicit prior can be learned either by a direct mapping approach or an indirect mapping approach depending on the patch reconstruction strategy used. The indirect mapping approaches employ nearest neighbor embedding algorithm [16,17], which requires an exhaustive search to find the nearest neighbor which makes it computationally expensive for practical applications like satellite remote sensing. Direct mapping approaches will learn the relation between LR and HR image as a regression function, thereby computationally it will be efficient to super-resolve remote sensing images [18]. Despite, most of the conventional regression based SR algorithms vectorizes the image patches which results in loss of image-level information while learning the implicit prior. To address this recently a few matrix-based implicit priors have been reported [19] [20], which avoids the vectorization step and learn the implicit priors as a matrix-based regression operator. The regression operator establishes a direct mapping between the training image patches and can be effectively used to reconstruct the HR image.
In this paper, we propose an example-based SR algorithm to super-resolve spatially under-sampled cartosat-2 series MS images by learning an efficient matrix-based implicit prior from a set of HR-MS images. The proposed matrix-based implicit prior will preserve the structural similarities in the image thereby will not introduce any unpleasant artifacts.
The reminder of this paper is as follows. A brief discussion on implicit prior is presented in section 2. In section 3, the methodology of the proposed example-based SR algorithm for MS remote sensing image is presented. The performance of the proposed algorithm is evaluated and the results are reported in section 4 and finally, section 5 concludes the paper.

II. A BRIEF DESCRIPTION ON IMPLICIT PRIORS
The fine details that are explicitly missed during the degradation process are estimated by an example-based SR algorithm. Fig. 1 illustrates the process of example-based SR algorithm. Though the problem is severely ill-posed, efficient priors are used for regularizing the solution. For an example-based SR algorithms, the prior will be learned from a set of training examples itself and hence they are named as implicit priors. Training examples will be a collection of HR images and its corresponding synthetically generated LR images. An implicit prior which models the relation between LR image patches to its corresponding HR image patches can be learned as a regression function. In most of the state-of-the-art SR approaches, the regression function will be learned by vectorizing the training image patch-pairs. 319 | P a g e www.ijacsa.thesai.org In vector-based implicit priors, the correspondence between LR-HR image pair is learned from the feature vectors and hence instead of learning the image-level correspondence; feature-level correspondence is learned. In matrix-based implicit prior, the image patches will be preserved as matrix itself and therefore the image-level structural information will be preserved in the image. It is vital to preserve the structural information in an MS remote sensing image as it will possess a lot of HF details. In literature, quite a few matrix-based implicit prior have been reported. In these methods, the correspondence between LR and HR image patches will be learned as linear matrix-based regression operator.

A. Matrix-based Implicit Prior
The matrix-based implicit prior is learned as regression operator as follows. Let a set of training example patch-pair be denoted as = ( , ) =1 such that and are the LR and HR patches of size × respectively. A linear matrix-based regression operator : ∈ ℝ × serves as an implicit prior, such that for an image patch-pair ( , ) = .
(1) If the image patches are assumed to be full rank, then the regression operator can be defined as (2) However, as the regression operator will be learned from a collection of LR-HR patch-pairs, it is required to find a suitable optimal regression operator by solving a least square regression problem. The optimal regression operator can serve as an implicit prior to learn the correspondence between LR and HR image patches.

III. METHODOLOGY OF THE PROPOSED ALGORITHM
The overview of the proposed SR methodology is shown in Fig. 2. The example-based SR algorithm to super-resolve MS images captured by cartosat-2 series satellite will typically have two phases, viz. training and reconstruction phase. In training phase, the required prior information is learned as a matrix-based implicit prior which is performed offline. In the reconstruction phase, the learned implicit prior is used to reconstruct the HR image.

A. Training Phase
In the training phase, high-quality MS images are collected from a remote sensing database (For instance, to super-resolve existing cartosat 2 series MS images with a spatial resolution of 2 meters, cartosat-3 images with a spatial resolution of 1 meter is collected). These HR images are synthetically degraded to obtain the LR images. The degradation process includes a blur operator which is modeled by the movement of sensor element and a decimation operator, which corresponds to the insufficient sensor elements. Also, atmospheric noises can degrade the quality of satellite images. Step(2): Calculate the initial estimate,

Step(3): Calculate the optimal regression operator
Output: Optimal regression operator, R* 320 | P a g e www.ijacsa.thesai.org (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 7, No. 9, 2016 As the patch-pairs are extracted from the same location, the linear regression model can be adopted to relate the LR and HR patch-pairs. Therefore = (5) The matrix-based implicit prior is learned from the training patch-pairs by solving a least square regression as follows.The objective function to learn the optimal regression operator is given by * = argmin ‖ − ‖ 2 (6) In the above equation, let the initial estimate of the regression operator is obtained by taking the inverse of , such that Let the global constraint to estimate the regression operator is given by The above optimization problem to find the optimal regression operator is solved by conjugate gradient decent method. The term � − � � 2 is the priori for the optimization problem. This is an iterative approach and the update equation for the iteration is given by [21], is the error due to the ℎ iteration and is the learned regression operator after ℎ iteration. The optimal regression operator is used to reconstruct the HR image. Algorithm 1 summarizes the steps involved to learn the optimal regression operator.

B. Reconstruction Phase
In the reconstruction phase, the LR MS cartosat images are super-resolved using the matrix-based implicit prior which is learned as a regression operator given by Eq. (6). The test LR MS image is up-scaled by an interpolator by a scale-factor . Non-overlapping patches of size × are extracted from the interpolated image. The collection of the extracted patches is represented as a set = � � =1 . All the test LR patches are super-resolved using the matrix-based regression operator, given by The super-resolved patches are merged to obtain the superresolved HR image . The steps to super-resolve a LR MS image is summarized in Algorithm 2.

IV. RESULTS AND DISCUSSION
The effectiveness of the proposed SR algorithm to superresolve LR multi-spectral image is evaluated on a set of remote sensing images captured by cartosat-1 satellite as shown in Fig.  3. All the experiments are simulated in MATLAB using a personal computer with Intel core-i5-2400 @ 2.7 GHz processor with 4 GB RAM. To generate training examples, HR multi-spectral images captured by COMSAT-1 are collected. Sample training images are shown in Fig.4. These HR images are downgraded with a scale-factor using bi-cubic interpolation to synthetically generate the LR images. In all the experiments, the patch-size is 11 × 11 and the LR images are super-resolved by a scaling factor of 2 and 4. The test images are shown in Fig. 3 are super-resolved by various state-of-theart SR algorithms such as Yang et al.'s sparse representation based approach [22], and Dong et al.'s non-local autoregressive modeling (NARM) [23]. The results of the above algorithms are obtained using the source code available on the author's webpage. Step (2): Obtain the HR patches using regression operator obtained from Algorithm-1, Step ( Qualitative evaluation of SR depends on a few attributes of the reconstructed SR image. The image is visually inspected for its naturalness and sharpness to assess the quality of the reconstructed image. The sharpness of an image is assessed based on the high-frequency details present in it. It is desired that the SR algorithm should not introduce any counterfeit HF details. Similarly, image naturalness is attributed to the distortions and artifacts present in the image. If the fine-details in the image are not preserved, it will introduce jaggy and ringing and staircase artifacts. These artifacts will severely affect the quality of the image. These attributes in the images can be evaluated by visual comparison of the images.  The quantitative measure to evaluate the quality of the reconstructed image is figured by the PSNR (peak signal-tonoise ratio) and SSIM (structural similarity index measure).A high PSNR score indicates that the magnified image is free from distortions and is more likely to carry HF details. SSIM value [24] (typically close to 1) indicates the similarity structure between the reconstructed image and its ground truth.
The PSNR of an image is defined by, is the width of the image patches and , is height of both the image patches.
The SSIM of the reconstructed image is obtained using   Table I summarizes the quantitative comparison of the proposed method with various SR algorithms on multispectral LR images. The results tabulated in Table-1 shows that the proposed SR algorithm has the highest quantitative measures compared with other state-of-the-art algorithm. From Table-1, it is evident from the PSNR index that the proposed algorithm reconstructs the HR image with minimum distortions and the high SSIM index validates that the image-level information is preserved by the proposed matrix-based implicit prior.

V. CONCLUSION
In this paper, an example-based SR algorithm to superresolve multi-spectral remote sensing image is presented. The proposed SR algorithm will learn a matrix-based implicit prior to map the correspondence between LR and HR images. The implicit prior is learned as a regression operator using the conjugate decent method. The learned matrix-based implicit prior is effectively used to super-resolve clean LR multispectral remote sensing images. In the future, the proposed algorithm will be extended to super-resolve noisy MS remote sensing images. The proposed algorithm is evaluated on clean images. Qualitative and quantitative experiments on various remote sensing images validates the efficacy of the proposed algorithm.