A Technique for Panorama-Creation using Multiple Images

Image stitching, which is a process of integration of multiple images to create a panoramic image using all contents fitted into one frame, finds wide-spread applications in medical, high resolution digital map, satellite and video imaging. This paper proposes a framework to develop panorama image with multiple images. The framework is an automatic process that takes multiple images, checks correlation of the sequential images and removes overlapping area if exists and creates the panorama.We have done experimentations using different imagesets consisting multiple images with and without overlapping and got satisfactory results. Keywords—Panorama; image stitching; correlation; multiple images; image features


I. INTRODUCTION
Image stitching is a system for creating panoramic images. The overlapping areas of input images can make noticeable seam among the images [1]. So, it is basically a procedure for recognizing and eliminating those seams of the overlapping regions and blending [2].
Image stitching is usually applied in various applications, such as, X-ray image stitching, HDR (High Dynamic Range) image stitching, image stabilization, high resolution photo mosaics in satellite photos, medical imaging, digital maps, multiple-images with super-resolution, video stitching, microscopy image stitching, object insertion and group photographs/panoramas.
There are numerous techniques to construct panoramic images. Image stitching is generally a software-based method for making panorama. It takes several standard images to wrap up the complete viewing area and stitches collectively all those images to produce a panorama [3], [4]. Alignment or registration of images is a requirement for stitching images. It is done considering translation, scaling and rotation of images [5].
There are different types of stitching methods [6]. Direct technique is extremely efficient for stitching images which do not contain any overlapping area. Image stitching using correlation [7], [8] is one of the most primordial stitching techniques, which is mainly an intensity-based method and appropriate for images with overlapping areas.
Another method is feature-based stitching [9], which locates all corresponding feature points in every image pair. Then it evaluates all the features in one image against features in another image through feature descriptors. Feature extraction, registration and blending are different steps required for feature-based image stitching.
Two-image stitching is very common; however, multiple image stitching is somehow tricky. Therefore, the main purpose of this paper is to create a panorama using multiple images. We have prepared our paper as follows: Section II illustrates the literature review. Section III describes a concise representation of our method of multiple image stitching. Section IV presents the experimentation and assesses the performance of our method. Then, Section V comments on future works along with challenges. Finally, Section VI concludes the paper.

II. LITERATURE REVIEW
Many researchers have already been worked of image stitching. A comprehensive evaluation on panorama creation methods is presented below: Adel et al. [10] presented a feature-based image stitching based on ORB (Oriented FAST (Features from Accelerated Segment Test)) and Rotated BRIEF (Binary Robust Independent Elementary Features).
Zomet et al. [11] proposed a method of seamless image stitching by minimizing false edges. In this approach, their main target was on seam removal.
Mclauchlan et al. [12] presented an image mosaicing process using sequential bundle adjustment. The newness of this approach is the transfer of photogrammetric bundle adjustment and line measurement which enables to use lines in cameracalibration.
Hua et al. [13] presented a method of image stitching based on SIFT (Scale-Invariant Feature Transform) and MVSC (Mean Value Seamless Cloning). It is basically a SIFT feature detection and matching oriented technique. Szeliski [14] presented an image alignment and stitching method. The core focus of this technique is alignment of the images.
Fatah et al. [15] proposed an automatic seamless image stitching technique. Qiu et al. [16] proposed a SIFT (Scale Invariant Feature Transform) and transformation-parameter based image stitching method.
Jia et al. [17] proposed a method of image stitching using structure deformation. Their technique is for getting the stability in image intensity. Ostiak et al. [18] proposed a totally automatic HDR (High Dynamic Range) panorama stitching method which used SIFT for the identification of the matching feature points.
Fang et al. [19] presented a feature-based stitching method of a clear/blurred image pair. Koo et al [20] proposed a (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 2, 2020 feature-based image registration algorithm for image stitching applications on mobile devices.
Zhao et al. [21] presented a self-adaptive algorithm based on Harris-corner detection. Wang et al. [22] presented an automatic image stitching technique based on graph model. In this method, they have used Weighted Shortest Path algorithm and Dijkstra algorithm for multi-image stitching. Yang et al. [23] proposed a phase-correlation and Harris operator-based image-mosaicing process. This approach is a correlation and feature based hybrid method.
All techniques have some advantages and disadvantages [24]. Through these observations, in this paper we have presented methodology to stitch multiple images. Our method presents a robust stitching system where we determine the correlation among the images, if overlapping exists between the images, then we should remove overlapping area from the first image and merge the second image after first one to create panorama and assign newly created panorama as first image and another image as second.
On the other hand, if overlapping does not exist between the images, we will merge second one after first image and produce a panorama and again, we will assign this panorama as the first and another image as the second. We should repeat the above steps till nth inputted image to create the final panorama.

III. PROPOSED ALGORITHM
We have proposed a simple but efficient and robust algorithm for panorama-creation using multiple images. The proposed algorithm is as follows: Term the merged Image as I(n) 16 Input image I(n − 1) 17 Display I(n) In the proposed method, n represents the number of images, c(λ) is the correlation coefficient of two consecutive images, λ represents the width of the overlapped region and T H is the limit of c(λ) and its value set to 0.2. The steps of our proposed algorithm are detailed and explained below:

A. Image Acquisition
We must input n images I(n), I(n−1), . . . , I(3), I(2), I(1) to continue the whole process. But, at first, two consecutive images I(n) and I(n − 1)should be inputted. Then the rest of the images should be inputted sequentially for the further execution of the process depending on the criteria and conditions. Then the images are converted to gray-scale. Another important thing is all of these images must be aligned.

B. Determination of Correlation
The next step is, determining of correlation. Correlation usually used to find correspondence of two images [25]. So, we have to compute correlation coefficient c(λ) between two images A and B using the following formula, Where, A = mean(A xy ), B = mean(B xy ).

C. Removal of Overlapping Region and Panorama Creation
If the value of correlation coefficient c(λ) surpasses the threshold value T H, then we should remove overlapping region λ from I(n) and merge I(n − 1) after I(n) to create panorama and decrement the value of n and assign newly created panorama as I(n) and input next consecutive image as I(n − 1) and go through the previous step.
On the other hand, if correlation does not exceed the threshold value T H which means there is no common region between the images and we don't need to remove any overlapping region. So, we will merge I(n − 1) after I(n) to generate a panorama and then decrement the value of n. After that,we will assign the newly-produced panorama as I(n) and acquire the next consecutive image I(n − 1). The above steps will be repeated depending on different measures and conditions till the last input image to create the ultimate panorama image.

IV. RESULTS AND DISCUSSIONS
The experiments are done on ten image-sets using our technique. We have prepared input images from these ten original images using Microsoft Office Picture Manager. In seven image-sets, first 5 images contain a portion of overlapping region as well as repetition, the next 4 images does not contain any overlaps and 9th and 10th images contain a segment of overlapping area, but in the next three image-sets, all the images contain some amount of overlapping area.
Then, we used these images to generate the panorama using our stitching method. All the outputs are generated using MATLAB R2018a with Microsoft Windows platform, Intel Core i3, 3.50 GHz processor and 4.00 GB RAM. As a sample we have demonstrated the results of four image-sets here. Figure 1 depicts an image (2725×600 pixels) which is considered as a ground truth. This image is cut into ten images shown in Figure 2. Among those, the first five consecutive   images contain some overlapping regions, then the next four do not have any overlaps but the last two images contain some overlaps. Then we inputted all these images consecutively for the proposed technique to create desired panorama. Here, Figure 3 shows the final panoramic image.
Similar to Figure 1, we have done the similar actions on an X-ray image(2325×650 pixels) shown in Figure 4. So, it is cut into ten images which are depicted in Figure 5 and Figure 6 shows the generated panorama using the proposed technique. Then, we have taken another image (2325×600 pixels) as the ground truth (shown in Figure 7) and cut it into ten images which are shown in Figure 8. Figure 9 demonstrates the panorama of these images using our stitching method. Similarly, Figure 10 shows another original image (2275×600 pixels), Figure 11 presents ten parts of this image and each of these parts contain some portion of overlaps. Figure 12 shows the panorama generated from these images using our proposed method.  Table I depicts the performance (accuracy and computation time) of our proposed technique. We have used the following equation to calculate the accuracy:     In Eq. (2), I 1 is the ground truth and I 2 is the output of the proposed method.

V. FUTURE WORKS AND CHALLENGES
In this paper, we have experimented with natural-, HDRand X-ray-image and got satisfactory outputs. Multiple image stitching is a challenging issue. Seam is another important issue needs to be eliminated. Different methods have been presented in the recent years and we have a vision to work on seam elimination for producing quality panorama. The experiments are done on sequential images with straight edges, but, we have a desire to work on unsequenced images with uneven edges which is another challenging matter.

VI. CONCLUSION
In this paper, we have publicized a structure to construct a panoramic image using sequential multiple images. Some of these images consist of overlapping regions and some images do not have any overlapping area. We have eliminated the overlapping area from the first one before joining the next image after it and assigned the newly produced image as the first one and inputted the next consecutive image as the second.