Method for 3d Object Reconstruction Using Several Portions of 2d Images from the Different Aspects Acquired with Image Scopes Included in the Fiber Retractor

—Method for 3D object reconstruction using several portions of 2D images from the different aspects which are acquired with image scopes included in the fiber retractor is proposed. Experimental results show a great possibility for reconstruction of acceptable quality of 3D object on the computer with several images which are viewed from the different aspects of 2D images.


INTRODUCTION
Medical surgery is possible through a not so large hole using medical surgery instruments such as fiber retractor, image scope, etc.It is called Laparoscopic surgery [1]- [3].Damage due to Laparoscopic surgery is much smaller than the typical medical surgery with widely opened human body and retracts the nidus in concern.In order to make a medical surgery plan, 2D images which are derived from "image fiber scope" are used usually.It is not easy to make a plan because 2D images are not enough.Medical doctor would like to see 3D image of objects entirely.On the other hand, fiber retractor contains not only one fiber scope but also several fibers can be squeezed in one tube (acceptable size of the human body hole).The image fiber scope which is proposed here is containing several fibers in one tube.Anoptical entrance is attached at each tip of the fiber.The several fibers are aligned along with fiber retractor.Therefore, 2D images are acquired with the different fiber image scopes.It is also possible to reconstruct 3D object image using the acquired 2D images with the several fiber image scopes.
Simulation studies are conducted with simulation data of 2D images which are derived from fiber image scopes.3D object image is reconstructed successfully with an acceptable image quality.The following section describes the proposed Laparoscopic surgery with the fiber image scopes which are aligned along with fiber retractor followed by simulation studies.In the process, geometric calibration is highly required for the system together with a high fidelity of 3D image reconstruction.Finally, conclusion and some discussions are described.

II. PROPOSED LAPAROSCOPIC SURGERY WITH THE FIBER IMAGE SCOPES WHICH ARE ALIGNED ALONG WITH FIBER RETRACTOR
A. Laparoscopic surgery Illustrative view of the laparoscopic surgery is shown in Fig. 1.Laparoscopy output of 2D images is monitored by computer display in a real time basis.Looking at the monitor display image medical surgery is operated with surgical instruments.Thus a portion of the nidus of survival lottery is removed with retractor.In order to make a surgery plan, 3D images of the nidus containing survival lottery is highly required.3Dimages can be reconstructed with several 2D images acquired from the different aspects.2D images are acquired with image scope.

B. Image Scope
Outlook of the image scope is shown in Fig. 2. Fig. 2 (a) shows the fiber optical entrance of the image scope while Fig. 2 (b) shows aft-optics of the image scope.Although Fig. 2 shows just one of fiber image scope, the proposed system includes several fiber image scopes into one fiber tube.
Thus 2D images from the different aspects can be acquired with the several fiber image scopes.Then 3D image is reconstructed on the computer using the acquired 2D images.www.ijarai.thesai.org

C. Fiber Retractor
The aforementioned several fiber image scopes into one fiber tube are shown in Fig. 3. Namely, optical entrances of 8 fiber image scopes into one fiber tube, in this case, are aligned along with circle shape of fiber ring.Original shape of this fiber ring is just a line.As shown in Fig. 4, fibers in the fiber tube are closed loop shape at the begging.This is called fiber retractor hereafter.After the line shaped fiber retractor is inserted into human body, the tips of fibers are expanded.The shape of fiber tips becomes circle from the line.Thus the tips of the fiber of which optical entrance and light source aft-optics are attached are aligned as shown in Fig. 3.This is called Fiber Retractor with Image Scopes: FRIS.where [C ij ] is called camera parameter.The camera parameter can be determined by camera calibration.It, however, is difficult to calibrate camera geometry in human body.Therefore, camera calibration is used to be conducted in laboratory in advance to the 3D object image acquisition.In the camera calibration, 2D images, A and B which are acquired from the two different locations are used.Thus four equations can be obtained as shown in equation (2).
(2) Using these equations, all the camera parameters are determined based on least square method.www.ijarai.thesai.org

E. Process Flow of the Proposed 3D Image Reconstructions
Fig. 6 shows the process flow of the proposed 3D image reconstruction with FRIS.First, 2D images are acquired from the different aspects surrounding of the 3D object in concern.Then geometric feature is extracted from the 2D images for tie point matching.Because the two adjacent 2D images are acquired with 60% of overlapping ratio, 3D image can be reconstructed using these 2D images with reference to 3D space coordinate.Thus 3D shape is reconstructed.Then 2D images are mapped onto the 3D image surfaces and rendering is applied to the reconstructed 3D shape.2D images for mapping are created as shown in Fig. 7. Namely, corresponding 3D image coordinate is calculated with the pixels on the 2D image coordinate.From now on, spherical shape of object is assumed to be 3D object shape.In this process, the number of tie points is important because mapping accuracy depends on the number of tie points.Lattice points on the 2D image coordinates are selected as tie points as shown in Fig. 10. Figure 11 shows how to combine two adjacent two image strips into one 2D image for mapping.In this process, the corresponding pixel locations are referred in between in the two adjacent 2D images.www.ijarai.thesai.org

F. Texture Mapping
UV mapping method is applied to the 2D mapping images as a texture mapping.Namely, image coordinate system is converted to the mapped 3D image coordinate system, UV coordinate.3D object shape is converted to the top and bottom view of the UV coordinate systems as shown in Fig. 12. Fig. 13 shows the examples of the top and bottom view of the mapping images

G. Rendering
Finally, rendering is conducted and displayed onto computer screen as shown in Fig. 14.Thus 3D object image is reconstructed in the computer.As shown in Fig. 15, rendering has to be made with smooth surface as much as it could be.Although the textures of the two adjacent 2D images have to be matched each other, both texture patterns do not match perfectly due to mapping error derived from coordinate conversion.Therefore, some smoothing process has to be applied as post processing.Using LightWave3D software tool, a simulation study is conducted.10 cm of diameter of sphere with surface texture is assumed to be an object.Light source is situated at the same location with camera.Camera of which focal length is 33.8 mm with aperture angle of 25 degree is used for simulation study.The distance between the camera and the 3D object is 20 cm.When the 3D object is acquired with the camera, the cameras are assumed to be aligned along with the circle with every 20 degree of angle.Therefore, 60 % of overlapping 2D image acquisition can be done.Corresponding points for tie point matching are extracted manually.Fig. 16 shows the simulation result with the aforementioned procedure.At the top left of Fig. 16 shows top view while the bottom left shows front view of the reconstructed 3D object images.Meanwhile, the top right of Fig. 16 shows oblique view while the bottom right of Fig. 16 shows side view of the reconstructed 3D object.All these images are reasonable.
This representation of 3D object image is specific to the LightWave3D software tool.Another example is shown in Fig. 17.If the lattice point locations are given for the top view, front view, and side view, then 3D object image is appeared on the top right of the window of the computer screen.Even if the real 3D object image is complex shape and texture as shown in Fig. 18, the proposed method may create 3D object image onto computer screen.Further investigations are highly required for making smooth texture surfaces between two adjacent 2D images.

Figure 1 .
Figure 1.Illustrative view of Laparoscopic surgery

Figure 2 .
Figure 2. Outlook of the image scope

Figure 3 .
Figure 3. Proposed fiber retractor with image scopes for 3D image acquisitions

Figure 6 .
Figure 6.Process flow of the proposed 3D reconstruction with 2D images acquired with FRIS.

Figure 8 .
Figure 7. Creation of 2D images for mapping.

Figure 9 .
Figure 9. Rotation conversions with the different angles.
Figure 10.Tie points (corresponding points between two adjacent 2D images)

Figure 11 .
Figure 11.Method for combine two adjacent 2D images

Figure 12 .
Figure 12.UV coordinate for the top and bottom view Fig.15 (a) shows a potion of 3D object surfaces while Fig.15 (b) shows side view of the reconstructed 3D object image.

Figure 18.
Figure 18. Figure 17Sub-window assignments for the top view, the front view, the side view and the reconstructed 3D object image for LightWave3D software tool Figure 18. Figure 17Sub-window assignments for the top view, the front view, the side view and the reconstructed 3D object image for LightWave3D software tool C