Variational Formulation of the Template-based Quasi-conformal Shape-from-motion from Laparoscopic Images

—One of the current limits of laparosurgery is the absence of a 3D sensing facility for standard monocu-lar laparoscopes. Significant progress has been made to acquire 3D from a single camera using Visual SLAM (Simultaneous Localization And Mapping), however most of the current approaches rely on the assumption that the observed tissue is rigid or undergoes periodic deformations. In laparoscopic surgery, these assumptions do not apply due to the unpredictable and elastic deformation of the tissues. We propose a new sequential 3D reconstruction method adapted to reconstructing organs in the abdominal cavity. We draw on recent computer vision methods exploiting a known 3D view of the environment at rest position called a template. However, no such method has ever been attempted in-vivo. State-of-the-art methods assume that the environment can be modeled as an isometric developable surface: one which deforms isometrically to a plane. While this assumption holds for paper and cloth-like surfaces, it certainly does not fit human organs and tissue in general. Our method tackles these limits: it uses a non-developable template and copes with natural 3D deformations by introducing quasi-conformal prior. Our method adopts a new two-phase approach. First the 3D template is reconstructed in-vivo using RSfM (Rigid Shape-from-Motion) while the surgeon is exploring – but not deforming – structures in the abdominal cavity. Second, the surgeon manipulates and deforms the environment. Here, the 3D template is quasi-conformally deformed to match the 2D image data provided by the monocular laparoscope. This second phase only relies on a single image. Therefore it copes with both sequential processing and self-recovery from tracking failures. The proposed approach has been validated using: (i) in-vivo animal data with ground-truth, and (ii) in-vivo laparo-scopic videos of a real patient's uterus. Our experimental results illustrate the ability of our method to reconstruct natural 3D deformations typical in real surgery. I. INTRODUCTION Over the last few years significant efforts have been made toward developing systems for computer aided laparosurgery. The main goal is to assist the surgeon during the intervention in order to improve their perception of the intra-operative environment as described by [1]. 3D sensing can aid laparo-surgery by providing different view points of the abdominal cavity and is one of the major possible improvements to the current technology. Various methods for intra-operative 3D sensing have been recently proposed. they can be classified as active and passive. The active approach consists of techniques that acquire …


I. INTRODUCTION
Over the last few years significant efforts have been made toward developing systems for computer aided laparosurgery.The main goal is to assist the surgeon during the intervention in order to improve their perception of the intra-operative environment as described by [1].3D sensing can aid laparosurgery by providing different view points of the abdominal cavity and is one of the major possible improvements to the current technology.
Various methods for intra-operative 3D sensing have been recently proposed.they can be classified as active and passive.The active approach consists of techniques that acquire depth information by emitting calibrated wave beams (visible like structured light or invisible like infra-red).[2], [3] have proposed an approach based on the detection of a laser beam line is described.This approach requires the insertion of two monocular endoscopes: one for projecting the laser beam and one for observing the projected laser beam.[4] have proposed a prototype of ToF (Time-of-Flight) endoscope for which [5] has set up an incremental algorithm for 3D reconstruction which has shown promising results for the use of ToF endoscopes.Active approaches require one to modify the endoscope's hardware and may alter the surgeon's view.The passive approaches use only 'regular' images from the laparoscopes: both stereo and monocular endoscopes are concerned.[6], [7], [8] have proposed a set of methods based on disparity map computation for stereo-laparoscopy.A Visual SLAM method for dense surface reconstruction using a stereo-laparoscope has been proposed by [9].In the context of monocular laparoscopy, very few methods were attempted: Visual SLAM with soft deformations by [10], and RSfM by [11].The accuracy of reconstructed 3D shapes for these methods depends on the ability of the state model to account for complex phenomena occurring in the environment such as the use of surgery tools which may introduce unpredictable deformations.Errors may accumulate through navigation and produce artifacts in the reconstructed 3D shape.Some further developments have been made in the specific context of periodic deformations.Recently, [12] and [13] have proposed a method for 3D reconstruction of the beating heart and deforming liver under cyclic respiration respectively.The cyclic deformation was modeled as a linear combination of basis shapes.These methods cannot be used in laparoscopy where the cyclic deformation assumption does not hold.
The computer vision community has recently established www.ijacsa.thesai.orginteresting techniques in template-based monocular 3D reconstruction of deformable surfaces.Template-based methods provide a dense geometric description of the surface rather than just a sparse or partially dense description as in the previously cited methods.This allows one to render the surface from a new viewpoint, recover self-occluded parts, and opens applications based on Augmented Reality.We propose a novel approach to DSfM (Deformable SfM) that is well-adapted to the laparoscopic setting.Specifically, we extend recent 2Dtemplate-based deformable methods for developable (paperlike) surfaces proposed by [14], [15], [16].These methods reconstruct a 3D surface from sparse feature matches between the known template and a single view.Existing methods were designed for inextensible-developable surfaces.However, inextensibility is not a property generally satisfied by living tissue, and so these methods cannot be applied in laparoscopy.
Our contribution is to extend these works to handle the reconstruction of tissues and organs in the abdominal cavity.
Our work is based on introducing a deformable prior which handles elastic deformations.It is based on the assumption that for such surfaces, deformations tend to locally preserve angles and tolerate minor changes in area.This type of deformation is called quasi-conformal, and generalizes isometric deformations by allowing local isotropic stretching to happen.While classical NRSfM (Non-Rigid SfM) methods reconstruct soft or cyclic deformations our method reconstructs complex and unpredictable deformations.Moreover the fact that our method is based on the usage of a monocular single view prevents the reconstruction from accumulating errors like sequential NRSfM methods.This paper extends our previous work, [17], in several directions: (i) we provide a variational formulation of the quasi-conformal 3D reconstruction approach, (ii) we propose a new initialization step specifically designed for extensible surfaces using SOCP (Second Order Cone Programming), (iii) we provide results with 3D reconstruction of in-vivo organs with comparison to ground-truth 3D data, and (iv) all the results are compared to template-based isometric 3D reconstruction from a single view.
Paper organization.Section II presents the related work.Section III describes our 3D reconstruction system.Section IV presents the 3D template reconstruction.Section V gives a geometric characterization of smooth surfaces.Section VI gives our variational formulation of the 3D reconstruction of quasi-conformal surfaces.Section VII presents a discretization of the variational problem.Finally section VIII reports experimental results and section IX concludes.Our notation will be introduced throughout the paper.

II. RELATED WORK
In the absence of priors, the problem of template-based monocular 3D shape recovery is ill-posed because there is an infinite number of 3D surfaces that can project to the same image data.It is then of critical importance to constrain the problem to have a unique consistent solution or at least a small set of plausible solutions.Over the years, different types of constraints have been proposed which can be categorized in statistical and physical constraints.Statistical constraints often model the deformation as a linear combination of basis vectors which can be learned offline or online.These have been used either for human face reconstruction in the works by [18], [19], [20] or for generic shapes in the works by [21], [22], [23].Non-linear learning methods were applied in human tracking by [24], [25] and then extended for more generic surfaces by [26].NRSfM methods also rely on learned linear models to constrain the relative motion of 3D points.Early approaches proposed by [27] used known basis vectors, but the idea was extended to simultaneously recover shape and deformation modes from image sequences as shown in [28], [29].
Early approaches in physics-based modeling involve minimizing the sum of an internal energy representing the physical behavior of the surface and an external energy derived from image data as proposed by [30].Many variations have been proposed, such as balloon forces as used by [31], deformable quadrics and thin-plates under tension as proposed by [32].In works by [33], physical constraints are used as priors within a coarse-to-fine shape basis statistical model.Recently, an important physical prior, the isometry constraint, has been introduced by [14], [15] within a robust framework.It imposes that any surface geodesic distance is preserved after deformation.
In our work, we propose a reconstruction method which handles extensible, complex and unpredictable deformation.We propose to introduce a quasi-conformal constraint to model the deformation of the abdominal cavity organs as being locally isotropic with low tolerance to changes in local areas.While this models quite well the environment, a direct consequence is that the template cannot be taken as flat anymore, as was assumed by most previous methods.Our method thus reconstructs a 3D template shape using classical RSfM by taking advantage of the exploration phase where the surgeon navigates with the laparoscope inside the abdominal cavity.The reconstructed model is deformed afterwards at the surgery phase to fit the different shapes taken by the tissues, thereby providing 3D shape at run-time from a single image.Our algorithm is here dubbed DSfM (Deformable SfM).The technical part consists of three major improvements over state-of-the-art: (i) dealing with quasi-conformal instead of isometric surfaces, (ii) using a 3D instead of a flat 2D template and (iii) creating a custom 3D template using RSfM.This paper introduces template-based 3D reconstruction methods to 3D vision in laparoscopy.

III. OVERVIEW OF DSFM
As depicted in figure 1, our DSfM system has two main phases: 1) 3D template reconstruction.In this phase the 3D structure of the environment is recovered, by assuming that the scene remains approximately rigid as the surgeon  1. Principle of our DSfM (Deformable Shape-from-Motion) approach.In the first phase the surgeon explores the abdominal cavity without deforming it; RSfM (Rigid Shape-from-Motion) is used to find the 3D shape called the 3D template.In the second phase, the 3D template is used to infer the 3D shape deformed as observed from only a single laparoscopic view.This makes the approach resistant to registration and tracking errors and well-adapted to live sequential processing.
explores it with the laparoscope.Using a camera selfcalibrating RSfM algorithm ( [34]), a 3D point cloud representing the organ's shape is reconstructed.The 3D point cloud is then meshed to provide a dense 3D surface, parameterized on the 2D plane via conformal flattening as ( [35], [36]).Because this step takes thirty seconds in general, it has no major impact on surgery workflow.
2) Deformable 3D shape reconstruction.The surgeon is free to proceed and manipulate the target surface, and consequently induces non-rigid deformations with the surgery tools.Here, the template reconstructed in phase 1 is used to perform 3D reconstruction from raw laparoscopic images.The 3D shape is computed by conformally deforming the template such that its 2D projection in the laparoscopic image minimizes the template-to-image registration error.

IV. 3D TEMPLATE RECONSTRUCTION
At this stage, the surgeon explores the environment without manipulating it with tools.It is thus assumed that in this phase the environment remains approximately rigid.We capture the exploratory video and we track a set of feature points with the KLT tracker ( [37]).Since in the exploration phase the laparoscope is moving around the area of interest, we can have a set of frames where features which were not visible either because of specularity or because of occlusion become visible and then trackable.Note that a feature does not need to be tracked over the whole image set gathered in the exploration phase.We use RSfM to get the cameras intrinsic parameters and a 3D sparse point cloud from the tracked points.Specifically, we use the so-called stratified approach; we first compute a projective reconstruction from detected and tracked interest points.Then we self-calibrate the camera by upgrading the projective to a metric reconstruction.Details and variants of the stratified approach can be found in the literature in [34], [38], [39].
For the projective reconstruction, we combine the fundamental matrices estimated between consecutive views from the point tracks.We finally launch bundle adjustment to finely tune the reconstruction.This process outputs N 3D points (x j , y j , z j ), j = 1, ..., N .We then reconstruct a dense 3D surface from the point cloud.Assuming that the surface is smooth and well represented by the point cloud, this can be achieved well by Moving Least Squares ( [40]).The surface is bounded by a manually marked region of interest in one of the images, and texture mapped using that image.The surface is triangulated to form a mesh with N f faces F and N v vertices V. Finally, we map the mesh to the 2D plane via a conformal transform ( [35], [36]).The results of applying this method on an invivo video sequence from laparosurgery on a uterus is shown in figure 2. The 3D template mesh was reconstructed using a real in-vivo sequence acquired by a Karl Storz HD laparoscope during a hysterectomy surgery.In the exploratory phase, where the operator navigated the laparoscope over the uterus, 300 frames of 1280 × 720 pixels resolution were captured over 12 seconds.300 correspondences were tracked over the sequence, and the corresponding point cloud was used to reconstruct a dense surface via Moving Least Squares (MLS) surface reconstruction ( [40]).The resulting 3D mesh has 500 faces and 285 vertices.Note that the number of frames for template reconstruction does not have any bounded values as far as a decent 3D point cloud representing the 3D shape is obtained.Finally, a quasi-conformal transform is applied to flatten the 3D surface.In the next section, we introduce some basic concepts of differential geometry which will be used in our formulation.

A. Parameterization of Smooth Surfaces
A smooth surface Γ can be parameterized by a continuous C 2 -function Φ of two variables q = (u, v) ∈ Ω : We do not make a distinction between the surface Γ and the mapping Φ unless needed.The Jacobian matrix of Φ, denoted J Φ , is given by: It is a 3 × 2 matrix which at each q = (u, v) ∈ Ω maps its neighborhood to the tangent plane of Γ at Φ(u, v).The first fundamental form I Φ is defined as: It is a 2 × 2 matrix which locally maps distances from Ω to Γ.
The second fundamental form II Φ characterizes the curvature at different locations on the surface.It is a second order form on the tangent plane defined as a 2 × 2 matrix: where the dot stands for the scalar product.N(u, v) is the vector normal to the surface at point Φ(u, v) and: are 3-vectors.

B. Classical Surface Mapping
We may distinguish between three classic mappings which do not change the surface topology: isometric, conformal, and equi-areal.If Γ is an isometric surface, then I Φ is the identity.If Γ is conformal, i.e. angle preserving, then I Φ is of the form: where ϕ : Ω → R controls the amount of local isotropic scaling.If Γ is equi-areal, i.e. area preserving, then:

C. Surface Deformation Measurements
When the surface is deformed from Γ to Γ without changing its topology, the parameter space Ω does not change while the surface function varies.Such a variation changes some geometric properties of the surface like the length of the geodesics, the area and the curvature.It is known from differential geometry that the first and second fundamental forms can be used to measure these deformations ( [41]).For instance, given two surfaces Γ and Γ , the Frobenius norm of the difference between the first fundamental forms of the two corresponding surface functions Φ and Φ : measures the extensibility of the geodesics between the two shapes.The norm of the difference between the second fundamental forms of these two deformations: measures the change in curvature.Our variational formulation of the 3D reconstruction of quasi-conformal surfaces is based on these measures.

A. Problem Statement
Given the template surface function Φ, our objective is to retrieve the current surface function Φ given a single image after deformation.Function Φ minimizes the norm of 165| P a g e www.ijacsa.thesai.org the difference between the reprojected 3D points and their corresponding 2D points in the image (see figure 3 for a nondevelopable surface): where K is the 3 × 3 intrinsic matrix established in the exploration phase.Π: R 3 → R 2 : (x, y, z) → ( x z , y z ) is the projection of a 3D point to the image plane.W(u, v) establishes a continuous mapping between points of the template surface and their correspondences in the input image.In practice, such a function is replaced by a discrete set of N c 3D/2D point correspondences {Φ(u i , v i ) ↔ (u i , v i ) } i=1,...,Nc .Here (u i , v i ) is the pixel position in the deformed image corresponding to the point Φ(u i , v i ).
The formalization of the 3D reconstruction problem as the minimization of the functional (10) is under-constrained and we can obtain an infinite number of deformations as illustrated in figure (4).Depending on the nature of the surface, additional geometric priors are required.We use the surface's first and second fundamental forms.The 3D reconstruction problem can then be posed as a variational problem where the unknown is the functional Φ : This is the sum of three terms.The first term is the data fidelity term.The second two terms are used to enforce deformation priors.We split this into two components; the term E b is used to penalise non-smooth bending of the surface.
The term E e is used to penalize deformations which do not agree with the intrinsic material properties of the surface.
In the research literature, E e has been instantiated previously using an isometric prior which associates higher energies to extensible deformations.Although not immediately applicable for extensible surfaces, a convex approximation to problem (11) has been formulated by [14] for inextensible surfaces.
We review now this formulation in the next paragraph.

B. Isometric Surfaces
It is known that isometric and developable surfaces such as paper can be isometrically flattened to the 2D plane without stretching (see figure 5).Consequently, a planar template can be used, and any 3D embedding of the surface must be isometric with respect to this plane.Now, because the first fundamental form of planar surfaces is the identity matrix 1, the 3D reconstruction problem can be written as: where: If Φ is an isometry we can choose Γ ≡ Ω then Φ is the identity map.Thus the bending term ( 9) can be approximated by the second derivatives:

C. Quasi-Conformal Surfaces
Unlike isometric developable surfaces, a quasi-conformal surface cannot be flattened to a plane without inducing stretching or shrinking as shown in figure 3. Quasi-conformal surfaces include both extensible and non-developable surfaces.In the abdominal cavity, the organs are often extensible and nondevelopable surfaces: uterus, liver, kidneys, etc.For modeling the deformations of such organs, we could identify the mechanical properties of each of these different tissues.However, according to the patient (age, health of the organ, etc), the mechanical properties of the tissue would change.Our current 166| P a g e www.ijacsa.thesai.orgsolution uses a differential geometry approach rather than mechanical models.For a quasi-conformal deformation Φ , our constrained variational formulation of the 3D reconstruction is stated as: where: and: with I Φ = ϕ 0 0 ϕ and ϕ(u, v) is a real, positive scalar.
E c softly constrains the 3D embedding to stretch or shrink isotropically.Since local isotropy implies that angles on the surface are preserved, this therefore penalises non-conformal embeddings.By contrast, in E a we softly enforce equal determinant of the first fundamental forms, and this constrains the area between template and deformed surfaces to be locally equal.The priors are weighted by λ c and λ a respectively.Consequently, by setting λ c and λ a accordingly, we can relax the isometry constraints and tolerate either angle or area changes.Crucially, we have found experimentally that changes in areas should be tolerated more than in angles, allowing the surface to locally-isotropically deform.The bending term is weighted with a small λ b relatively to λ c and λ a to allow curvature changes and to obtain smooth 3D reconstructions.Problem ( 15) is non-convex and its resolution needs a descent initialization before minimization with a non-linear optimizer.
In the next section, we describe how we resolve problem (15).

A. Initialization
This initialization step allows us to have a proper initial estimate of the deformed shape using an SOCP formulation in the case of quasi-conformal surfaces.
1) Previous Approaches: In the case of isometric surfaces several SOCP formulations have been proposed.These formulations rely on the principles that a 3D surface point Q lies on the sightline linking its image projection (u , v ) and the camera center.It is obvious that this constraint is enough to fit the image reprojection constraint but since it does not have any constraint on the surface shape these have to be supplied by other geometric constraints.A pointwise SOCP formulation for isometric surfaces was proposed by [14].It is based on the observation that the euclidean distance between two surface points Q i and Q j cannot be greater than the geodesic distance d ij for any possible isometric deformations (see figure 6).The geodesic distances can be easily computed as the euclidean distances of the isometrically flattened template.The formulation is stated as: maximization of the depth is controlled by the euclidean distance between the 3D points which cannot be greater than the corresponding geodesic.I and τ are small real values which model the tolerance to noise in the correspondences and in the template.An SOCP formulation for isometric surfaces with mesh representation is proposed by [26].The reconstruction of one frame relies on the reconstructed mesh of the preceding frame.For the first frame, the initial pose of the mesh is assumed to be known and a failure in one frame can cause failures to chain over the video.
2) Our Formulation Using SOCP: In our work, the previously described formulations with SOCP cannot be directly used since they are not designed for quasi-conformal surfaces (see figure 7).Indeed, they cannot handle non-developable and extensible surfaces.
Let us denote V the set of vertices of the mesh of the deformed surface Γ .In our work, the 3D-2D correspondences between points x i , i = 1, . . ., N c in the template mesh and points (u i , v i ) , i = 1, . . ., N c in the deformed image are assumed to be known.In the triangular mesh they are expressed in barycentric coordinates: with a i , b i , c i ∈ [0, 1] and v j,1 , v j,2 and v j,3 are vertices of the face f j .Our first SOCP formulation of the 3D reconstruction of the deformed mesh can be stated as follows: where x i is the new location of the 3D correspondence point in the deformed mesh.κ is a real parameter chosen so that edges are able to shrink or to stretch.As expected, when the depth is maximized and the vertices move toward the correct sightline, the global shape of the surface can be corrupted since the edges are allowed to extend or to shrink.To avoid obtaining meaningless 3D reconstructions, a smoothing term based on a discrete laplacian is added.It ensures a global resemblance between the deformed surface and the template surface.Moreover, this smoothing term preserves the shape in occluded areas.Indeed, in non-developable surfaces like the uterus, it is mandatory to be able to handle occlusions since it is not possible to have a single view which covers the whole surface.
In the discrete differential geometry of 2-manifolds, there are various formulations of the discrete Laplace-Beltrami operator as described by [42].The one we use in our implementation is the linear combinatorial formulation expressed as: with N (i) the one ring neighbor of vertex i and #N (i) is the cardinal of this set.The norm of l i represents the discrete approximation of the mean curvature at vertex v i ( [42]).Allowing smooth changes of the norm of this vector over the mesh vertices allows us to keep the global shape of the surface.Then, an additional constraint can be added in our formulation of equation ( 20): with κ s a positive value which controls the tolerance to curvature change.In our implementation we use the YALMIPtoolbox ( [43]) to compute the solution of our SOCP formulation with κ s = 0.1.Even if problem ( 20) is convex, its solution is not optimal mainly because in practice the correspondences never cover densely the template surface.The refinement is done by using a discrete version of the variational formulation of equation (15).

B. Refinement
In our formulation of equation ( 15), the local non-isometry constraint is expressed as the sum of a local isotropy constraint and a local equi-areal constraint.The weights associated to each constraint allow us to penalize either the angle variation or the area variation of a local region during the deformation.Equivalently, using a triangular mesh representation of the surface, each triangle can be subject to shearing and anisotropy scaling for any quasi-conformal deformation.Henceforth, equation ( 15) can be re-formalized for a triangular mesh surface as: S i and A i are the 2D shearing and anisotropy scaling transforms from the template to the deformed i th face, λ an and λ sh are two real positive weights that tune the amount of penalty for shearing, anisotropy scaling, and the smoothing energy term.The inextensible formulation enforces the edges of the triangles to remain constant when fitting the data correspondence constraint.In contrast, this weighted combination of quasi-conformal transforms relaxes the inextensible condition and allows us to deal with local extensible deformations.S 0 and A 0 are local maximum amounts of shearing and anisotropy scaling for each face of the 3D template mesh.They can be either learned from training data or experimentally set.Practically, normalized shearing and anisotropy scaling 168| P a g e www.ijacsa.thesai.org).The weights λ an , λ sh and λ s are respectively set to 0.11, 0.14 and 0.12 using the method described by [45].They hardly enforce the motion term to fit the correspondence constraint and fairly constrain the shearing, the anisotropy scaling and the smoothness to allow the triangle to freely deform.

A. In-Vivo Data With Ground-Truth
To obtain in-vivo datasets with ground-truth we use two synchronized laparoscopes to explore and deform the abdominal cavity of a living pig.The experiment is done in the Centre International de Chirurgie Endoscopique (CICE 1 ) under respect of ethical constraints.We used two synchronised laparoscopes to construct ground-truth for metric comparison.To cope with the difficulty of having a non-constant rigid transforms between the two laparoscopes we put a reference checker-board inside the abdominal cavity.This checker-board allows us at any frame to register the left and right views to obtain ground-truth 3D information.In the first exploratory step we reconstruct the 3D template of three different organ's tissues: the bladder and the pericardium.The obtained shapes are shown in figure 9.In the deformation step, the bladder and the pericardium are deformed with the checker-board tool.A set of 100 deformed image frames are taken for each tissue.For our reconstruction method we use on average a set of 40, 25 and 30 point correspondences respectively for the bladder and the pericardium.They were generated using SIFT ( [46]).Outliers and points outside from the organs in concern were removed by the method proposed in ( [47]).In figure 10 we show a subset of different 3D reconstructions using our method from single views for different amounts of extensibility and curvature change with respect to the templates.We can see that globally our method gives meaningful 3D reconstructions according to the deformed images.Note that the features on the deformed regions with the quasi-conformal constraint give consistent recovery of the deformation.These observations are confirmed quantitatively in figure I where we show the reconstruction errors with respect to stereo and with comparison to isometric reconstruction.The reconstruction errors are computed with all the sets of images as the norm of the difference between the stereo 3D points and the reconstructed 3D points of each organ's tissue.Our method gives an order  of magnitude more accurate with an average of 5mm error on the 3D reconstruction.

B. Surgery In-Vivo Data
To validate the proposed approach on real in-vivo data, the experiment we propose is the 3D reconstruction of an uterus from in-vivo sequences acquired using a monocular Karl Storz laparoscope.The frames are acquired at 30 fps and have a resolution of 1280 × 720.The 3D template of the uterus is generated during the laparosurgery exploration step as previously described.Complex and unpredictable deformations may occur on the uterus when the surgeon starts to examine it.A set of 75 correspondences between the flattened uterus template and the deformed images were used.They were generated using SIFT ( [46]).Outliers and points outside from the uterus region were removed by the method proposed in ( [47]) (table 11, row 2).In figure 11, rows 3-4, we show the 3D reconstructed deformations with the corresponding deformed image in row 1.In row 4, we show synthesized views from novel camera views, and show qualitatively that the deformed uterus has been reconstructed well.

C. Discussion
Our experimental results have shown the effectiveness and the improvement of our approach above a previous method proposed by [14] relying also on a single view and a template (c.f.table I for quantitative comparison).State-of-the-169| P a g e www.ijacsa.thesai.orgart NRSfM methods for non-isometric deformations are only sequential for soft or cyclic deformations relying on deformed shapes at precedent time of the current deformed frame.Our approach relies on a template which can be more accurately recovered before starting to reconstruct deformed shapes.Moreover, it uses only a single image and thus does not rely on any temporal priors.

IX. CONCLUSION
In this paper, we have presented a new method to reconstruct a quasi-conformal deforming living tissue in 3D using a single laparoscopic image and a 3D template that is previously reconstructed using standard RSfM.Our method provides novel technical contributions and also a new way of tackling the 3D vision problem in laparoscopy.The experimental results show the effectiveness of our approach and clearly improve the state-of-the-art isometric reconstruction method.
The performance of our 3D reconstruction algorithm depends on the point correspondences between the template and the deformed image.When the tracking system may miss some features our approach can be joined together with shading approach in order to recover the 3D shape of those featureless regions.We are currently working on improving the matching between the template and deformed image and supplying our approach with shading cues in featureless regions.Finally, it would be interesting to investigate a mechanical modeling approach in future work.where {l i } i=1,...,N is the set of curvatures of the deformed mesh and {l 0 i } i=1,...,N is the set of curvatures of the template mesh.In order to evaluate the performance of our approach,

APPENDIX
Fig.1.Principle of our DSfM (Deformable Shape-from-Motion) approach.In the first phase the surgeon explores the abdominal cavity without deforming it; RSfM (Rigid Shape-from-Motion) is used to find the 3D shape called the 3D template.In the second phase, the 3D template is used to infer the 3D shape deformed as observed from only a single laparoscopic view.This makes the approach resistant to registration and tracking errors and well-adapted to live sequential processing.

ΦFig. 2 .
Fig. 2. 3D template reconstruction during the exploration phase using RSfM.(a): feature points are tracked through the video frames.(b): a sparse point cloud is extracted.(c): the 3D points are meshed and texture-mapped.(d): the resulting surface is conformally flattened.

Fig. 4 .
Fig. 4. Without prior, template-based monocular 3D reconstruction of a deformable surface is an ill-posed problem.All the shapes (a, b,c, d, e) project to the same correspondences in the deformed image.To retrieve the correct shape (c), additional constraints have to be added.

Fig. 6 .
Fig. 6. 3D reconstruction using SOCP with isometric formulation.(a) In a flat shape, the Euclidean distance is equal to the geodesic distance.(b) In a non-flat shape of an isometric surface, the Euclidean distance is lower than the geodesic distance.(c) This last observation allows one to put an upper bound constraint when the depth is maximized.

Fig. 7 .
Fig. 7. 3D reconstruction using SOCP with extensible formulation.It is obvious that the Euclidean distance from the flat shape cannot be used in the case of quasi-conformal surfaces.Instead we use an upper bound of extended template edge lengths.Further constraints on curvature preserving are added to keep a meaningful reconstructed shape.
transforms are experimentally set and then scaled by the triangle area of each face f i to obtain the transforms S 0 i and A 0 i .In all our experiments we set S 0 scaling and shearing for each triangle of the mesh.The additional weighted energy term smoothes the deformed shape with a tunable weight λ s .It is expressed through the linear Laplace-Beltrami discrete linear operator ∆ of dimension N × N ([44]

Fig. 8 .Fig. 10 .
Fig. 8. Shape and geometric measures of the 3D template surface.Left: texture-mapped 3D template surface.Middle: the length of the edges.Right: the conformal curvatures, in radians, computed at each vertex of the mesh ([42]).

Fig. 11 .
Fig.11.3D reconstruction on an in-vivo video sequence from a monocular laparoscope using our quasi-conformal method.First row: Single 2D views of uterus deformation with a surgery tool.Second row: Point correspondences between the template and deformed images.Third row: 3D reconstruction using our quasi-conformal method.Each 3D reconstruction is done using the single view above.The view is given in the laparoscope's view point.Fourth row: 3D deformed surface seen from a different point of view which provides visualization of the self-occluded part.Fifth row: Zoom in the deformed area.
Percentage of deformation with respect to extensibility:where {e i } i=1,...,Ne is the set of edges of the deformed mesh and {e 0 i } i=1,...,Ne is the set of edges of the template mesh.Percentage of deformation with respect to curvature change: Fig. 9. Pig datasets: 3D templates of three different organ's tissues: The bladder and the pericardium.For each template we indicate in mm the size of the box bounding the 3D shape.

TABLE I DETAILED
QUANTITATIVE RESULTS FOR DIFFERENT IN-VIVO TISSUES.THE ERRORS ARE IN MILLIMITERS.