Human Lips-contour Recognition and Tracing

—Human-lip detection is an important criterion for many automated modern system in present day. Like computerized speech reading, face recognition etc. system can work more precisely if human-lip can detect accurately. There are many processes for detecting human-lip. In this paper an approach is developed so that the region of a human-lip can be detected, we called it lip contour. For this a region-based Active Contour Model (ACM) is introduced with watershed segmentation. In this model we used global energy terms instead of local energy terms because, global energy gives better convergence rate for malicious environment. At the time of ACM initialization by using H∞ based on Lyapunov stability theory, the system gives more accurate and stable result.


INTRODUCTION
Aut;omatic lips contour detection and tracking is always a crucial prerequisite process for various kinds of applications.It has been extensively utilized in the state-of-the art of audiovisual speech recognition.[1] Fig. 1

. Different shape of lips contour
There are several uses of lips contour detection like Audiovisual speech authentication [3], intelligent human-computer interaction Human expression recognition [2] etc. Automatic Speech Recognition (ASR) [4] systems use only Acoustic Information, for this system show poor performance in noisy environment.Bimodal Audio-Visual Systems signal often contains information that is complementary to audio information.Again visual information is not affected by acoustic noise.For this overall performance of the combined system is better [5]- [7].
Moreover, when working with the video sequence, an automatic and precise initial contour placement for the ACM is a crucial issue.Therefore, lips tracking algorithm is adopted into the system to keep track on the lips feature points at the subsequent incoming video frame as the initialization of the ACM lips contour process.In this paper, the conventional H∞ filtering is modified according to the LST as to assure that the system is always at the stable condition.Additionally, by properly selecting the Lyapunov function V (k), during the system design, as the time approaches to infinity, the tracking error of the proposed H ∞ filtering would asymptotically converge to zero.This is because the Lyapunov function [11] selected for the proposed system, This consists of a unique global minimum point at the system origin.From the simulation results demonstrated, the modified H∞ shows an appreciable improvement in terms of reducing the lips feature points tracking error compared to the conventional H∞ approach In this paper some of the terminology will be discussed like

A. Contour:
A contour line of a function of two variables is a curve along which the function has a constant value.Minimal Cost www.ijarai.thesai.orgPath: The optimal path from every pixel in the image to the seed point is determined by using Dijkstra's algorithm.[9] B. Active contour model: Active contour model, also called snakes, is a framework for delineating an object outline from a possibly noisy 2D image.This framework attempts to minimize an energy associated to the current contour as a sum of an internal and external energy.[12] Fig. 3. Active contour model.

C. H∞ filter:
After a decade or so of reappraising the nature and role of Kalman filters, engineers realized they needed a new filter that could handle system modelling errors and noise uncertainty.State estimators that can tolerate such uncertainty are called robust.The H∞ filter [8] was designed for robustness.

D. Lyapunov stability theory:
The most important type is that concerning the stability of solutions near to a point of equilibrium.If all solutions of the dynamical system that start out near an equilibrium point e stay near e forever, then e is Lyapunov stable.[12]

II. LIPS CONTOUR DETECTION & TRACKING
In this paper, instead of utilizing the global region model, the active contour energy with the local information interpretation corresponds to the watershed segmentation is proposed.This gives better performance on the contour detection compared to the global energies calculation, particularly when the inner and outer region shares the similar image statistic.Performs is better compared to the localized ACM algorithm.
Compared to the aforementioned approach, the integration of ACM and watershed segmentation presented in this paper not only provides high deformability but is also better in handling the unclear boundary between lips and facial skin.
For video sequence, an automatic and precise initial contour placement for the ACM is a crucial issue.Keep track on the lips feature points at the subsequent incoming video frame as the initialization of the ACM [6] lips contour process.H∞ filtering is modified according to the LST as to assure that the system is always at the stable condition.By properly selecting the Lyapunov function, V (k), as the time approaches to infinity, the tracking error of the proposed H∞ filtering would asymptotically converge to zer This is because the Lyapunov function selected for the proposed system, The overall process of the proposed lips contour detection and tracking is demonstrated in Figure 5.The input image is first sent to the lips localization to roughly obtain the lips location.Subsequently, the watershed region segmentation and at the same time, also the ACM initialization are working on the lips image.The output from the watershed and the ACM initial contour position are then passed to the localized regionbased ACM for further lips contour detection.Once the lips contour is successfully detected, the feature points from the detected contour are extracted and used by the modified.H∞ lips tracking process to keep track on the initial contour position of the succeeding image.The lips localization is triggered only once per 20 rounds of the overall lips detection process.www.ijarai.thesai.org

III. LIPS CONTOUR DETECTION WITH LOCALIZED WATERSHED-BASED ACTIVE CONTOUR MODEL
Each pixel is situated at a certain altitude levels.Where the white pixel (intensity value = 255) is referred to as the maximum altitude while black pixel (intensity value = 0) is known as the minimum altitude.
The watershed algorithm which applied in this paper is based on the rain-flow simulation.By employing the falling rain concept, the raindrop falls from the highest altitude to the lowest (known as catchment basin) according to the steepest descent order.
The coloured watershed regions are known as the local mask region M, for a particular point within the narrow band.The local exterior region is illustrated in orange colour while the red region is known as the local interior region.

The local mask region can be mathematically construed:
Where w is known as the segmented watershed region situated at the local point to be analyzed, and including the segmented watershed regions located at the points of north, east, south and west from the local point.The energy function, F of an image, i(x, y) could be mathematically written as follows: Where α ≥ 0, λ > 0, u and v are the average intensities levels inside and outside of the curve C, respectively.By applying the level set approach onto the variation of the ACM, the unknown C is substituted with the unknown value φ, φ(x, y) is taken as the signed distance function, where φ(x, y) > 0 inside the curve, φ(x, y) < 0 outside the curve, and φ(x, y) = 0 on the curve.

IV. MODIFIED H∞ BASED ON LYAPUNOV STABILITY THEORY FOR LIPS TRACKING SYSTEM
An active shape model sample grey level perpendicular to the lip contour and centered at the model points.
After obtaining the lips contour from the proposed localized ACM system as discussed in the previous section, six feature www.ijarai.thesai.orgpoints from the detected lips region is tracked using the modified H∞ filtering.[9] Fig. 8. Profile points & grey-level.
After obtaining the lips contour from the proposed localized ACM system as discussed in the previous section, six feature points from the detected lips region are tracked using the modified H∞ filtering which is elaborated in this section.The feature points include the right and left lip corners, the lower central point, and three points of the Cupidon's bow as depicted in Fig. 1.The six feature points (P 1 -P 6 ) have to be first extracted from the detected contour.P 1 and P 5 are, respectively, obtained from the right and left most of the contour.Whereas P 6 situated at the bottom of the lips contour while P 3 at the upper contour that has the similar vertical coordination as P 6 .P 2 , which is the left Cupidon's bow is obtained by calculating the maximum altitude between P 1 and P 3 , while P 4 (the right Cupidon's bow) is the maximum altitude between P 5 and P 3 .H∞ filtering is mathematically represented according to the state-space concept.State equation:

Where
, A is known as the P ×P dimensional state transition matrix, which bring forward the state value, Xk from time k to k + 1, wk is the P-dimensional model error that analogous covariance matrix is known as .
Observation equation: y k = CX k−1 + v k Where y k is known as the N-dimensional sequential observation vector, C is the N ×P dimensional observation function, v k is the observation error that analogous covariance matrix is known as .In this paper a new gain for the H∞ filtering parameter updating rules is proposed.
The modified H∞ is interpreted as follows.
Theorem: With the given linear parameter vector, H k and desired output, d k , the state vector, x k is updated as follows: The modified H∞ adaptation gain that fulfills the condition of Δv < 0 is proposed as follows: Where γ is known as the user-defined performance bound, Q, W and V are the respective weighting matrices for the estimation error, process noise, and measurement noise.The priori estimation error, α k , is denoted as follows: When the time, k, approaches infinity, the tracking error, e k , would be asymptotically converged to the value of zero.www.ijarai.thesai.org

V. FUTURE WORK
This method can be applied in 3-D system by some modification.Face recognition, emotion detection, speech recognition from a video etc. area can achieve a high percentage of accuracy by applying lips detection method.

VI. CONCLUSION
It is well known that visual information from lip shapes and movements helps improve the accuracy and robustness of a speech recognition system.Hence, the development of an accurate and robust algorithm for extracting lip features relevant to speech information becomes vital.Watershed-based active contour model and modified H∞ filtering is illustrated in this paper.Compared to the global region-based ACM, the performance of the proposed localized ACM is successfully improved.The modified H ∞ tracking approach validates improved lips feature points tracking compared to the conventional H∞ algorithm [11].
By applying Shape Constraint factor this method could be more robust.Given an image, the region of the lip may be located by using Shape Models.The overall system would be implemented into the audio-visual speech recognition for further research in the future.This system can be used to identify face or face recognition system.
But the system has High time complexity.Tracking through large number of frames point P1-P6 sometimes difficult to identify in robust situation.Sometime the skin & lip color are too similar that hard to distinguish.As a result contour detection might be tuff.For different expression like smiley, sad etc. finding accurate contour of the lip is a challenge.

VII. ACKNOWLEDGMENT
A special thanks to Prof. Dr. Mohammad Mahfuzul Islam for giving us a chance to work with this topic.Also a superior thanks goes to Siew Wen Chin, Kah Phooi Seng, Li-Minn Ang, King Hann Lim, there works really was very helpful.

Fig. 6 .
Fig. 6. a) Original image (b) expected watershed segmentation (c) oversegmentation.So by applying Sobel filtering & watershed segmentation the system can gain expected segmentation.

Fig. 7 .
Fig. 7.The outcomes of the pre-processing procedures.

Fig. 10 .
Fig. 10.The process flow of the H ∞ filter algorithm