Tai Chi Care: An Exergaming Software using Microsoft Kinect V2 for Blind or Low Vision Person during Confinement

Blind or low vision people need to practice activities for their mental and physical health to minimize the risk of suffering from articulation pain but they have problems due to difficulties and inaccessibility of displacement especially during the COVID-19 pandemic where everyone in this world was asked to stay at home during confinement. To solve these problems, we have developed a software tool for a care Tai Chi exergaming to encourage them to practice exercise at home using body tracking by Microsoft Kinect V2 and audio feedback. This software acts as a Tai Chi treatment, teaches four poses, and has a customized audio feedback to help person to understand each pose and generates progress graphs to evaluate the success of these exercises. We used the SDK libraries of the Kinect to obtain 3D joint position from sensors of the Kinect to calculate the angles and distances between joints to help the person to position in front of the Kinect, evaluate the different gestures of flexions and extensions of knees and elbows of each exercises, and body balance direction to avoid falling risk. These exercises have been evaluated with persons who are blind or low vision to improve feasibility and feedback. Keywords—Tai Chi; COVID-19; visual impaired; physical exercise; exergaming; audio feedback; Kinect; body tracking


I. INTRODUCTION
Tai Chi Chuan is a Chinese martial art that focuses on incorporating body movements, mind adjustments and breathing in one practice. Indeed, it is theoretically the manipulation of the good flow of the body in order to control bodily functions.
Several studies have shown that the practice of Tai Chi in elderly and middle-aged persons allows them to benefit from ideal bodily functioning and as well as prevention against various diseases. This practice leads to the improvement of cardiac and pulmonary functions [1]. Physical activity, of course, does not protect against the risk of contracting COVID-19 in case of exposure to the coronavirus. However, good physical condition contributes to the proper functioning of the immune system so that it can fight the virus. Tai Chi also helps support muscles and tendons [2] and reduce the deficit in bone density [3].
As well, Wu and his colleagues [4] focused on the patterns of muscular action during exercise. They analyzed the approach of Tai Chi and its consequences on balance, flexibility as well as strength. As in [5], they highlighted a virtual environment, which imitates the execution of the exercise by a Tai Chi master, which can provide the teacher with new ways to improve his skills.
Different studies are done on the practice of Tai Chi on different categories of people but they have not been interested in visually impaired or blind people who also need it to do physical exercises and especially when it is mainly dedicated to therapeutic care. For example, [6] a study examines the effect of practicing Tai Chi during one year on the physical functioning of the elderly, which is effective in maintaining and improving the state of physical condition of old people. However, it should be done at least three times a week to guarantee this effect. In [7] the Microsoft Kinect is used to record the spatial coordinates of the joints of the upper limbs of the body during the practice of Tai Chi Chuan as well as the data in order to perform a quantitative and qualitative analysis of the joint positions and the angle of the elbow joint. A study was conducted by [8] with participants from a trial on the effects of a 12-week Tai Chi training on the risk of ischemic stroke in the elderly, and this study showed that regular exercise Tai Chi can have positive benefits in terms of improving physical health and mental state of elderly.
Another study presented in [9] provides evidence that choosing Tai Chi Chuan as a form of exercise is beneficial for children with asthma, indicating that 12 weeks of exercise improve the quality of life for these children, improves pulmonary function and decreases airways inflammation. In addition, in [10], the authors are presented a qualitative research on the physical and psychosocial effects perceived on the practice of Tai Chi Chuan on an educational group in a clinical trial of patients suffering from chronic heart failure.
Physical activity is very important in our daily lives. In fact, it helps us to protect our health from various diseases and syndromes. Thus, many studies particularly address the importance of physical training to improve the health of people and maintain them in a better physical condition. Especially during the last COVID-19 epidemic where almost the entire world population was forced to respect the confinement and to stay at home, which is not arranged health either physically or psychological. This situation has generated in many people a depressed mood and generalized anxiety. Indeed, the www.ijacsa.thesai.org prohibition and the deprivation of liberty directly confront people with feelings of weakness. Besides, being forced to stay at home is not natural for anyone and everyone has a basic need to feel free to do what they want.
For the blind or those with reduced vision, it is difficult to practice sports activities without being helped by another person because of the risk of falling and being injured and to correct they positions for those had difficulties. These people also suffer from being deprived of video games. These two shortcomings cause for these people: problems of physical balance (lack of sports activity) and psychological balance (Boredom and routine).
The repeated practice is supposed to recycle posture, encourage circulation throughout the body of people, maintain flexibility through their joints, and reduce the incidence of depression and discomfort for the visually impaired and improve their quality of life.
In particular, the use of video games for re-education (exergaming) is playing an increasingly important role towards its positive impact on patient attitudes and has been necessary to improve and maintain both their strength and especially their mobility [11]. This is an ideal solution during confinement due to the COVID-19 virus which has put us in stressful and restricted movement conditions. Indeed, sitting and walking too little increase the feeling of heaviness in the legs and pain even for people who do not have circulatory problems. To improve the blood flow in the legs, it is recommended to do certain sports exercises.
Among the many Motion Capture (MoCap) sensors, Kinect is the most prominent tool for medical applications and especially those related to physiotherapy [12,13] that can be used for preventive purposes when detecting falls of the aged people [14], or rehabilitation exercises dedicated to people with temporary disability following a cardiovascular attack, or with limited mobility, etc.
In this paper, we will detail the tools and the system used in Section II. In Section III, we will present the results of the Tai Chi care exercises with a graph, which details the variation of the angles and the success rate. This part will be followed by a discussion in Section IV. We will end with a conclusion in Section V.

II. MATERIALS AND METHODS
In order to interact and immerse in video games, Microsoft invented the Kinect in several versions. The second version ( Fig. 1(b)) of this device allows connection and interaction by detecting different gestures and movements of the human body for Microsoft X-Box ™ platforms (Microsoft Corp., Redmond, WA, USA). In order to allow passionate programmers to create their applications, Microsoft has developed SDK software [15].
As we said before, the Kinect exists in several versions. Indeed, different publications have concentrated on the study of the different uses of the first version ( Fig. 1(a)) of Kinect (V1) [16,17]. Some of them had to face the limitation of precision of this first version [18] in particular with regard to the measurement of the angle of articulation, which has been improved in recent studies giving satisfactory results [19]. It also presents some detection and tracking problems in certain lighting situations, thus rendering it totally devastated by the second version of Kinect (V2) [20]. Therefore, in terms of accuracy, it has been validated that the Kinect V2 is much more precise than the previous one, thus allowing better results during new studies and giving the possibility of adaptation for new exercises intended for several disabilities or medical problems such as patients with balance disorders [21]. Table I presents a comparison between the different characteristics of the two versions of the Kinect sensor; it shows that the second version is more precise and that it has a wider field of vision than the first.
However, in our research, skeletal tracking is the application offered by the Kinect that interests us the most. Indeed, it is feasible from depth maps to obtain skeletal data of up to six users at a time, and this thanks to the automatic learning algorithms [22]. These data we just mentioned are composed of 25 points joined by different segments. This version also allows us to deduce the position of an invisible joint that can be hidden during detection by an object or a piece of furniture.
The exercises used in our work were performed by blind or visually impaired people in a standing position in front of the Kinect sensor. Users are invited to position themselves in the Kinect detection field in order to avoid the risk of loss of skeletal data of the joints of interest.   Fig. 3 represents the flowchart of the system, which is mainly based on skeletal data extracted from depth images by applying the random decision forest algorithm to detect joints [22]. Thus, after the acquisition of the x, y and z coordinates of the various joints of the skeleton, a calculation of the distances along the Z and X axes between the joints of the ankle and of the sensor (Fig. 2), is carried out to position the user in front of the Kinect and in its detection fields for a good detection of the skeleton and to widen a maximum of fields for the movement of the user without loss of skeletal data.
Also a calculation of distance between two points as indicated by equation (1) for the body balance calculation which is defined by calculating the distance between two joints of the torso, to warn the user if he is leaning or not and in which direction to return to his steady state in order to avoid the risk of falling. This step is evaluated in 4 directions along the X and Z axes, which are North (front), East (right), South (back) and West (left).
The angle associated with each articulation of interest was calculated by defining a couple of vectors (⃗ , ) ∈ 3 formed by the adjacent body sections of the joint and taking the angle in degrees between them, as presented by equation (2). Then, a skeleton overlay on the color image is displayed in real time on the computer screen with audio feedback from the speakers to instruct the user using voice notification.
The four exercises used in this work are represented in the Fig. 4, which are composed of a set of gestures. There are organized like this, the first exercise ( Fig. 4(a)) the "Handing Ball pose" which consists of a movement motion of the arms which are in front of the body forming a circle as if there was a ball between them and which does not exceed the breast levels; this means that the conditions apply on the position of all the joints of the two arms (Shoulder, Elbow, Wrist, Hand and Hand-Tip), as well as a gesture to the legs which require a slight bending of the knees to have an angle for the two knees joints which is less than 180° and higher than 100°.
The second exercise (Fig. 4(b)) the "Qi pose" which is composed of a knee flexion gesture like that of the first exercise and a gesture for the hands which make a form of "qi", it means that both hands should be in front of the torso and bended, one between the "spine Shoulder" joint and the "spine Mid" joint and the other between "spine Mid" and "spine base" with the palms facing each other as if there is a small object between them.
The third one (Fig. 4(c)) is the "Ma Bu pose", it requires two gestures: one for the knees which must be bent by an angle of 90° and the other for the hands which are the level of the pelvis, this means the joint of the "hip".
In addition, the last exercise ( Fig. 4(d)) is the "Gong Wu pose" which is made with a hand gesture like the previous one and a leg gesture so that one is in front and forms an angle at the knees of 90° and the other at the back and straight with an angle of 180°.
These exercises are used for the flexibility and the rehabilitation of different joints of body. They help to make the joints more flexible, especially the knee joints and to balance the body. The two first poses are specially intended for the arms joints, and the other are for the legs joints. Fig. 5 shown an example of screenshot for each exercise.  (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 7, 2020 150 | P a g e www.ijacsa.thesai.org  III. RESULTS Fig. 6 shows sample graphs for each exercise. The knee flexion for the "Handing Ball" and "Qi pose" exercises is defined below 175°, for that of the "Ma Bu", it is limited between 70° and 120° because the position is difficult since it is hard to bend the knees by 90°, with regard to the "Gong Wu" the bending of the leg in front must be between 85° and 95° as for that in the back, it must be greater than 165°. These conditions are imposed after advice provided by a Tai Chi referee while taking into account the limitations of the mobility of blind or partially sighted people.
To facilitate the gestures of these four caring exercises, everyone is asked to repeat each exercise three times for a period of five seconds each time. These exercises deal with the flexibility of the knees and also the elbows. This set of different gestures is performed successfully but with a slight difference, in fact this is due to the difficulty of certain gestures for some people.
Regarding the success rate of elbows for witnesses, it is 100%. For the knees, this rate is divided into two parts, one for the "Handing Ball" and "Qi position" exercises where it is 100% and for the "Gong Wu" and "Ma Bu" exercises where it is only 80 %. This is at the beginning of the practice of these exercises, but after a few days of application, they can perform them successfully.
Verbal correction or instructions differ between gestures. Some of these have more correction than another depending on the difficulty of the exercises. For example, as already mentioned for knee flexion gestures in the poses of "Ma Bu" and "Gong Wu", also for the gesture of the positions of the hands for the poses of "Qi" and "Handing Ball".   Test 3 Test 2 Test 1 www.ijacsa.thesai.org As mentioned in the previous section, these exercises are healing exercises to maintain the flexibility of the joints due to lack of mobility and especially the knees. This is why these gestures are based on the angle of the knee. Also, do not forget that people who are blind or visually impaired suffer from imbalance that is why we took into account the balance of the body to avoid any risk of falling for this way we use a verbal feedback to alert them if the body is leaning and in which direction.

IV. DISCUSSION
The Tai Chi care exercises chosen in our work are applied for the flexibility of the joints and in order to keep the body in balance. These exercises are based on flexions and extensions gestures of the knees and elbows with conditions on the angles so that they can be easily detected in order to complete the requested exercise.
For the movements of the elbows in the four exercises and the knee flexion movements for the "Handing Ball" and "Qi pose", the success rate is 100% for most of the people involved in our study. Most of exercises were successfully performed in the all the performed tests. The "Gong Wu" and the "Ma Bu" are more difficult than the two other gestures at the knee bending level, that is why we have a lower success rate than the other results at the beginning of the practice, but after that, all the gestures are executed correctly.
Our work is also based on audio feedback for verbal correction of gestures and giving instructions to people facing the Kinect sensor. We tried to detail as much as possible the different instructions so that the voice messages are clear for all people. For example for the position of the arms for the exercise of "Handing Ball", the instruction is as follows: "raise the arms in front, just below the level of the elbows, and turn the palms facing you as you like you have a ball between your arms". In some cases, the failure of blind or partially sighted people is due to the lack of flexibility in the different joints of the body, as well as the fear of making unusual movements because they are very careful in their movements since they suffer from the problem of balance to avoid the risk of falling.
There is a wide range of commercial MoCap sensors that are used, for example: Orbbec Astra, RealSense R200, ZED stereo camera, Leap Motion and Kinect sensor.
Kinect V2 has several advantages compared to alternative MoCap sensors: The latter can be used to perform complete monitoring of the skeleton as well as of several bodies simultaneously, this can allow the parallel acquisition of data from more than one person; its price is relatively low. It supports several variety of software toolboxes and languages; and it also has mature drivers, and an SDK well documented and open accessed.
The Kinect V2 has an advanced depth sensor with higher resolution, as well as the ability to track more bodies and joints per body. As a result, Kinect V2 becomes a valid alternative for clinical applications, as presented by some authors. In our tests, Kinect V2 can measure the angles between the different joints of the skeleton when they are not hidden by any object or obstacle. Therefore, the limitation since the first Kinect V1 continues to be a source of error in V2. This means that, the user must be installed facing the Kinect without any obstacle between the sensor and the body, does not exceed the recommended range (0.5 m to 4.5 m) and in uniform lighting.
In our work, the user is asked through voice messages to position themselves in the correct detection position in front of the Kinect sensor. This step helps us to correctly detect the skeleton without loss of data and calculate the angles of the joints. However, in some cases, the poor detection of the angles of the knees for the exercises "Ma Bu" and "Gong Wu" is due to the difficulty of some people to bend the knees. This is why the performance does not exceed 80% success for the tests at the beginning.

V. CONCLUSION
The role of physical exercise is to keep our health in good condition. That is why we have given importance to people who are visually impaired to help them work out at home with a Kinect sensor, who will receive an audio feedback message for the instructions of four poses of Tai Chi treatment to keep the joints flexible and to help them to strengthen their balance, as the blind or partially sighted people suffer also from the problem of balance. This is also of great importance during this epidemic of the covid-19 virus, since most countries have imposed the confinement and people were forced to stay at home.
However, to do an activity in front of this sensor, you must position yourself correctly so that the latter can detect the person. We used the Microsoft SDK V2.0 from Kinect to obtain the 3D coordinates of the joints that are used to calculate the distance to position persons in the right detection area, the balance of the user to avoid falling risks, and joints angle who are used for the Tai Chi treatment instruction.
The instruction of Tai Chi treatment used in this work is able to detect successfully the flexions and extensions of the knees and elbows joints for the different gestures and measure body leaning to keep it in balance.
Although this work is not intended only for blind or partially sighted people, it is accessible for any person who want to practice this treatment without looking to the screen just by receiving a useful and customized audio feedback.