Improvement of Body Movements and Stability of Blind or Visually Impaired Adults by Physical Activity using Kinect V2

People who are blind or low vision need to follow activities routines for their mental and physical health to minimize the risk of suffering from bleeding in articulation but they have problems due to difficulties and inaccessibility of displacement. This paper introduces and evaluate a set of exercises to improve the bodily movement and stability using body tracking by Microsoft Kinect V2 and audio feedback. These exercises are composed of a sequence of different postures, has an audio feedback personalized to help people to understand each gestures and can correct them if it is not correct, and generates a summary graph to evaluate the success rate of exercises. To obtain the 3D joint coordinates from the depth sensor, we used the SDK V2.0 of the Microsoft Kinect. We use these coordinates to calculate the distances and angles between joints of interest firstly to position the user in the area field of the Kinect sensor, evaluate the different postures of movements of knees, elbows and shoulders, and detect the body balance if he is leaning and in which direction to avoid falling. These physical exercises have been evaluated to improve feasibility and feedback with persons who are blind or low vision. Keywords—Posture; visual impaired; physical exercise; audio feedback; Kinect; body tracking; balance; falling


I. INTRODUCTION
In our daily lives, physical activity is very important; it helps us to protect our health from various medical problems. Thus, many studies indicate the importance of physical training to improve the health of people and maintain them in a better physical condition.
According to the World Health Organization (WHO), we have at least 2.2 billion people have a vision impairment or blindness [1]. These people have an inability to practice physical activities and had a problem with balance that puts them in a great possibility of falling. In addition, they cannot move without any help especially during this epidemic of the COVID-19 virus, which requires putting a distance of at least one meter between two people.
The practice of sports activities for blind person or those with reduced vision is difficult without being helped by another person because of the risk of falling and being injured due to the lack of practice of physical exercises.
The daily practice of physical activities for the blind or visually impaired person helps them to recycle posture, maintain flexibility in the body articulations and reduce the falling risk by reinforcing the body balance. In order to give people a certain autonomy by offering them a better quality of life, certain studies have been developed to study and use several tools and techniques.
Postural control is the maintenance of balance of the body either by keeping the body's center of mass or by returning it to its own base of support, which is defined as the maintenance, achievement or restoration of state of equilibrium. Some studies have shown increased risk of falls as well as impaired balance due to decreased ability to control posture [2,3]. Thus, given the importance of fall avoidance, a method to assess postural stability that can predict the risk of falling has been developed [4,5].
Indeed, falls present a major health problem. According to data provided by the World Health Organization (WHO) in January 2018, falls are the second-largest accidental mortality in the world causing the annual death of an estimated 646,000 people. In addition, the number of people who require medical attention because of falls has been estimated globally to be around 37,300,000 per year [6]. According to the WHO, the elderly (> 65 years) are the people most affected by fatal falls. It is also reported that approximately 28-35% of this age group (32-42% for> 70 years) fall annually [7]. Therefore, in order to assess the risk of falling, it is necessary to develop measures that characterize and determine the effects of aging on postures.
The Kinect sensor, thanks to these advantages: simplicity, affordability, reliability as well as validity, has been extensively tested and verified in numerous experimental studies devoted to measurements of the human joint center [8][9][10][11]. Several studies have focused on its measurement errors [12]. These results support the use of the Kinect sensor to assess walking and balance performance [13][14][15][16][17]. It is already mentioned the availability of some similar RGB depth sensors (RGB-D). Readers interested in it are referred to these alternatives to the Kinect [18].
Several recent studies regarding depth camera applications for analyzing and capturing human movements have been investigated. Among these, the Microsoft Kinect V2 sensor offers both high-resolution color images, as well as wide field of view (FOV) areas and powerful software development kits to detect the skeleton and joints. In addition to the above, the reliability and validity of the Kinect sensor has been studied and verified as much, which is why we have opted for the Kinect V2 as a measuring device for monitoring the skeleton and joints.
Several study are done on the practice of physical activities for different categories people. For example, [19] used the airflow to reinforce the motivation and the bodily movements for children who are visually impaired to increase positive interactions. In [20] the Microsoft Kinect V1 is used in a yoga exergaming dedicate for blind and low vision people. However, most of them have not been interested for blind or visually impaired person, for example in [21] they develop a simple algorithm for the automatic classification of human posture detection for body rehabilitation, and in [22] they developed a sports exercise for an athlete using the Kinect V2 for the distance and angles calculations. Another study presented in [23] presents an exergaming called HemoKinect to supervise patients with hemophilia during physical exercise. In [24] they used a rehabilitation exercise-tracking algorithm, based on a pose classification scheme combined with a trajectory recognition approach.
In this paper, in the next section (Section II) we will detail the materials and the methodology used. Then in Section III, the results will be presented with summary graphs of the joints angle trajectory and followed by a discussion in Section IV. We will finish with a conclusion in the last section (Section V).

II. MATERIALS AND METHODS
Kinect v2 sensor is a device used for human machine interaction. As shown in table I, the Kinect support three types of original data stream can be extracted via the Kinect sensors, it is indeed the depth data stream which is extracted using a depth sensor formed by an infrared CMOS camera and several infrared emitters. Also, the color video feed provided by a color camera. As well as the audio data stream delivered by the microphone.
The distance between the object and the camera in the visible range is reflected by the depth image formed by the Kinect depth data stream. In order to locate and track the joints of the human skeleton, the depth image captured by Kinect is used. The Kinect has the ability to recognize skeletal data from the entire body of six people in measurement space, as well as generate around 30 frames of skeletal data per second and provide three-dimensional coordinate information for 25 joints of the human skeleton (Fig. 1).
Indeed, an algorithm, which belongs to the background race, makes it possible to calculate the 3D depth and to carry out the follow-up of the skeleton of the users. These data are used to express the 3D positions of the joints using vectors.
However, the internal structure of the human body has not been taken into consideration. Indeed, taking into account the structure of the human body, there was an extraction of the distance characteristics of the three-dimensional coordinate data of the skeletal joints were used for the purpose of representing posture.
The Kinect sensor has a detection area ranging from 0.5m up to 4.5m. Nevertheless, for good detection and tracking of the skeleton it is preferable that the user is located in a distance of 1 to 4 m that covers the right area (sweet spot) with an angle width of 70 ° degrees (Table I). For this why the sensor is placed it at a height of 1 m (y-axis) and a Z distance of 2.5 m as shown in Fig. 2. In order to guarantee obtaining 3D articulation positions, we benefit in our research from the precision of the Microsoft Kinect SDK v2.0 libraries in order to calculate the distances and angles between joints to set the conditions for the evaluation of the exercises and body balance. Using the random decision forest algorithms [25], skeletal data was provided from the depth images.
Where Joint first = (X first ,Y first ,Z first ) and Joint second = (X second ,Y second ,Z second ) The flowchart of the system is represented in Fig.3 as we said we are interested of the depth flow to obtain the skeleton data. After the acquisition of the 3D coordinates of the different joints of the body (x, y and z), which are used to calculate the distance and the angle between joints. The distance between the sensor and the user is calculated along the longitudinal (Z-axis) and the transverse (X-axis) direction to position the user in the good detection fields of the sensor to guarantee a wide field for the acquisition of skeletal data. After the validation of the user position, the distance and the angle associated with interest joints were calculated as indicated by equation (1) and (2), to detect the posture of the user and evaluate the gesture used to improve the physical body movement. The angle was calculated by defining a couple of vectors (�⃗, ⃗) ∈ 3 formed by the adjacent body sections of the joint and taking the angle in degrees between them. Then, a skeleton is displayed in real time on the computer screen and an audio feedback from speaker to notify the user by a voice message. Finally, summary graphs of the gestures are generated.
The exercises used in this work are composed of a sequence of different postures created to enhance the body movement (Fig. 4). The poses used for the training algorithm to enhance movement are described like this; the first pose is the neutral pose, which define the standing posture. The second, arms opened and straight with a condition at the angle of the elbow that should be almost equal to 180 ° and the wrist at the shoulder level. The third pose, right and left arm is opened and up. The forth, both hands are forward and straight. The fifth pose, the user opened her arms and make an elbow angle of 90°. The sixth pose the user joined her hands forward. The seventh pose the user make two hands on waist. The eighth, left knee up and the last pose is the right knee up with a knee angle lower than 165° for the two latest poses.
Movement exercises are described as a sequence of postures. The simplest movement is described by the start and end position (as shown in Table II). Each user was asked to perform seven exercises and repeat each of them three times.
These exercises are used for the flexibility of different joints in the body and improved the physical movement and the body balance because the visually impaired person suffer from imbalance, for this , we took into account and use a voice notification to alert the person when he is leaning and in which direction to correct his posture and avoid falling. The balance is evaluated in four directions along the X and Z plane, which are right, left, forward and backward (Fig. 5).

III. RESULTS
A sample graphs for each exercise are represented in Fig. 6. For each exercise, an ensemble of rules is used to evaluate the body movement and detect the posture of the person if he is leaning or not. These conditions are imposed with taking into account the limitations of the mobility of blind or partially sighted people.
To facilitate the set of postures sequences exercises, every participant is asked to repeat each exercise three times. These exercises deal with the flexibility of the knees, elbows and shoulders. Considering the difficulty of some gestures for some people, a slight difference is perceived in the results but this does not prevent the success of this set of different gestures. Table III, the average of detecting the exercise achievement is more than 97%, which indicated that the proposed algorithm is feasible, through the use of equations (3),(4) and (5).

As shown in
These exercises are based on instruction to verify the state of gestures (correct or not), and correct them if they are false. To inform the user, a verbal message is send through a speaker. The instructions and correction differ between postures. For difficult gestures are difficult they need more instructions than another one.

IV. DISCUSSION
The physical exercises chosen in this work are applied for the flexibility of the joints in order to improve the physical body movement to keep the body in balance and avoid the risk of falling. These movements are based on flexions and extensions gestures of the knees, elbows and shoulder with conditions on the angles and distances so that they can be easily detected in order to make the requested exercise.
For the movements of different joints which interest as in this work the success rate is 100% for most of the people concerned. The majority of the exercises were performed successfully in all tests performed. Most of the gestures are difficult to execute at the beginning of the training then they are successfully executed.
The aims in this study is to reinforce the physical movement and maintain the body in balance throw a set of different sequences of postures with an audio feedback for instructions and correction of movements facing the Kinect.
To ensure that the messages are clear and understandable enough, we have tried to detail the various instructions as much as possible. For example, for the forth exercise, the instruction is as follows: "raise your arms, open at the level of your shoulder then make an elbow angle of 90°".
Blind or visually impaired person are very careful when they are doing any movement or gestures in the daily life to avoid the risk of falling as they are suffering from imbalance. For this, they fail in some postures because they feel fear of making physical movement, which they not habituated, and the lack of flexibility in body joints.
Different commercial Motion Capture sensors exist such as RealSense R200, Orbbec Astra, ZED stereo camera, Leap Motion and Kinect sensor.
In fact, compared to alternative Motion Capture sensors, Kinect V2 has many advantages. In addition, it can be used to carry out comprehensive monitoring of the skeleton and many bodies collectively. At the same time, this allows the acquisition of data from more than one person. It also benefits from its low price as well as its support for certain varieties of toolkits and software languages. In addition, it has mature drivers, and is privileged by a well accessible and documented SDK.
The Kinect V2 is opted for a sophisticated depth sensor characterized by a higher resolution, as well as by the possibility of carrying out the following of more bodies and articulations per body. Thus, the Kinect V2 becomes a satisfactory alternative for clinical applications, as has been presented by some authors.
In our tests, Kinect V2 is able to measure the angles between the different joints of the skeleton when they are not hidden by any object or obstacle.
To guarantee a good detection and extraction of skeletal data to calculate the angles and distance of the different joints of interest, the user must not go beyond the recommended range with a uniform lighting and without any obstacle or object between the body and the Kinect. In this study, the user can position herself facing the Kinect sensor through a vocal notification.

V. CONCLUSION
To maintain our health in good condition, it is crucial to practice physical exercises. This is why in order to give more importance and dedication to the visually impaired by helping them to train at home with a Kinect sensor via audio feedback messages for the instructions.
For helping them to strengthen their balance, since the blind or visually impaired suffer greatly from balance problems. This is also crucial during this epidemic of the Covid-19 virus, since it is obligatory to keep a distance between people.
The user must position herself to do exercises in the correct fields and direction in front of the sensor used to detect the person. The Microsoft SDK of Kinect V2 is used to obtain the 3D coordinates of the joints data to position person, detect the posture to check if the person is leaning or not to avoid falling risk and evaluate the physical movement through the calculation of distance and angle between joints used for the exercises instructions.
The instruction of the exercises used in this work allow to detect successfully the flexions and extensions of the knees, 462 | P a g e www.ijacsa.thesai.org