Proactive Intention-based Safety through Human Location Anticipation in HRI Workspace

The safety involved in Human-Robot Interaction (HRI) is an important issue. This is the key point for the increase or decrease in HRI activity. A novel solution concerning the safety of HRI is proposed. The solution considers the near future human intentions. A set of possible human intentions is known to the robot. The robot also knows the places that can be visited by the interacting human according to his current intention. The proposed solution enables the robot to avoid a potential collision by anticipating the future human location and dividing the workspace into safe and unsafe zones. The solution contributes for the improvement of HRI safety measures but further efforts are required for achieving an enhanced safety level. Keywords—Intention Recognition; Human-Robot Interaction; Human-Robot Interaction Safety; Unsafe Zone; Workspace


INTRODUCTION
The increase in the robotics research has increased the presence of robots in our daily life.The existence of robots in industry is increasing day by day [1].This increase triggers the already active research area of Human-Robot Interaction (HRI) and has set a rapid pace to achieve milestones in order to meet expectations.In the field of HRI, there are different sub-research areas in focus.One of the important sub-research areas of HRI is the HRI safety.Simple and light weight robots may not be harmful to the human during HRI but the large and powerful robotic arms moving at high speeds can be harmful to the human working in the shared vicinity.In the industry, the robotic arms are mostly fenced where the humans are not allowed to enter [2].In case if the human enters the area, the robot is switched off or the speed of the robot is decreased.Therefore it is still a question that human and robot can work / coexist together without any threat to the human.
Different approaches have been proposed to solve the safety problem for HRI.Safety measurement concerning HRI can be broadly divided into two categories, i.e., collision avoidance and after a potential collision detection reducing the collision intensity.There exist different kinds of approaches for avoiding a collision during HRI, e.g., camera based solutions.Camera based approaches come under collision avoidance.HRI workspace can be monitored by the range and vision sensors.The speed of robot can be decreased due to the presence of the human or the robot's path can be differently planned.Decreasing the speed of the robot or simply stopping is the only solution but is not an efficient one.Reconsideration of path is acceptable but it is not risk free.The chance of collision exists if the monitoring is performed using the vision or range sensors.Vision based HRI safety is not reliable as there can be many issues while working with the camera, e.g., human occluded by objects / robot, camera failure, etc.That may cause the collision between the human and the robot.The camera approaches also involve image reconstruction for human localization, e.g., [3] reconstructs the unknown objects (humans, etc.) with a known environment, with the aim of surveillance / human-robot coexistence.Some of the camera based approaches constitute a safety circle / sphere around the human in the vicinity of the robot, e.g., in the approach given in [4] a safety circle is used to delimit the robot motion.The approach in [1] used the safe guard zone strategy, i.e., if a human is detected in the safeguard zone then robot's control sequence is altered to ensure HRI safety.In [6] pre-collision strategies are discussed to avoid collision between the human and the robot.The focus of the approach in [6] is control strategies based on explicitly defined measure of danger while in [7] pedestrians behaviour prediction is done to ensure safety.
Other approaches involve torque sensing in the joints of the robot to decrease the intensity of collision between the human and the robot, e.g., [8].The safety solutions for HRI also involve the approaches that use mechanical design of the robot.The approach in [8] applies the whole body robot viscoelastic covering.The approaches [9] and [10] use distributed parallel actuations as a mechanical design solution for HRI safety.Similarly approach [11] proposes mechanical safeguarding measures.The study presented in [12] estimates contact forces through depth sensor (Kinect).Two control schemes are introduced for generalization of impedance and direct force control paradigms.Another approach presented in [13] uses RGBD camera and to compensate its shadowing effect a series optical distance sensors is used.Robot Operating System (ROS) is used as a platform to run the codes www.ijacsa.thesai.orgfor behavior prediction of the co-worker(human), robot path planning and collision detection.The approaches [3,14] use inertia reduction and force controller saturation filter in order to reduce the impact of force if the collision has occurred.The approach in [15] presents design strategies of human symbiotic robot.A danger evaluation method is developed in [16] using the potential impact force.The danger index is calculated by the product of factors that affect potential force between the human and the robot.The factors involve relative distance between the human and the robot, relative velocity, inertia and stiffness of the robot.The approach described in [17] discussed the HRI safety using the human intention.The focus of the approach is the human-intention-recognition based approval of the robotic action.It means that the robot checks before performing an action that its action that it is to perform in next 2-3 seconds will be accepted / allowed by the interacting human or not.The proposed solution discusses a previously unaddressed issue concerning HRI safety.The issue corresponds to the anticipation of future human location with respect to the current human intention.The HRI safety may be improved by predicting the possible human location(s) in HRI workspace, i.e., the robot can anticipate the future human location and actions and thus the robot can plan the path avoiding any expected collision.
The rest of the paper is organized as follows: In Section II, a general introduction to mechanism used to recognize the human intentions is described [18].Section III discusses the proposed approach for a potential collision avoidance between the interacting human and the robot.Section IV describes the experiments performed using the proposed approach.The conclusion and the future work are discussed in the Section V.

II. INTENTION RECOGNITION
A Finite State Machine (FSM) represents the action sequence concerning a unique human intention [18].A probabilistic weight is associated to each FSM that represents how closely the intention represented by the FSM relates to the currently estimated human intention.The high weight of a FSM means that the FSM closely relates to the currently estimated human intention and vice versa.A FSM is shown below in Fig. 1.The probabilistic value P(a j |S i ) of each action a ji at the state S i describes how likely an action a ji is for the state S i of the FSM.The action a ki represents an action that has highest probability for the state and the state transition only occurs if it occurs as shown in Fig. 1.All the other actions a ji has the low values at the state S i therefore they lead to the same state S i and no transition occurs as shown in Fig. 1.The formal description of the FSM given in Fig. 1

is descreibed.
A FSM is a tuple of Q, ∑, q 0 , F, and  .The symbols Q and ∑ represent sets of states and actions respectively.The symbols q 0 and F represent start and final state of a FSM respectively and the symbol  represents the state transition from S i to S i or S i+1 .
The sum of probabilities of all the actions for a state adds up to 1 and for each state there exist an action with the highest transition probability for that state and leads to the next state.
The flow of the algorithm for probabilistic intention recognition using finite state machines is given in Fig. 2. It is shown in Fig. 2 that an intention is recognized if the concerned FSM reaches its final state and it has the highest weight as compared to other FSMs.

III. HUMAN LOCATION ESTIMATION
Grid based approach for safe HRI is proposed, where the HRI workspace is divided into a grid of cells (lines).The paths that may potentially be visited by the human and the robot are calculated and estimated.The path of robot is easy to calculate as the destination location for the robot is given.The destination location for the robot depends on the current task assigned to the robot.Any of the existing path planning [19] algorithms can be used to calculate the optimal less optimal alternative paths for the robot to complete the current task.The path that may be followed by the human given the human intention is estimated.For the considered problem it is assumed that the human follows the optimal path from its current location to the destination location.The destination location of the human is estimated by the human intention.The human intention is recognized using the algorithm discussed in Section II [18].It is assumed that the human follows the straight line (optimal) from his start location to his goal location.Thus straight line is used as a heuristics to calculate the anticipated human path.The straight line heuristic is used to estimate the human path if there is no obstacle between the start and destination location of the human.
In case, if one or more obstacles exist between the start and goal location of the human then Manhattan distance can be used to estimate the path.After performing the three subtasks, i.e., the robot's path calculation, the human path estimation and the discretization of HRI workspace into grid, it is checked if there exist a collision between the calculated robot and estimated human path (Algo.1,line 8).
The collision checking and the auxiliary tasks required for the collision checking are presented in Algo. 2 and Algo.3. In case if there exist no collision between the optimal robot path and the estimated human path then the robot simply executes its motion command and completes its task (Algo.

Algorithm 2. Expected collision checking using the anticipated occupancy values
If more than one alternative paths exist then the optimal alternative path is selected (Algo.1,line [10][11].Otherwise the only existing alternative path is used to execute the motion command (Algo.1,line 13).If there is no alternative path then the robot waits till the robot path has no collision with the human (Algo.1,line 15).It means that the human has performed his task by following completely / partially the estimated path.At that time the human has no such intention that there exist a collision between the robot path and the estimated human path.
The collision check is performed using the occupancy values of the grid G, given in Algo. 2. The occupancy values of the grid cells are calculated with respect to the human intention.The human path is estimated based on the current location of the human L and the current human intention I (Algo.2. line 1).The estimated path of the human is comprised of cells in a grid (Algo.2. line 5).The occupancy value of each cell C i,j is calculated with respect to the estimated path P est (Algo.2. line 6, 8).The occupancy values O i,j of the cell C i,j is used to determine potential collision.After the calculation of occupancy value O i,j , it is continuously monitored if the human follows the estimated path P est (Algo.2. line 9-16).If the human does not follow the estimated path P est but the deviation from the estimated path P est within allowed limits then the recognition of current human intention and re-estimation for the anticipated path of the human is not performed.The limit values correspond to the soft threshold values defined manually.In case if the intention of the human is changed and the estimated human path P est is totally different from the path P act that is actually followed by the human then once again the current human intention is www.ijacsa.thesai.orgrecognized and the human path is estimated using the current human intention and the current human location (Algo.2. line [10][11].The new occupancy values are calculated based on the new estimated path (Algo.2. line [12][13][14].The continuous monitoring of the human concerning the estimated path is performed till the human performs the current task according to its current intention.The calculation of cell occupancy is explained in Algo.3. The cell occupancy is calculated for each cell C i,j of the grid G (Algo. 3. line 4-8).A 2D Gaussian is placed on each cell of the grid that belongs to the estimated path P est of the human path.Thus a series of Gaussian exist in a sequence of connected cells that belong to the estimated path P est .

Algorithm 3. Cell occupancy calculation
The occupancy value at these cells is the highest as compared to the neighboring cells.For each neighboring cell C i,j it is first searched that what is the nearest cell that belongs to the estimated human path P est (Algo.3. line 6).Then using that cell (C x,y ϵ P est ) the occupancy value O i,j of cell C i,j is calculated using Gaussian probability density function (pdf) (Algo.3. line 8).The zero man Gaussian probability density function is considered.The term C i,j -C mean corresponds to the distance between the cell C i,j and the nearest cell that belongs to the estimated human path.

IV. EXPERIMENTACTION
The experiments focus on the human safety during the human movement in the HRI workspace.The experiments are conducted in a simulated environment.The human movement, robotic motion and the existence of the static objects in HRI are simulated by the grid cells with specific occupancy values.The human location, the start and destination of the robotic arm with respect to a certain task are given in the simulated experiments.Ten to twenty repetitions have been performed for the validation of each discussed scenario.The robot can calculate all the possible and feasible (taking into account the robot arm singularities) paths to reach the destination.These calculated paths may not relate to all the expected scenarios in which we want to test our approach.Therefore we consider those paths that are according to the possible expected scenarios.Thus specific paths are selected for robot arm movement.
It is assumed that there is no obstacle in the human intended path (collision free).The only collision that can occur is with the robot.For the sake of simplicity, only the robot is considered to be a dynamic entity in the HRI.The mid-point line algorithm is used to calculate the human intended path from the current human location to the predicted human destination.The destination is predicted according to the estimated human intention.For the sake of simplicity, the mid-point line algorithm is used to select the cells that will be occupied by the human during the human motion.
A grid based HRI workspace is used.A grid of 50 x 50 cells is used to simulate the HRI workspace.Robot is placed at the location starting at (22, 25) on grid and its arm is at (35, 25).Robot destination is an object placed at position starting at (8,25).Robot controller is at location starting from (0, 3).The current human location starts at (20, 0).The estimated human intention is to reach a table placed at location starting from (37, 37).
With respect to the above described grid, human and robot location, the human intended path and the robot's possible paths, there exit different HRI scenarios with respect to human-robot collision and different robot paths.The different HRI scenarios are given below 1) First optimal path 2) Colliding first optimal path 3) Colliding first and second optimal path 4) All colliding paths In the middle of the Figures 3-9 the brown color box with the bold black color arrow represents the robot (robotic arm) and the span of its arm.The head of the bold arrow represents the location of the tool tip of the robot and the brown box in the middle represents the place where the robot is situated.The small arrows indicate the possible paths of the tool tip of the robot.The white color arrow represents a secure and optimal path.The normal red arrows represent the possibly insecure paths.The paths represented by the arrows correspond to the trajectory followed by the robot from its www.ijacsa.thesai.orgcurrent location of tool tip to the destination concerning the robotic task except the purple arrows which show the estimated path that may be followed by the human according to his recognized intension.The human is currently at the location represented by the yellow box at the bottom of the figure.A Gaussian is placed on each point (grid point, i.e., box) of the estimated human path and a zone is calculated with respect to the presence probability of the human.The grid cells are considered as the mean μ that are expected to be visited by the human as she follows the estimated path.The Gaussian based probability is calculated for each cell that belongs to the estimated human path.It means that sequence of Gaussian probabilities is calculated for the estimated human path.The vertical bar on right side in the Figures 3-9 represents color based gauge for dangerous and safe zones.The pure red color represents the color of most dangerous zone and the pure green color represents the safest zone with respect to robotic arm's movement.The red or reddish regions (box / boxes) represent the high risk region with respect to the probability of the human presence and the greenish or green regions represent low risk or safe zones depending upon the probability of the human presence in the concerning region.
Any robot path that has intersection with the area other than green is considered insecure with respect to the possible human collision.The insecurity increases as the color of grid cell changes from green to red.

1) First optimal path
Figure 3 shows the first scenario.There exist five possible paths that are represented by the blue and pink arrows.The paths are selected as discussed earlier in Section IV.The two paths shown in Figure 3 (pink at the lower half) passes through danger zone, and three are in safe area (blue in upper half of Fig. 3).The robot selects the minimum distance path which is also safe as shown in Figure 4 with white color arrows.

2) Colliding first optimal path
The second scenario is represented by Figure 5.There exist four possible paths, two paths (bottom in Figure 5) are in danger zone and two (upper in Figure 5) are in safe zone.The paths (bottom in Figure 5) are in danger zone as they have got some overlapping region with the human's expected path.The robot starts with the shortest distance path between source and destination.The robot considers the minimum distance path (red arrows in Figure 6) but it is in danger zone so it is rejected.The robot considers the second optimal (shortest distance) path.The path (highlighted as white in Figure 6) is second optimal but secure thus it is selected for navigation.

3) Colliding first and second optimal path
In Figure 7 three robotic paths are shown.The two paths (at bottom in Figure 7) are in danger zone due to the intersection with the reddish region and one (top in figure) is in safe region.
The robot considers the minimum distance path that is shown in the middle of Figure 8.The path is in danger zone and thus it is rejected.The robot then considers the second minimum distance path (lowest in Figure 8).The path (lowest in Figure 8) is also rejected as it passes through the danger zone.The robot considers the third optimal path.The path (highlighted as white in Figure 8) is safe and thus selected for robotic navigation.Fig. 6.Selection of second optimal secure path for robotic navigation www.ijacsa.thesai.orgFig. 7. Collision in all the optimal paths Fig. 8. Selection of the least optimal but safe path for navigation

4) All colliding paths
The Figure 9 shows the fourth scenario.There exist two paths for robotic arm to reach the destination object, and both the paths pass through the danger zone.The robot does not have any safe path to navigate thus the robot has to wait until the human changes his intention or performs the intended task.If there are n-1 colliding paths and n th path is least optimal but secure (without collision) then the least optimal (n th path) will be elected.CONCLUSION AND FUTURE DIRECTIONS In this paper we propose a novel approach for avoiding an expected collision between the human and the robot working in the shared workspace.The focus of the approach is on the human intention, i.e. given the human intention the intended human paths are estimated.The estimated paths are considered as danger zones for the robot to enter in the region(s).The presented approach focuses on the future expected human-robot collision(s).Thus the provided solution to avoid the expected collision can improve the human-robot interaction safety.The experiment section discusses different safety scenarios that may occur in a human-robot interaction workspace.The robot trades off between safety and optimality (minimum distance) while selecting the path for its navigation.
In order to further improve the HRI with respect to safety the robot speed will be changed so that the robot does not need to wait if no safe path exists.In case, if the robot can easily pass the intersection region without colliding with the human by increasing the speed then the robot can increase its speed.For this purpose, the occupancy grid values will be changed with respect to the current human existence, i.e., if the patch of intended path is at some specific distance from the current human then that patch is safe for robot to pass through if the robot moves with a specific faster speed.

Fig. 1 .
Fig.1.A Finite State Machine corresponding to a unique human intention[18]

Fig. 4 .
Fig. 4. Simple no collision in First optimal path

. Possibly secure HRI
1, line 8,  17).If there exist a collision then it is checked that if there exist alternative collision free path(s) for the robot (Algo.1,line 9).