Eye-Base Domestic Robot Allowing Patient to be Self-Services and Communications Remotely Helping Patients to Make Order, Virtual Travel, and Communication Aid

Eye-based domestic helper is proposed for helping patient self-sufficient in hospital circumstance. This kind of system will benefit for those patient who cannot move around, it especially happen to stroke patient who in the daily they just lay on the bed. They could not move around due to the body malfunction. The only information that still could be retrieved from user is eyes. In this research, we develop a new system in the form of domestic robot helper controlled by eye which allows patient self-service and speaks remotely. First, we estimate user sight by placing camera mounted on user glasses. Once eye image is captured, the several image processing are used to estimate the sight. Eye image is cropped from the source for simplifying the area. We detect the centre of eye by seeking the location of pupil. The pupil and other eye component could be easily distinguished based on the color. Because pupil has darker color than others, we just apply adaptive threshold for its separation. By using simple model of eye, we could estimate the sight based on the input from pupil location. Next, the obtained sight value is used as input command to the domestic robot. User could control the moving of robot by eye. Also, user could send the voice through text to speech functionality. We use baby infant robot as our domestic robot. We control the robot movement by sending the command via serial communication (utilizing the USB to serial adapter). Three types of command consist of move forward, turn left, and turn right are used in the system for moving the robot. In the robot, we place another camera for capturing the scenery in the front of robot. Between robot and user, they are separated by distance. They are connected over TCP/IP network. The network allows user control the robot remotely. We set the robot as server and user's computer as client. The robot streams the scenery video and receives command sending by the client. In the other place, client (user) receives video streaming from server and control the robot movement by sending command via the network. The user could control the robot remotely even in the long distance because user could see the scenery in the front of robot. We have tested the performance of our robot controlled over TCP/IP network. An experiment measuring the robot maneuverability performance from start point avoiding and passing obstacles has been done in our laboratory. By implementing our system, patient in hospital could self-service by them self.


INTRODUCTION
Recently, the development of technology for handicap and elderly is growing rapidly. The need for handicap and elderly person in the hospital environment has been given much attention and actively developed by many researchers. According to the United State census in 2000, there are 35 million elderly (12%, with the age more than 65 years). It also reported that the aging has made the percent of people needing help with everyday activities by age become linearly increased. The functional limitations of persons 65 years that has been investigated by US census consist of walking (14.3%), getting outside (15.9%), bathing or showering (9.4%), transferring (9%), dressing (3.9%), using toilet (2.6%), eating (2.1%), preparing meal (8.6%), managing money (7.1%), using the telephone (7.1%), doing light housework (11.4%).
Many robot helpers have been investigated by researchers. The ARM9-based Car controlled remotely has been developed by Wang Shaokun et all [1]. The embedded Linux system was installed under ARM9-structure processor for real time robot operation. It also optimized and improved the versatility and rapid data transmission of wireless remote car. The robots collected the data sensor and relay it to main PC station over WIFI network. Another robot also has been developed by Ding Chengjun et all [2].
Based on embedded WinCE5.0 operating system, they created remote control for mobile robot. The low power consumption and perfect real-time controller is the main goal of this. The data was sent using TCP/IP protocol over WIFI network. Another research concern in remote robot has been developed by Niu Zhigang and Wu Yanbo [3]. They developed a wireless remote control special design for Coal Mine Detection Robot. They investigated the embedded motion control system and apply it for Coal Mine Detection Robot's control system and wireless remote control.
The scenery around the robot in coal mine environment was transmitted to the main station, used it for controlling the robot movement such as forward, backward, turning left, turning right and tipping over the front arm. Ofir H et all have evaluated the telerobotic interface components for teaching robot operation [4]. www.ijarai.thesai.org They evaluated the control method of the robotic arm and the use of three alternative interface designs for robotic operation in the remote learning. Another system has been proposed by He Qingyun et all [5]. They created an embedded system of video capture and the transmission for monitoring wheelchair-bed service robots remotely. The embedded linux, S3C2410AL microprocessor, AppWeb 3.0 server, and blockmatching motion estimation were taken into account for obtaining better video compression data. The remote control robot system with a hand-held controller has been proposed by Dmitry Bagayev et all [6]. The dog robot for accompanying the elderly has been proposed by Wei-Dian Lai [7]. The improved interaction technique between users and the robot has been developed for making elderly easy to use.
In this research, we develop a new system in the form of domestic robot helper controlled by eye which allows patient self-service and speaks remotely. The main goal of this research is how we could develop a domestic robot helper that could be used by a handicap person in hospital environment for helping them to do not always depend on the nurse or the assistance.
First, we estimate user sight by placing camera mounted on user glasses. Once eye image is captured, the several image processing are used to estimate the sight. Eye image is cropped from the source for simplifying the area. We detect the centre of eye by seeking the location of pupil. The pupil and other eye component could be easily distinguished based on the color. Because pupil has darker color than others, we just apply adaptive threshold for its separation. By using simple model of eye, we could estimate the sight based on the input from pupil location. Next, the obtained sight value is used as input command to the domestic robot. User could control the moving of robot by eye.
Also, user could send the voice through text to speech functionality. We use baby infant robot as our domestic robot. We control the robot movement by sending the command via serial communication (utilizing the USB to serial adapter). Three types of command consist of move forward, turn left, and turn right are used in the system for moving the robot. In the robot, we place another camera for capturing the scenery in the front of robot. Between robot and user, they are separated by distance. They are connected over TCP/IP network. The network allows user control the robot remotely. We set the robot as server and user's computer as client. The robot streams the scenery video and receives command sent by the client. In the other place, client (user) receives video streaming from server and control the robot movement by sending command via the network. The user could control the robot remotely even in the long distance because user could see the scenery in the front of robot.

A. System Configuration
Our proposed eye-based domestic robot is shown in Figure  1. Block Diagram Of Eye-Based Domestic Robot The entire system consists of two parts: Client (User) and Server (Robot). Between client and server are separated by different place. The user lay on the bad while robot commutes to everywhere following user control. The signal control is initiated from user eye. We place the camera mounted on user glasses acquiring the eye image. Based on the acquired image and several image processing, we estimate the user sight. We estimate the resulted sight point on display to produce the signal control. The entire system consists of two ways/directions of communication: (1) from user to robot and (2) from robot to user.
The first direction of communication sends the command to the robot. After sight estimation, system translate it into the robot command such as go forward, turning left, turning right, and stop. We do not use the backward for safety reason (Moving in backward could cause robot in dangerous situation because user could not see the scenery in the backside. Also, the client could send voice by user typing a word and translate it into voice using text to voice function. Over WIFI network, the client and server connected each other. In the other hands, robot broadcast the scenery video via web camera placed in the robot. After acquiring the scenery, robot stream it to client using TCP/IP network. It will enable robot to be controlled as long as the covered area of the network. In locally, robot is controlled by minicomputer placed on the robot body. They connected using Bluetooth interface.

B. System Components
We implement our system by utilizing baby infant type robot with netbook PC as main controller. In the inside of baby infant, there is a controller (tiny power type) enabling us to control its movement via serial communication.
This robot is driven by two motors in the right and left sides. These two motors enable robot moves forward, turn right and left, and also backward. To make the robot turning, we just adjust the combination speed of these two motors. This robot was powered by 11.1 V 2200 mAh lithium battery.
We modify the Bluetooth to serial adapter by adding new power supply taken from tiny power controller. Also we change the mode communication of GBS301 by making a direct connection between RTS and DTS pins. The 5V voltage was taken from this microcontroller to supply the power of Bluetooth to serial adapter. The hardware of our robot is shown in Figure 2.  We make software to allow communication between two PCs or more over TCP/IP network. We utilize Winsock library to establish the connection. We make Client-Server application that will be used to control the robot over the TCP/IP Network. The application software has been created and the communication has been established. By utilizing the WIFI connection, robot could commute to other rooms. The screenshot of this program is shown in Figure 3. Also, the type of streaming data and the direction is shown in Figure 4.
There is automatic control operation mode. In the automatic control mode, user can look at moving pictures of images which are acquired with the camera attached at the tip of the robot as shown in Figure 5 (a). There are five keys which allow switch between menu and automatic modes, left/right/forward selection of arrows, and switching to key-in mode. Therefore, user can move the robot to forward direction, and to turn left or right.
Also, in the key-in mode, sentences can be input. Furthermore, sentences are read-out using voice output software tools. Then user can return to the automatic control mode again. At the initial stage, main menu of Figure 5 (b) appears. From the menu, user chooses one of the menu functions, I need meal, I need drink, move to the automatic control mode, I need toilet, I will watch TV, I would like to listen audio programs, I would like to use Internet, I need a help from nurse, I need medicines.

C. Hardware
There are three major components of the proposed system. One is robotics and the other one is cameras which are mounted on the tip of the robotics and mounted on the glass of which user is wearing. The Head Mount Display: HMD of TAC-EYE LT is also attached to the camera mounted glass. All these three hardware specifications are shown in Table 1, 2, and 3, respectively.

A. Experimental Conditions
We tested the robot control performance by controlling it moves forward about two meters and return again while we were recording the time. Before experiment, first we have to connect the robot to local network via WIFI connection. Also, the PC station network has to be established. After the connection between them was established, we began to start the server program and video streaming. Also, we start the program on client to receive the video and enable sending robot commands.
The Map used for experiment is shown in Figure 6.

B. Experimental Results
Screen shot image of moving picture of image which is acquired with the camera attached at the tip of robot is shown in Figure 6. Also, the acquired image of hand held camera of which one of the authors traced to the robot is shown in Figure  6 (bottom right). Using this camera acquired image, user can avoid obstacles. Also, the robotics can avoid obstacles using the attached ultrasonic sensor, even if the obstacles are transparent (invisible with the camera). The robotics control speed is evaluated. The time required for the route stating from the start location to the finished location is measured with five trials. The results are shown in Table 4 Start/Finish  The required time ranges from 177 to 202 seconds for traveling the route of the approximately 16 m of travel length. Therefore, robot speed ranges from 7.92 to 9.04 cm/s. It is about half speed in comparison to the specification. This would be good enough for the robot which helps patients to make order, to travel in the hospital virtually, and enjoy conversations with the other persons when the robot meets with them.

IV. CONCLUSION
The proposed eye-based robot helper has been successfully implemented. The real-time remote controller has been successfully implemented and tested in our laboratory with the average time 182 seconds for robot travelling along the 5.75 meters. Also, our robot has good maneuver to avoid the obstacles and pass it. The user easily controls the robot via the transmitted scenery image. By implementing our robot in the real application, it could help the handicap patient when they are sick in the hospital.