Speed and Vibration Performance as Well as Obstacle Avoidance Performance of Electric Wheel Chair Controlled by Human Eyes Only

—Speed and vibration performance as well as obstacle avoidance performance of the previously proposed Electric Wheel Chair: EWC controlled by human eyes only is conducted. Experimental results show acceptable performances of speed vibration performance as well as obstacle avoidance performance for disabled persons. More importantly, disabled persons are satisfied with the proposed EWC because it works by their eyes only. Without hands and finger, they can control EWC freely. I. INTRODUCTION The Electric Wheel Chair: EWC controlled by human eyes only is proposed previously. It works well in principle. Some experiments show acceptable performances previously. This paper describes the performance of speed control and vibrations as well as obstacle avoidance performance.


I.
INTRODUCTION The Electric Wheel Chair: EWC controlled by human eyes only is proposed previously.It works well in principle.Some experiments show acceptable performances previously.This paper describes the performance of speed control and vibrations as well as obstacle avoidance performance.
The proposed system consists of forward and backward looking Web cameras mounted glass and pocket PC that allows Bluetooth communications.Thus users can be moved using the system.Pocket PC can be communicated with not only with Input and Output devices but also the other pocket PCs mounted on the other Electric Wheel Chairs: EWCs so that created and updated map information can be shares with many EWCs.The system provides obstacle finding with forward-looking camera so that EWCs can avoid obstacles.Location information of obstacles is uploaded to the other EWCs through Bluetooth communications.Thus all the EWCs can be controlled safely avoiding obstacles with the shared map information.
The following section describes the proposed system followed by experiments for the proposed system in terms of control performance of EWC, in particular, obstacle avoidance performance.

A. Hardware configuration
Hardware configuration is shown in Figure1.The proposed system consists of (1)

B. The method used and system parameters
In order to control EWC, at least four keys, move forward, turn right, turn left and stop are required.For the safety reason, users have to look forward so that the key layout that is shown in Figure 2 is proposed.Stop key is aligned on the other rows, top and bottom rows.Users understand the location of desired key so that it can be selected with users' eye-only.The backward looking Web camera whose resolution is 640 by 480 pixels acquires users' eye and its surrounding.Using OpenCV (14) of eye detection and tracking installed on the Pocket PC, users' eye is detected www.ijarai.thesai.organd tracked.If the OpenCV cannot detect users' eye, then EWC is stopped for safety reason.EWC is also stopped when users look at the different location other than the three keys aligned on the center row.When users are surprised human eyes used to be large.Such a situation can be detected with acquired image with backward looking Web camera so that EWC is stopped.
Intentional blink can be detected if the eye is closed for more than 0.4 seconds because accidental blink is finished within 0.3 seconds.In this connection, it is easy to distinguish between intentional and accidental blink.Also, key selection can be done every 0.4 seconds.Thus the system recognizes user specified key every 0.4 seconds.In order to make sure the user specified key, 10 frames per seconds of frame rate is selected for backward looking camera. (17)igure 3 shows the process flow of eye detection and tracking.Eye is detected by Viola-Jones classifier.The viola-Jones classifier employs ADABOOST at each node in the cascade to learn a high detection rate the cost of low rejection rate multi-tree classifier at each node of the cascade.To apply the viola-Jones classifier on the system, we use viola-Jones function in OpenCV (15) .Before use the function, we should create xml file data.The training sample (face or eye image) must be collected.There are two sample types: negative and positive sample.Negative sample correspond to non-object images.Positive sample correspond to object image.After acquired image, OpenCv will search the face center location and continue with search the eye center location.Advantage this function is fast and robust.

D. Template Matching
Eye behavior is detected by template matching.Template matching which used is not based on histograms; the function matches an actual image patch against an input by sliding the patch over the input image.There are several template matching methods:

1) Square difference matching methods
These methods match the square difference, so perfect match will be 0 and bad matches will be large.
This method will obtain good result only if both images have same pixel intensity.Because of output result only 0 and 1, it not sophisticated for the proposed system.

2) Correlation matching methods
These methods multiplicatively match the template against the image, so a perfect match will be large and bad matches will be small or 0.
These methods match a template relative to its mean against the image relative to its mean, so a perfect match will be 1 and a perfect mismatch will be -1; a value of 0 simply means that there is no correlation.(5) In this system we used Correlation coefficient methods and give additional normalized to reduce the effects of lighting difference between the template and the image.
Result values for this method that give the normalized computation are: In this system, four template images are used to determine eye gaze.These images are acquired during calibration step.The best result will obtain eye gaze.The threshold is set to 90% match.www.ijarai.thesai.org

E. Calibration Step
Eye gaze measurements that apply template matching methods will does work on fix illumination and fix condition.Illumination changes, shadow, different of eye shape when it is used by other users, and others difference condition will obtain mismatch result.This problem can be solved by calibration step.The calibration step consists of acquiring template image and self evaluation as is shown in Figure 4. System will acquire eye image when looking at down, up, left, and right keys.Next, system will evaluate template image.If template image is good template, it will be used.If not, system will acquire eye image again until good template are obtained.When system is started, it will check that template images are proportional.Illumination, shadow, and eye shape will compare with current eye image.If there are not proportional image, calibration step will be ran.

F. Custom Microcontroller
Yamaha JW-I Electric wheel chair type is used.To control EWC using Pocket PC, custom microcontroller circuit is used.USB interface on pocket PC is used to connect with other peripheral.The custom microcontroller circuit is use RS232 communication.Microcontroller AT89S51 type is used.To connect between pocket PC and custom microcontroller circuit, USB to Serial communication is used.The microcontroller will drive relay to move the EWC.Microcontroller connection is shown in Figure 5.

G. Obstacle Avoidance
In order to safety reason, obstacle avoidance system is implemented in our system.Obstacle avoidance system will able to identify the obstacle in front of EWC and avoid it.This system consists of two approaches: (1) Obstacle detection, and (2) Best Path Finding.

H. Obstacle Detection
Obstacle detection is consisting of image processing based and ultrasonic sensor based.Image processing based utilizes background subtraction between current image I(x, y) and background image B(x, y). Background subtraction method will obtain black-white image S(x, y) which represent obstacle.On this image, obstacles appear as white pixel.By using searching of outer line from white pixels, we can determine position and size of obstacle.
Weakness of background subtraction method is working only if two images have same position of translation, scale, shear, and rotation.To solve these problems, we utilize affine transformation.This transformation requires 3 points that appears on both images as is shown in Figure 6.Translation, scale, share, and rotation parameter can be determined from these points.Affine transformation required at least three noticeable important points.These points should be appears on both images.It can be detected by several ways: corner, edge, specific object, and etc. Corner and edge have many points and its will create computation problem.In our system, we decide to use specific object.The specific object can be an easily recognized object, text character, chessboard wall, and etc. Flow process of obstacle detection using affine transformation is shown in Figure 7.
The specific object will obtain one coordinate from center of area.Obstacle detection using affine transformation requires identifying at least three kind of object, so it will obtain three noticeable important points.
First step of obstacle detection is convert source image into gray image.By using template matching, system will find specific object position.Using six noticeable points, system calculate translation, share, scale, and rotation parameter.These parameters will used for creating affine matrix.www.ijarai.thesai.orgAffine matrix apply on background image will obtain affine image.Normalization is used to eliminate disturbance such shadow, noise, and etc. Next, subtract between output form current image and background image.Smooth filter is used to reduce noise which is caused by subtraction process.Last step is applying threshold on image and it will obtain black and white image.Obstacle will be signed as group of white pixel.After black white image is obtained, we should return center of white pixel area into current coordinate and coordinate of obstacle is founded.Because of so many type of specific object, we got best performance of specific object by using chessboard wall.The advantage of the chessboard (Figure 8) is easily detected and robust on distance changes.We use 3 types of chessboard: 3 by 4, 3 by 5, and 3 by 6 as are shown in Figure 9.The other obstacle detection is use ultrasonic sensor.This sensor has advantage when visual system does not work.In case EWC move into surrounding glass door, smoke condition, and minimum lighting will caused visual system of obstacle detection fail.Ultrasonic sensor consists of transmitter and receiver part.We use ultrasonic sensor with 40 kHz frequency resonance.

I. Best path finding
When EWC detect the obstacle, it must understand where the best path should be chosen if EWC want to go into specific place.Best path will be chosen based on floor layout map and image map.Image map is created by acquiring background images in every 1 m as is shown in Figure 10.In every location (x, y) have one image.Instance, if the location has size 10 x 10 m, it will have 100 images.After acquire background images, floor layout map will be added manually as initial condition.This map is setup manually based on room layout.Example is shown in Figure 11.
Combination between background images and layout path will obtain main map.EWC will move consider the main map.After obstacle is detected in current path, EWC will switch to another path which have same destination.Obstacle avoidance methods are also useful when user is not confident to pass the obstacle.EWC will take over and pass the obstacle.Best path is chosen based on Dijkstra algorithm.Furthermore, System also able to renew existing map.

A. Key selection
The proposed method experiments by measuring success rate eye gaze detection as hit keys selection.Success rate of eye gaze detection is measured by change user distance to camera.Experiments say that minimum distance of success rate is 13 cm and maximum distance is 38 cm.The distance beyond these, OpenCV will not detect the eye and caused the system fail to measure eye gaze.Hit keys selection experiment was done by using real time video 640 by 480 pixels.Data experiments when measure hit keys selection on range distance 24 -30 cm is shown in Table 1.The result of hit key measurement is shown almost perfectly, Error which cannot understand the eye input is caused by the system fail to verify the template image.Error will zero if good quality template is used.To make sure that good quality template is used; System will always verify every template and analyzed it.If templates have poor quality, system will conduct calibration step again.This step will repeat until good quality template is obtained.

B. Obstacle avoidance performance.
Experiment of obstacle avoidance is conducted by acquiring image in corridor with distance 1m per images.Location which image was acquired, is set with (x, y) coordinate.So, in every (x, y) location will have 1 background image.This image will be used as background reference.Obstacle is detected by subtract background image with current image.To eliminate problem which caused by different position between these images, affine transformation is used to transform background image to affine image which have same position with current image.Applying affine transformation will does work if only if three noticeable important points are appears on both images.These important points are represented by three types of chessboard.If the chessboard is successful detected, then by using affine transformation and subtraction between background image and current image, obstacle is founded.
Experiments of chessboard detection is conducted by measuring maximum location which still detected by system.Chessboard is put in fix location.After that, EWC is move around the chessboard.Maximum location which detected by system is recorded.Data experiments are shown in Figure 12.Data show that boundary shift will decrease when distance between camera and chessboard is increase.This experiment is equal to obstacle detection.After three types of chessboard are detected, affine transformation will use these points (three chessboard center of areas) to subtract with current image and obstacle position will be found.

C. Ultrasonic Sensor
Objective of this experiment is measure accuracy of ultrasonic sensor before used in the system.This experiment is conducted by measure sensor output for measuring distance and comparing with real data.Some object is put on front of sensor with varies distances and measure it.The experiment is conducted on distance 0 cm until 3 m.Ultrasonic sensor use PING type parallax product and microcontroller AT89S51 as processing data (convert from time value to distance output).Graph of accuracy sensor is shown in Figure 13.Fig. 13.Experiment of ultrasonic accuracy.This shown that minimum distance is around 3 cm and maximum distance is 3 m.This range is appropriate for detect the object.
Elevation angle is require to know how width of the beam sensor.Ultrasonic sensor with width beam is not benefit to our system.Narrow beam will obtain good result because it will not influence with any disturbance.This experiment is conducted by measure elevation angle from 0 cm until 3 m.Graph of elevation angle is shown in Figure 14.

D. Performance of Electric Wheel Chair
This experiment is conducted for measure EWC performance on starting up acceleration, forward and backward breaking deceleration.Also conducted speed measurement when EWC move forward, backward, turn left, and turn right.EWC is drive by user who has weight is 73 kg.We record the duration time and convert it into speed value.Experiment data of speed measurement is shown in Table 2. Graph of EWC acceleration and deceleration when start and stop duration is shown in Figure 15.

E. Processing time each process
In order to apply whole method into EWC application, processing time should be measured to identify performance of our real time system.Figure 16 shows transient time of eye detection and tracking, n the beginning of chart, it seem this method take long time around 300 ms.In this time, system still process face detection, eye detection and others process before running template matching method.After eye location is founded, then system bypass previous step and cause process working fast.Meanwhile, Figure 17 shows processing time of eye detection and tracking on steady state condition, it looks faster than transient condition.Fig. 16.Transient time of eye detection and tracking, n the beginning of chart, it seem this method take long time around 300 ms.In this time, system still process face detection, eye detection and others process before running template matching method.After eye location is founded, then system bypass previous step and cause process working fast.

F. Eye detection and tracking
This experiment is conducted using Optiplex 755 Dell computer with Intel Core 2 Quad CPU 2.66 GHz and 2G of RAM.We use NET COWBOY DC-NCR131 camera as visual input.Experimental result show average steady state processing time is 32.625 ms. it also shows difference processing time between transient and steady state condition.Transient time require more time than steady state time.

G. Eye gaze measurement
Objective of this experiment is measure processing time on Eye gaze identification.It is conducted by using ACER computer ASPIRE 5572 Series Core Duo T2050 1.6 GHz CPU and 1G of RAM.Result data show average processing time of this method is 342.379ms. Figure 18 shows processing time of Eye gaze method.

H. Obstacle detection
This experiment also was conducted using Optiplex 755 Dell computer with Intel Core 2 Quad CPU 2.66 GHz and 2G of RAM.NET COWBOY DC-NCR131 camera as visual input is also used.Experimental result show average processing time is 32.625 ms. Figure 19 shows processing time of obstacle detection.

I. Ultrasonic sensor
We implemented ultrasonic sensor parallax PING type.This sensor is controlled by using custom microcontroller AT89S51.Data was stored into computer by using USB communication.Result data show average processing time is 568.658ms. Figure 20 also shows processing time of ultrasonic sensor, it look take longer time than others.IV.CONCLUSION It is concluded that the proposed key-in system with human eyes only works 100% perfectly for the number of keys is four, start, stop, turn right and left.Also it is concluded that the proposed EWC control system does work in a real basis avoiding obstacles on range bellow 3.4 m using image processing method and bellow 3 m using ultrasonic sensor.By the proposed system, EWC is able to identify obstacle and avoid them.Obstacle avoidance can assist user into interest place although undesired condition such as dark areas, glass wall or door, smoke area, and etc.By implemented this system, EWC will move more safely and comfortable.

Fig. 2 .
Fig. 2. 3 by 3 of key layout for EWC control Namely, key consists of 9 keys (3 by 3).Move forward and turn left/right are aligned on the middle row.Stop key is aligned on the other rows, top and bottom rows.Users understand the location of desired key so that it can be selected with users' eye-only.The backward looking Web camera whose resolution is 640 by 480 pixels acquires users' eye and its surrounding.Using OpenCV(14) of eye detection and tracking installed on the Pocket PC, users' eye is detected

Fig. 4 .
Fig. 4. Result of calibration step.These steps obtain up, down, left, and right image.Up image is used for move forward, down is used for stop, left is used for turn left, and right is used for turn right

Fig. 5 .
Fig. 5. Microcontroller AT89S51 connects to other peripheral through serial communication.Serial communication type should be converted to USB communication using USB to serial converter.

Fig. 6 .
Fig. 6. Background subtraction method, left-top is background image and right-top is current image.Background subtraction method will obtain blackwhite image (bottom)

Fig. 7 .
Fig. 7. Obstacle Detection.This methods rely on subtraction between two images and affine transformation.

Fig. 14 .
Fig. 14.Experiment of Elevation angle.Top side is distance versus angle, and bottom side is distance versus distance elevation.

Fig. 17 .
Fig. 17.Processing time of eye detection and tracking on steady state condition, it looks faster than transient condition.

Fig. 20 .
Fig. 20.processing time of ultrasonic sensor, it look take longer time than others.