Image Processing based Task Allocation for Autonomous Multi Rotor Unmanned Aerial Vehicles

Nowadays studies based on unmanned aerial vehicles draws attention. Especially image processing based tasks are quite important. In this study, several tasks were performed based on the autonomous flight, image processing and load drop capabilities of the Unmanned Aerial Vehicle (UAV). Two main tasks were tested with an autonomous UAV, and the performance of the whole system was measured according to the duration and the methods of the image processing. In the first mission, the UAV flew over a 4x4 sized color matrix. 16 tiles of the matrix had three main colors, and the pattern was changed three times. The UAV was sent to the matrix, recognized 48 colors of the matrix and returned to the launch position autonomously. The second mission was to test load drop and image processing abilities of the UAV. In this mission, the UAV flew over the matrix, read the pattern and went to the parachute drop area. After that, the load was dropped according to the recognized pattern by the UAV and then came back to the launch position. Keywords—UAV; multi rotor; quad rotor; image processing; search and rescue; task allocation


I. INTRODUCTION
Nowadays Unmanned aerial vehicle (UAV) technology is becoming very popular.It is finding a place in many sectors and studies.This technology is attracting attention to the defense industry, but the usage of UAVs is rapidly increasing every day in civil-life.
From firefighting to search and rescue, agriculture to photography, in many studies UAVs have been used for a long time.So it can be said that the field of use of UAVs is vast.For example, while Karaca et al. were researching in order to localize and rescue the victims on mountainous areas [1], Zhou et al. were studying on real-time video registration for forest fire monitoring [2].In [1] and [2] UAVs were used for video capturing and data transmission.However, the technology kept evolving.Some researchers made a significant improvement on the mechanical design of the UAV in order to increase the performance of the search and rescue [3], and some of them changed the camera type with an infrared one and increased the performance of the fire detection [4].In next years, a group of researchers changed their point of view [5].Getting closer to recent days, Yuan et al. developed an algorithm to detect and track forest fire [6].Giving another example of an image processing application for UAVs, Moranduzzo et al. added machine learning to image processing and used support vector machine (SVM) in order to detect cars from UAV images [7].
UAVs are used not only for such applications but also for transportation.In their study, Goodchild et al. wrote a paper estimating the carbon dioxide emission of the vehicles while delivering goods and reducing this emission rate by using UAVs for the delivery [8].Another interesting paper is written by Zheng et al. simulating and evaluating the safety of a long distance search and rescue mission executed by a UAV [9].
Considering this amount of applications and the importance of image based tasks on UAVs, in this paper, we have allocated tasks to a UAV in order to measure the performance of its autonomous flight, image processing ability and autonomously load drop capability.These features are considered as the essential features for a smart UAV.There were two missions to evaluate the performance of the UAV.In the first mission, the UAV flew over a 4˟4 sized color matrix.16 tiles of the matrix had three primary colors, and the pattern was changed three times.The UAV recognized 48 colors of the matrix and returned to the launch position autonomously.The second mission was to test load drop and image processing abilities of the UAV.In this mission UAV flew over the matrix read the pattern and went to the parachute drop area.Three loads with parachutes were dropped according to the recognized pattern of the matrix, and then the UAV returned to the launch position.
More information about the missions and the detailed design of the UAV is given in the methodology part.The output of the missions and the performance of the UAV are given in conclusion.

A. Mission I
In this mission, autonomous flight and image processing capabilities of the UAV is tested.UAV has a system to fly over a 4˟4 color matrix, which contains 1m˟1m sized color panels, detect and record the matrix colors during the flight.In this mission, color detection and recording will be done three times, and every time the matrix color changes colors will be given randomly to the matrix.Panel colors will be stable for 10 seconds, and there will be 5 seconds waiting time between color changes.Fig. 1 shows the color changes in the first mission.

B. Mission II
In this mission, autonomous flight, image processing and parachute dropping in the correct order capabilities of the UAV www.ijacsa.thesai.orgare tested.UAV flew over a 4˟4 color matrix contains 1m˟1m sized color panels.As it can be seen in Fig. 2, there are three different colored rows among which every row's panels have the same color.The last row will have random colors.According to the first three rows' colors, the parachutes are dropped with 5 seconds gaps.The mission area is shown below.

C. Autonomous System
A flight controller board (Pixhawk 2.1) was chosen in order to accomplish the mission and to control its process, Mavlink Protocol is used.MAVLink (Micro Air Vehicle Link) is a protocol for communicating with small unmanned aerial vehicles.It is designed as a header-only message sequence library [10].
The software to control the autonomous flight was written in Linux based using Drone-kit API in Raspberry Pi 3. The algorithm of the software is shown in Fig. 3.

D. Image Processing Algorithm
Several image processing algorithms are used in order to perform both missions.In the first mission which is to recognize the given patterns, the algorithm waits for the UAV to fly on the pattern.After the UAV arrives at the given position on the pattern, image processing algorithms start.After capturing the first frame of the pattern preprocessing is applied, the image is converted to the HSV color space [11].The main reason for the usage of HSV color-space is to detect color under any circumstances independently of light intensity and brightness.The median filter is used to get rid of the noise in the image.H (hue), S (saturation) and V (value) value range for every color are determined so that the background will be subtracted and every color will be detected separately (Fig. 4.).After detecting every color midpoints of every square, coordinates are written in an array, and that array is sent via RX-TX ports into the SD card.The output of the system can be seen in Fig. 5. www.ijacsa.thesai.orgThe same algorithm for the second mission is also used.After getting the colors of the pattern first three rows' colors are read, according to those colors the related command is sent to the processor to drop the parachutes in the correct order.
The first mission is planned to be completed in totally 50.7 seconds.When necessary calculations are made the UAV speed is estimated about 58km/h, which makes 16.11m/s, and the flight height is planned about 10-18m.With this speed and the height, the way going to the color matrix and coming back would take 10.7 seconds.For every color change on the matrix, the process speed of the processor to detect the colors is estimated about a maximum of 10 seconds.The algorithm for the first mission is given in Fig. 6.
The second mission, when the flight speed and the height are considered, is estimated about 23.67 seconds.For color detection, the minimum process time is aimed, so the color detection and the parachute-drop order planning time is estimated about minimum 5 seconds.Due to weather conditions, all of these calculations can be changed.That is why estimation is used when planning the mission performance.With the best estimation, the best performance is aimed.The algorithm for the second mission is given in Fig. 7.The lightest and the fastest UAV design for the missions is chosen.The designed UAV's measurements are 231˟231˟88.40mm.Considering motor and propeller features, the cross motor span is designed 280 mm.The gaps between vertical and horizontal motors are 200 mm.GPS is not included in these measurements.Technical drawing of the UAV is given in Fig. 8.The final design of the UAV is given in Fig. 9.As mentioned before for image processing tasks, Raspberry Pi 3 is used and for obtaining images Pi Camera which is a compatible camera with Raspberry Pi cards.For autonomous flight, an autonomous flight card called The Cube Pixhawk 2.1 is used.Pixhawk 2.1 includes 32 bit STM32F427, 256 KB RAM, 32 bit STM32F103, two inertial measurement units (IMU), barometer, 3 axis gyro, accelerometer and magnetometer.

IV. EXPERIMENTAL RESULTS
On the first mission, the flight time was planned to be 50,3 seconds.Real-time performance of the mission was 41 seconds.Image processing performance is shown below.The recognition of the first pattern is shown in Fig. 10.Only one mistake in 16 squares was made.In Fig. 11, the second pattern is shown, and the recognition rate of the pattern is %100.In Fig. 12, recognition of the third pattern was done with 100% success rate.In the second mission, the flight time was estimated about 23.67 seconds.The real-time flight time is calculated for 26 seconds.During the flight, a small GPS problem occurred, and the UAV could not hover precisely on the pattern.However, the recognition performance of the pattern was 100% successful.In Fig. 13, the pattern is shown, and the first three rows were recognized correctly.After the recognition, the parachutes were dropped in the correct order (Fig. 14.).www.ijacsa.thesai.org

V. CONCLUSIONS
During flights, although some problems occurred, the missions were completed successfully.In the first pattern of the first mission, one square could not be recognized due to the angle of the camera and the light.In order to prevent this situation, some machine learning algorithms can be used.For example, the pixels of the image can be clustered with K-Means algorithm into few clusters so that the colors of the image would have been clustered into a few primary colors [12].In this way, recognition can be done in many different environments.In this study, the UAV is designed for the missions, and we believe this study can lead to many different studies such as firefighting, search, and rescue, surveillance applications.With the addition of deep learning and other machine learning algorithms, image processing performance can be increased, and the area of study can also be widened this drone could be beneficial for the defense industry.Authors believe that UAVs will have a massive part in our life and this study shows the advantages of UAVs in a significant way.

Fig. 5 .
Fig. 5.The Output of the Color Detection Algorithm.

Fig. 6 .
Fig. 6.The Algorithm of the First Mission.

Fig. 7 .
Fig. 7.The Algorithm of the Second Mission.