Onion Crop Monitoring with Multispectral Imagery using Deep Neural Network

The world’s growing population leads the government of Pakistan to increase the supply of food for the coming years in a well-organized manner. Feasible agriculture plays a vital role for sustain food production and preserves the environment from any unnecessary chemicals by the use of technology for good management. This research presents the design and development of a multi-spectral imaging system for precision agriculture tasks. This imaging system includes an RGB camera and Pi NoIR camera controlled by a raspberry pi in a drone. The images are captured by Unmanned Aerial Vehicle (UAV) and then send images to the Java application. Images are processed to sharp, resize by application. The Normalized Difference Vegetation Index (NDVI) is calculated to determine the crop health status based on real-time data. The Deep Learning (DL) technique is used to recognize the onion crop growth stage using the captured dataset. We express how to implement a progressive model for the deep neural network to recognize the onion crop growth stage. The performance accuracy of the system for batch size 16 is 96.10% and for batch size 32 is 93.80%. Keywords—UAV; deep neural networks; onion crop; NDVI; crop monitoring; VGG16


I. INTRODUCTION
In all homes around all the year, Onion is one of the essential condiments broadly used. Recent research has recommended that in the diet, onions may play a part in avoiding heart disease and other illnesses. Onion is one of the essential crops in all continents with worldwide production of about 25 million tons. In Pakistan, there has been a progressive growth in the area and production of onion and is commercially grown with a production of 1.8 million tons on an area of 131.4 thousand hectares. The major onion growing districts are Hyderabad, Badin, Mirpurkhas, Nawabshah, Sanghar, Shikarpur, Nosharo Feroze, Ghotki and Dadu in Sindh; Kasur and Vehari, in Punjab; Chaghi, Kalat, Killa Saifullah, Nasirabad, Mastung, Khuzdar, Kharan, Turbat, and Jaffarabad in Balochistan and Swat and Dir in Khyber Pakhtun Khuwa [1,2]. The general objective of the Agriculture Division is to grow farm yield and guarantee increased incomes for the farmers, particularly smallholders. With the predictable growth in the country's population and per capita income, marketing is usual to play an important role in guaranteeing that clients obtain food at reasonable costs and what is the food quality.
Remote sensing can be described as the obtaining of information about an object through sensors without approaching in to direct interaction with it [3]. Since the last part of the 1980s, incredible advancement has been made in remote sensing in precision agriculture [4]. Precision agriculture can be measured as a management methodology of temporal and spatial inconsistency in fields utilizing communication and information technologies [5] with the means to improve sustainability agro-environmental services. In this respect, expanding the accessibility of technological solutions are obtainable to remotely fetch and transfer environmental parameters [6,7]. Unmanned Aerial Vehicle (UAV), usually identified as a drone, is an aircraft without a human pilot on panel. UAVs are a module of an unmanned aircraft system (UAS), which consists of a UAV, a groundbased controller, and a method of communications between both. The airlift of UAVs may function with several grades of autonomy: either control by a human with help of remote control or automatically with help of onboard computers denoted as an autopilot [8]. In previous years, a variety of UAV models running on military and civilian applications [9]. The essential fact related to UAV technology is, it has been probably 75% of UAVs like drones from 2016 to 2024 will be used in applications related to the precision of agriculture. Reliable and fast information of crop fields by capturing Images through UAV. The " Fig. 1" shows the drone which is used in this research to capture images with help of RGB and Raspberry Pi NoIR Camera and both cameras are embedded with Raspberry Pi 4. Now a day, Farming is done in the traditional ways in Pakistan. The reality is that our farmers are mostly nontechnical persons or poor persons and they have lack of appropriate information that makes it more unreliable. A big part of farming activities is depending on the guesses and these guesses mostly fail on time. Therefore, we come with an idea of crop monitoring via UAV. We consider that this idea becomes standard in the agri-business cause of its reliability and remote monitoring. With the help of this idea, we try to digitalize agricultural activities and farming, and farmers can test on necessities of the crop and perfectly calculate the growth of the crop. We believe that our idea will confidently speed up their business to reach new heights and it will be more profitable. The implementation is mostly based on awareness among farmers, which will be produced due to its various benefits. In precision agriculture applications, the use of UAV [10] has been increased in the last three years. This is primarily due to UAV ability is to deliver the farmers essential information regard crop condition (health) for good input management. *Corresponding Author 303 | P a g e www.ijacsa.thesai.org VGG16 is a variant of Visual Geometry Group (VGG) [11] with 16 convolution layers and is very attractive because of its very unvarying architecture. It is presently the most preferred choice in public for extracting features from pictures. The weight formation of the VGGNet is widely accessible and has been utilized in several other applications and challenges as a baseline feature extractor. Instead of having a big number of hyper-parameter, they concentrated on having convolution layers of 3 X 3 filter with a stride 1 and every time utilized comparable padding & maxpool layer of 2 X 2 filter of stride 2 and this is the most inimitable thing about VGG16. It follows this procedure of convolution & maxpool layers regularly through the entire model. At last, it has 2 Fully Controlled (FC) layers tailed by a softmax for results.
Considerable damage is done to onion crops by using high or low quantities of fertilizers and water. Farmers are facing difficulties in the crop monitoring field at a bigger scale, therefore the use of UAV will allow farmers to constantly monitor their crop. This research aims to design UAV-based application to improve the health and production of onion crops. One of the major goals of this research was to achieve a state of the art classifying results using widely helpful data and models, with transfer learning to speed up training processes & balances the limited sample size so that the latest hardware can deliver sensible outcomes. This paper shows monitoring of onion crop, through the use of Near-Infrared (NIR) & RGB imagery captured by Unmanned Aerial Vehicle.

II. RELATED WORK
A brief overview of a few significant contributions from the existing literature is provided in this section.
A. Montes, et al. [8] proposed a Low-cost multispectral imaging system for crop monitoring. The author used digital & NIR cameras to calculate orthomosaics images from NIR & RGB images using built-in software FIJI. The limitation of this system is the use of JPEG format images which makes final NDVI representation lower as compared to the TIFF format.
Gaetano Messina et al. [12] proposed Monitoring Onion Crops Using Multispectral Imagery from UAV. The author used a fixed-wing UAV, fitted out with Multispectral Camera Sequoia Parrot (R-G-RedEdge-NIR). The results of the analysis of the three datasets showed a high correlation of Green Normalized Difference Vegetation Index (GNDVI) and NDVI with Soil-Adjusted Vegetation Index (SAVI). The use of a higher resolution sensor might probably have partly solved these problems.
Kim et al. [13] proposed Machine vision-based automatic disease symptom detection of onion downy mildew aim to train the Deep Neural Network (DNN) model for detection of disease symptoms using Pan, Tilt, Zoom (PTZ) camera, wireless transceiver, a motor system, and image logging module. They achieved an accuracy of 90.7%.
Chauhan, S. et al. [14] proposed Wheat lodging assessment. This study's main object was to analyze spectral changeability of the features derived for UAV data and their capability to differentiate between dissimilar grades of lodging severity. Using an object-based segmentation result, the nearest neighborhood classification was executed with an overall accuracy of 90%.
Raeva, Paulina Lyubenova, Jaroslav Šedina, and Adam Dlesk [15] proposed monitoring of Corn, Barley crop fields. The author used multispectral and thermal cameras to monitor these crop fields. 3 vegetation indices were determined. Furthermore, 2 thermal maps are existed to show the relation between vegetation and soil temperature. The conclusion is that the values vegetation indices obey with definite growth of crops as known from certainty. It means that thermal & multispectral imagery has great potential in agriculture.
Biao Jia et al. [16] proposed "Use of Digital Camera to Monitor the Growth and Nitrogen Status of Cotton". The objective of this study was to develop a non-destructive technique for monitoring the growth and nitrogen status of cotton crops using an RGB camera. The use of digital cameras as a tool for near-ground remote sensing in precision agriculture is a new field of research. C Y N Norasma et al. [17] proposed Rice crop monitoring using multi-rotor UAV and RGB digital camera at the early stage of growth. The paper's objective is to monitor rice crops by using UAV and RGB digital camera in Kelantan.
Shan et al. [18] purpose to predict Covid-19 in Computed Tomography (CT) scan by utilizing the deep learning model named VB-Net. They used 250 images for training, and 300 pictures for validation. A precision of 91.6% they achieved.

III. MATERIALS AND METHOD
In this section, we have discussed in detail our proposed methodology. The " Fig. 2" shows the Overall System Architecture based on the two phases. In the 1st phase, a UAV is used to capture images of the onion crop using both Raspberry Pi NoIR and RGB camera from the field and send these images to the application for further processing. The 2nd phase is to develop an application that monitors onion crop health and recognizes the onion crop growth stage. The dataset is also collected from different fields of onion crops with different growth stages. Images captured by the UAV are further processed in the application where an image sharpen filter is applied and resize the image to get more accurate 304 | P a g e www.ijacsa.thesai.org results. After that application provides two features, NDVI & recognizes the onion crop growth stage.
To calculate NDVI (Normalized Difference Vegetation Indices) values of the captured images of onion crop using the application. The NDVI is computed as following expressions: Where NIR resembles to the Near-Infrared's reflectance values and RED correspond to Red radiations imitated by plants. Images containing NIR and RED radiations from vegetation are mandatory to obtain NDVI.
To recognize the onion crop growth stage, we use the deep neural networks method (VGG16) consisting of a convolution layer, maxpool layer, fully connected (hidden layer), and an output layer. The " Fig. 3" gives details of the VGG16 model.

A. Input Layer
The input layer is responsible for fetching a pre-handled onion crop picture dataset. It carries primary data into the system for additional processing by following layers of deep learning. The input layer is the very starting layer of the workflow of Deep Learning (DL) [19].

B. Convolution Layer
This layer will perform most computations that why this is the critical layer in our proposed DL model. Convolution layers are the significant building blocks utilized in deep learning. Convolution is the basic application of a filter to an input that outcomes in an activation. This layer's main purpose is to recuperate features from the dataset of images and to preserve the spatial relationship between pixels.

C. Max Pool Layer
Max pooling is a pooling process that chooses the most extreme component from the locale of the feature map covered by the filter. Accordingly, the outcome after the max-pooling layer would be a feature map having the most noticeable features of the past feature map.

D. ReLU Layer
This layer generates the non-linearity plan of the DNN features. The aim is to put the negative values of a pixel with 0 in the convolved features.

E. Fully Connected / Hidden Layer
All activation functions of the previous layer are associated with the neurons of this layer. The principal assignment of this layer is to order the gathered features in the predefined classes from the image data sets.

F. Softmax Layer
This layer is purely used to deliberate the conceivable values of the preceding layer activation function. The values can be taken in four sets of 'Initial Stage', 'Middle Stage', 'More than Middle Stage', and 'Final Stage' in the recognition case.

G. Output Layer
The output / final layer of the DL model can be categorized with outcomes of previous layers. It results in the recognition of the onion crop growth stage, for example, Initial Stage, Middle Stage, More than Middle Stage, and Final Stage.
The " Fig. 3" shows the VGG-16 architecture of the RGB images. Convolutions are used to expand the 3rd dimension but 1st and 2nd dimensions are regularly remains untouched. Pooling layers are used to reduce (dividing with 2) 1st and 2nd dimensions (Height X Width) & left 3rd dimension unaffected. VGG-16 has convolution kernels of uniform size 3 X 3 with stride as 1 just applies them several times. Using equation (2), it's not difficult to obtain comparable values.  Where ConvMat is convolutional matrix dimensions, OriMat is the original matrix, P is the padding number, S is the striding number of positions and F is the filter dimensions.

IV. RESULTS AND DISCUSSION
We demonstrate and discuss the outcomes of our system developed and tested under preset conditions in this section. This study was conducted in an onion field located in Hala New, Sindh, Pakistan. The total field acreage was 6 acres with a well-organized plot.

A. Flight Plan and Image Pre-processing
The UAV takes a flight of 2 minutes with an altitude of 7 feet and captures the images with Raspberry Pi NoIR camera for NDVI calculation with a 10-second interval in an acre of field and after that, it reduces altitude to 3 feet and again starts capturing the images from 3 feet altitude with RGB camera for Crop Growth Stage and send these images to the application. After sending images to the application the image sharpens filter is applied, resize the image and then process the images for NDVI calculation & Onion Stage recognition.
The " Fig. 4(a)" is the original image captured by the RGB camera and then the RGB image is sharped and resized to process and recognize the crop growth stage by application. We have the sharper image, as shown in " Fig. 4(b)", the edges are visible to recognize the image correctly and results will be accurate. We have also sharper the NIR images to compute NDVI results accurately because the images are taken from 7 feet height with UAV. The " Fig. 5(a)" is the original image capture by the Raspberry Pi NoIR camera and " Fig. 5(b)" shows the sharper image by using the image sharpening technique through the application.

B. Compute NDVI
The NDVI computation outcomes range from -1.0 to 1.0. Negative values relate to regions with rocks, manmade structures, snow, water surfaces; simple soil typically comes in between 0.1 to 0.2 ranges, and plants will constantly have positive values 0.2 -1.0. Healthy plants should be above 0.5, and damages and unhealthy plants will most likely come in between 0.2 to 0.5.  The " Fig. 6" shows that we take the near-infrared image, process it using the NDVI calculation algorithm, and then we calculate the NDVI of the processed image. The system shows NDVI of 0.78 -Onion Crop is very healthy.

C. Onion Crop Growth Stage Recognition
In The RGB images are processed to recognize the onion crop growth stage. Computer Vision problems tend to be difficult to resolve with an incredible improvement of deep learning. Neural Networks (NN) are attempting to learn profoundly complex functions as much as possible, like image recognition or image object detection.  It is one of the motives of the NN. We have a cluster of pixel values, and from that point; we might want to sort out what is in the image, so this is an unpredictable issue. We are reusing previously successful architecture by decreasing the required time for selecting alternate neural hidden layers, convolution layers, and other configuration parameters, for example, learning rate. We used java deep learning to train our system that system is capable to recognize the onion crop growth stage. For use on distributed GPUs & CPUs, Deep Learning for Java (DL4J) fetches Artificial Intelligence (AI) to business environments. By creating a java application, the methodology was established. The execution was GPU defined. All trials were done on an HP Elite Book AMD PRO A10-8700B R6, 10 Compute Cores 4 CPU + 6 GPU of 1.8 GHz, 12 GB of RAM. All research trials were carried out with 80% of the data-set for training while 15% of the dataset for testing the leftover 5% is for validation. Now we train our system with java DNN because DNNs capability is to achieve record-breaking accuracy. The " Fig. 8" shows the feature extraction with help of the convolution layer. VGG-16 is capable to predict 1000 classes of images, while we require only 4 classes that are Initial Stage of Crop, Middle Stage of Crop, More than Middle Stage of Crop, and Final Stage of Crop. We have slightly modified this model to results in only four classes instead of 1000. The " Fig. 8" shows the feature extraction from images by inserting some convolutions and pooling layers that are utilized to train a big neural network with FC / hidden and Softmax layer to recognize the output from the image. A batch size of 16 & 3 epochs we are using. 1 epoch is a complete traversal throughout data and one loop is one forward and one backward propagation on batch size (16 images). This model trains with small strides of 16 images and each time get smarter and more intelligent. As we know every Machine Learning (ML) [20,21] issue begins with the data. For the performance of the system, the quantity and nature of the data are very crucial and most of the time; it needs excessive deal exertions and resources.
Testing accuracy is a prediction that demonstrates the precision and accuracy of any selected deep learning model. The outcome of the testing and training model is shown below. " Fig. 9(a)" shows the accuracy with batch size 16 & " Fig. 9(b)" shows the accuracy with batch size 32. Using  4) F1-score: As denoted in equation (6), F1-score is a total degree of correctness that chains the precision and recall. F1-score is the twice of the fraction between the multiplication to the sum of recall and precision metrics. 307 | P a g e www.ijacsa.thesai.org  Table I gives the information about the performance of the model in terms of accuracy, precision, recall, and F1-scores respectively. VGG-16 has a total number of 138 million parameters. The significant point is that all convolutional kernels are of fixed size 3 X 3 and max-pool kernels are of size 2 X 2 with a stride of two. VGG-16 [22] has a fixed size of convolutions that's why it takes less training time & predicts high accuracy.
V. CONCLUSION A full imaging system has been established to get NDVI & Growth Stage information of the onion crop using an Unmanned Aerial Vehicle. A pair of cameras is used, one of both has been modified to capture NIR images and the other captures the RGB images. If images taken are not corrected, then so image sharpening techniques have been used to clarify the image and we have got accurate results. In upcoming years, it is probably expected to use a fixed-wing aircraft in place of a UAV to cover the large areas of the crop fields. Also, another camera could be used to capture red edge radiation and calculate the Normalized Difference Red Edge (NDRE). Different outcomes are achieved as shown in Table I, due to variation of some parameters. From the present research, the interpretations accomplished verified that the recommended method is capable and can be more applied to the multi-step on the estimate of a various group of disease factors of the onion crop.