Olive Oil Ripping Time Prediction Model based on Image Processing and Neural Network

The agriculture sector in Jordan depends very much on planting the olive trees. More than ten million of olive trees are planted in the Jordanian soil. Olive fruit are harvested for two purposes; either to produce oil or to produce olive table (pickled olive). Olive fruit harvesting time for extracting the oil from the olive fruit is crucial. Hence, harvesting the olive fruit on ripping time gives the best amount and quality of oil. It also, could lose 15% to 20% of multiple values because of harvesting time. Olive fruit ripping time is varied since it depends on the rainfall, temperature and cultivation. A system to predict the optimal time for harvesting olive fruit for producing oil only is introduced. It based one Digital Image Processing (DIP) and artificial intelligent neural network. Moreover, four features were extracted from the olive fruit image based on the red, green and blue colors. The proposed system tested olive fruits in three stages of ripping time; under ripping, on ripping and over ripping. The classification accuracy achieved in the three stages was 97.51% in under ripping stage 95.10% in ripping stage, and 96.12% in over ripping stage. The proposed system performance was 96.14%. Keywords—Neural network; image processing; olive ripping time; prediction; classifications


I. INTRODUCTION
Olive tree is the most important tree in Jordan. It represents 70% of the fruit tree in the country [1]. Approximately 560,000 dunums (560,000,000 m2) are planted with more than 10.5 million of olive trees, which represent 72.00% of the agricultures land in the kingdom of Jordan [2]. There are three common kinds of olive trees in Jordan; Nabali olive with high oil content between 28% and 33%. Rasi olive with percentage of oil varies from 15% to 28% and Sori olive for olive table. Furthermore, the investment in the olive sector passed one Billion Jordanian Dinar (JD) in 2018. Also more than 80,000 Jordanian families have benefit from this sector. According to an Agriculture Ministry of Jordan, the olive oil production increases from year to year, where 24,000 tons of olive oil were produced in 2018 with average price varied between 134,880,000 JD to 142,500,000 JD depending on the amount of produced oil in the season and the type of the olive oil. The amount and the quality of olive oil depend on rainfalls, cultivation and the harvesting time. By harvesting the olive trees on the ripping time, the best amount and quality of olive oil can be achieved [3]. Jordan loses 15% to 20% of the amount of olive oil because of harvesting time. By either harvesting the olive fruits before or after the suitable harvesting time [2]. The olive oil characteristics have its own rank among all kinds of oils consumed by human being because of its pleasant flavor, plat ability, stability and health benefits. Harvesting the olive tree before or after ripping time, reflects the quality grade of oil characteristics, oxidation, stability and nasturtium value of the obtained product [4]. The majority of olive oil (94%) does not have the best commercial quality, because the olive fruit is not picked on the optimal harvesting time [5]. According to [6] harvesting olive fruits must be done when the oil reaches the best quality and the highest amount of oil level [3,7]. In the Harvesting olive fruit season, the farmers attempt to estimate ripping time based on the color, texture, size and shape of the fruits. This estimation can be done by walking through the olive tree field. Factors such as the size and the shape varied depending on the amount of the rainfall, dryness and cultivation during the year. Also, factors like the color and texture of the fruit are subjective and depend on the farmer experiences. Therefore, farmers need to have additional opinion to determine the harvesting time precisely. As an additional authenticates opinion researchers have proposed different methods like analytical static [7] and intelligent system based on Digital Image Processing (DIP) and Artificial Intelligent (AI) and indeed Artificial Neural Networks (ANN). For instance, in the medical field DIP and ANN were used to classify image breast cancer and detecting brain tumor [8,9] also new model of neural network was proposed to classify six types of disease; the proposed model has shown very high performance. A convolutional neural network was used in face image manipulation detection [10,11]. DIP and ANN have shown great potential in managing road traffic system as in [12][13][14]. In addition, ANN have been widely used to predict the landslide hazard map [15]. Moreover, as a tool of classification ANN and DIP have been widely used in different fields as mentioned in [16][17][18][19][20]. This paper presents a model to determine the optimal time for harvesting olive fruits for oil. The new model relies on Digital Image Processing and Neural Network.
In this section, the aim of this study and general back ground for the proposed system was introduced. In Section 2, the related previous work that used Image Processing and ANN is summarized. Section 3 the data collection and the structure of Cascade Forward Neural Network (CFNN) were explained. Section 4, Image Preprocessing, Section 5, Feature Extraction, Section 6, Classification Model, Section 7, Experimental Methodology, Section 8, Results and discussions and Conclusion in Section 9.

II. RELATED WORKS
Maturities, ripeness, quality and diseases of the fruits have been subject of interest for many studies and included www.ijacsa.thesai.org different types of fruits such as Grape, Apple, Tomatoes, Cucumber, Orange, Palm oil fruits and many other fruits [21][22][23][24][25]. Studies on Olive fruit can be divided into classification the quality of olive fruits for olive table (pickled olive) [26,27], external damage and diseases of olive fruit [28,29] and olive fruit ripping time for olive oil [30][31][32].
To determine the stage of maturity of the olive fruit for oil, various methods were applied [29]. Maturity index (MI) is the common traditional used method to determine the optimal time for harvesting olive fruits for oil. To determine the MI, one KG of olive fruit must be collected randomly from several trees with same verity. Then, 100 olives fruit picked up randomly from 1 KG. Each 100 fruit sample separated into different eight categories of color starting from "0" deep green to "7" purple or black color. Multiply the number of fruits in each color category by the number of that color category (0 to 7) and add up all the numbers and the result divide by 100. When the MI value is 5 it's the optimum time to harvest the olives. The MI method is not accurate since it depends on the human judgment. The work in [33] approves the machine more accurate than the human on the classifying olive table. Thus, finding an automatic system to determine the optimal harvesting olive fruits time has become subject of interest for many studies. In this section, the automatic system based on DIP and ANN will be reviewed.
An automatic model for Ripping Index (RI) based on DIP and ANN introduced by [30]. One KG of olive fruits samples were collected randomly and classified to three classes, then analyzed to measure some chemical parameters; oil, sugar and total phenolic content in the olive fruits. The chemical parameters were compared with color of the fruits. The three chemical inputs and the fruit color were used as input to Multi-Layer Perceptron. The results prove that the proposed system has better results than the traditional used RI.
A study introduced on Arbquina olive fruit type. It aimed to classify the maturity of the olive fruits based on changing the color during the fruits maturity stages; green, yellow green, turning, purple, and black. The study highly recommended using the color of olive fruit as maturity index [34].
A color scale was used to determine the maturity of olive and grape fruits. The color scale generated using the wellknown support vector machine based on the multidimensional regression. 250 and 200 samples respectively of grape and olive fruit were used for experimental purposes. The developed color scale can be used in various phenolic states on the fruit [21].
Image analysis was used to determine the maturity of the olive fruits. The system relied on used image segmentations include edge detections and the histogram values for the red, green and blue colors values were calculated. The olive fruit was classified to five categories based on fruit skin color; bright green, greenish yellow, greenish yellow with red black or reddish spots greater than reddish-brown, greenish-yellow with black and/or reddish spots less than reddish-brown and 100% blackish-purple or black [35].
Olive ripping phase was estimated to six phases based on the fruit skin color. The skin color histogram was manipulated in the feature selection phase, and Multi-Layer Perceptron neural network as a classifier. The obtained results showed promising results [36].
A technic was proposed to determine the olive oil based on olive fruit ripening stage using the DIP. The method start by acquiring the olive fruit image then image processing features extraction, features selection and finally the regression mode for classification the oil quality [37].
The quality of the olive oil strongly related to the quality of the collected fruits. A mathematical model based on partial least squares Discriminant Analysis (PLD), and image processing with neural network introduced to classify four type of olive fruits (Mission, Manzanillo, Ascolano, and Sevillan) to four grades (unacceptable, borderline, acceptable, and optimal). 75% and 70% correct classification rates offered by PLD , whereas 93.0% and 90.0% classification rate offered by DIP and neural network [38]. Classifying the olive fruit into different categories are done manually before the oil extraction process. Therefore, an online system that automatically cans categories the olive fruits to their quality level in different class. The study, focused on the olive fruits picked up from the tree or from the ground. In this research, the online system constructed of DIP (image entropy, grey level co-occurrence matrix and statistics like Contrast, Correlation, Energy and Homogeneity) and for classification ANN and Support Vector Machine (SVM) with achieved calcification accuracy 98.8% and 98.4% for the former and later respectively [31].

III. METHODOLOGY
The methodology of this research can be divided into: collecting olive fruit images, image processing, features extraction and classification as shown in Fig. 1. 505 | P a g e www.ijacsa.thesai.org

A. Olive Samples
The data of the olive fruit were collected during the season 2010 from Rasi olive tree farm in Hawara, Irbid, Jordan. The data were collected between August and December olive fruit color, size and shape change faster than rest of the year. Therefore Olive fruit was randomly picked up and images were taken. Fig. 2 shows the differences between the olive fruit color between August and December. High temperature and dry weather speed up the change of olive fruit color and viceversa, as mentioned the Sori olive is the subject for this study.

B. Acquisition of Images
Olive fruit images were taken at the farm, the images where taken in day light. Fifty images were taken every week from August to December with totally 1200 images, only ten suitable images were chosen for further process with totally 240 images. The Image acquisition system consists of digital camera Canon EOS 7D, 18 Megapixels, and white sheet of paper was setup in the farm. The deformable durable material was chosen in an attempt to approximate the type that would be available in regular farmers' hands.
1) Image pre-processing: In order to have a proper image for extracting information, all images must be with a standard size; thus, olive image were scaled downed to 50% of its original size using Image Processing Tool Box in MATLAB Released 2013a. Then, filtering the image and remove any noise. Finally, removing the white back ground from the image, in order to have only the olive fruit as shown in Fig. 3.  2) Feature extraction: The aim of feature extraction is to achieve some features, which can be used in the classification system. In most cases the color of the fruit is the strongest indication for the fruit maturity. The color of the olive fruits starts changing gradually from dark green to dark black. The image color component Red, Green and Blue (RGB) was used to calculate the mean, median and Standard Deviation (SD) of RGB colors.

IV. CLASSIFICATIONS MODEL
Cascade forward neural network (CFNN) is one of the artificial neural network types. It construct of input, hidden and output layer. Where, input layer connected to one or more hidden layer which also connected to output layer. CFNN also has a connection between the input layer and every layer in the neural network. In addition CFNN also has another connection from each layer to the successive layers in the network Demuth [39]. CFNN has the same principle of work and methodology that multi-layer feed forward neural network do, but the extra two connections [40]. The later connections are introduced to improve the neural network generalizations [41]. Fig. 4. Shows topology of CFNN with three layers; input, hidden and output layer. Eq. (2) shows the total output of the hidden layer at time k ( ) . Where ( ) is the linear or nonlinear of hidden layer output function.
The output of one hidden layer of CFNN can be found as in Eq. (3) Where, ( ) is the outputs of CFNN, and n =1,2,...,N. is the output neurons.
( )is the linear or nonlinear output function of output layer. The connection weight between the nodes in the hidden layer and output layer represented by W 2 jn .

V. EXPERIMENT METHODOLOGY
The experimental process was divided into; preparing the data that will be used for CFNN, the structures of the CFNN and the CFNN performance measurement. Preparing the olive fruit data set includes: normalizing all the features that extracted from olive fruit such RGB color. The later has three different values namely mean, mean Standard Deviation (SD), median and median SD. For each feature the normalization can be varied between 0 and 1 based on the Eq. (7).
Where the value ( ) is the value to be normalized and ( ) is the minimum or the maximum pixels value for every single feature. The intelligent system targets are divided into three classes; before ripping time, on ripping time and over ripping time are represented by 0, 1 and 2, respectively. Moreover, dividing the normalized data before entering the CFNN is a crucial to avoid over fitting during the training process of the CFNN therefore; the normalized data was randomized and then divided to three different sets, 70%, 15% and 15% for training, testing and validation, respectively.
As mentioned, the CFNN with one input, one hidden and output layer structure can solve most of the classifications problems. The number of the neurons in the hidden layer has a high impact on the CFNN performance. For this research, the number of the hidden neurons was tested from 5 to 50 neurons, by increasing the number of the neurons by 5 each time and the accuracy was measured. CFNN with 9 neuron in the hidden layer was found to be the optimal for this study. CFNN Over fitting is another common errors that facing neural network during the training. To overcome this problem, the neural network will stop after 500 epoch, If the performance the neural network could not reach the 0.01 as Mean Square Error (MSE).
The performance of the CFNN classification accuracy was measured during the training and testing stage. The accuracy was calculated by using the Eq. (8). In this section and following subsections, the feature extractions and CFNN performance for the olive fruit maturity stages; under ripping time, on ripping time and over ripping time will be evaluated.

A. Under Ripping Stage
The olive fruit color, in under ripping stage, varied between the dark green and yellowish. In this stage the average space of the red color was 66.83% and the Mean SD was 2.56%, while the median and SD median for the red color was 66.71% and 4.50% respectively. The values for green color were 72.30%, 2.08% and 72.15%, 1.50% for the mean, SD mean, median and SD median respectively. Moreover, 1.50%, 40.78%, 3.01%, 39.8% and 2.82% were in orders the values of mean blue, means SD, median blue and median SD blue value. For this stage the red color is considered green color [42] as shown in Table I.

B. On Ripping Stage
The olive fruit color on ripping stage is varied between blackish to purple. The red color values for the mean, mean SD, median and median SD in were in ordered 59.71%, 2.5%, 60.12% and 3.08%. The average space of the green was 40.20% and the Mean SD was 1.90%, while the median and SD median for the red color was 38.99% and 2.21% respectively. The values for blue color were 35.90%, 3.56% and 36.79%, 3.31% for the mean, SD mean, median and SD median, respectively as shown in Table II. 98.58% of the olive was correctly classified for training and 95.10% of olive fruits were correctly classified in the testing stage as can be noticed from Fig. 6.

C. Over Ripping Stage
The olive fruit color for over ripping stage is dark black. The values of red, green and blue color as seen from Table III were Mean red 23.65%, mean SD 1.67%, Median red 23.63% and median SD 1.71%. The green color values for the mean, mean SD, median and median SD in were in ordered 21.8%, 1.34%, 21.90% and 1.33%. The values for blue color were 22.23%, 1.11% and 21.97%, 2.10% for the mean, SD mean, median and SD median respectively as shown in Table III.  In Fig. 7, it can be clearly seen that the CFNN achieved 97.00% of accuracy through the training stage and 96.12% accuracy through the testing stage.

VII. CONCLUSION
This work introduced a system to predict the optimal time for harvesting olive fruits for extracting oil. The methodology of this work was started by taking pictures for the Rasi olive in different period of time. Then extracting features from olive fruits RGB color. CFNN neural network was used to classify the olive fruits to three levels; under ripe, on ripe and over ripe. In this study an artificial intelligent system was introduced for decision support task of harvesting olive fruits for extracting oil, where the amount and the quality of the olive are increased. Future work should be apple to predict the olive ripping time in the real time.