Virtual Calibration of Cosmic Ray Sensor: Using Supervised Ensemble Machine Learning

In this paper an ensemble of supervised machine learning methods has been investigated to virtually and dynamically calibrate the cosmic ray sensors measuring area wise bulk soil moisture. Main focus of this study was to find an alternative to the currently available field calibration method; based on expensive and time consuming soil sample collection methodology. Data from the Australian Water Availability Project (AWAP) database was used as independent soil moisture ground truth and results were compared against the conventionally estimated soil moisture using a Hydroinnova CRS-1000 cosmic ray probe deployed in Tullochgorum, Australia. Prediction performance of a complementary ensemble of four supervised estimators, namely Sugano type Adaptive Neuro-Fuzzy Inference System (S-ANFIS), Cascade Forward Neural Network (CFNN), Elman Neural Network (ENN) and Learning Vector Quantization Neural Network (LVQN) was evaluated using training and testing paradigms. An AWAP trained ensemble of four estimators was able to predict bulk soil moisture directly from cosmic ray neutron counts with 94.4% as best accuracy. The ensemble approach outperformed the individual performances from these networks. This result proved that an ensemble machine learning based paradigm could be a valuable alternative data driven calibration method for cosmic ray sensors against the current expensive and hydrological assumption based field calibration method.


I. COSMOZ DATA AND SYSTEM
The Australian Cosmic Ray Sensor Soil Moisture Monitoring Network (CosmOz) (Figure 1) [1-2] is a near-real time continental scale soil moisture monitoring system originally inspired by the United States Cosmic-ray Soil Moisture Observing System (COSMOS) [3][4][5].CosmOz aims to test the utility of Hydroinnova CRS-1000 cosmic ray soil moisture probes [3] (Figure 2) for water management, water information, hydrological process research applications and test the feasibility and utility of a national near-real time soil moisture measurement network.
The cosmic ray soil moisture probe measures the neutrons released when cosmic rays interact with hydrogen atoms in water molecules found in the soil and atmosphere.The number of fast neutrons emitted into the atmosphere is inversely correlated with soil moisture.Figure 3 shows the fundamental principal behind these cosmic ray probes [5][6].
Data from the Hydroinnova CRS-1000 cosmic ray soil moisture probe deployed in the Tullochgorum site in Tasmania, Australia was used for this study.It consists of two neutron detectors: a bare detector that responds mainly to thermal neutrons and a polyethylene-shielded detector that responds mainly to epithermal-fast neutrons.Each counter has its own high-voltage power supply and a pulse module to analyse the signal generated by the neutron detector tube.An Iridium satellite modem then transmits the data at one hour time intervals to the CosmOz CSIRO data server [7][8].Tullochgorum CosmOz cosmic ray sensors were calibrated using soil samples collected around the probe.Soil moisture was measured using the oven-drying method, and area-average soil moisture was computed for all samples [9].Finally the average soil moisture content within the footprint of the probe was used to convert neutron counts into soil moisture [10][11].Figure 4 shows the rainfall, pressure corrected neutron count profile, and conventionally estimated soil moisture profile for the period chosen for this study (July 2011 to May 2013) at the Tullochgorum site.
The Intelligent Sensing and Systems Laboratory, CSIRO CCI and the Tasmanian node of the Australian Centre for Broadband Innovation are assisted by a grant from the Tasmanian Government which is administered by the Tasmanian Department of Economic Development, Tourism and the Arts.www.ijacsa.thesai.org

II. CALIBRATION PROBLEM SPACE
The current field calibration method for cosmic ray sensors is significantly time consuming and expensive.For the current method, it is also essential to conduct at least two field calibrations (dry calibration during summer and wet calibration during winter) to complete the minimum requirement for of the calibration process.Ideally monthly calibration is required, which makes the current calibration process very expensive and impractical.Within the current calibration method, corrective equations are presently used to nullify the effects of universal cosmic radiation, lattice water in soil, pressure and humidity.However, there are uncertainties and questions about the effectiveness of these calibration equations in order to produce real-time accurate soil moisture estimations.The process is prone to human error as it is based on limited experimental data, hard manual field sampling protocol, and theoretical assumptions [1][2][3][4][5][6][7][8][9].The main focus of this study was to investigate the possibilities of a complete data driven virtual sensor calibration approach which could be well suited for this purpose.We proposed an ensemble machine learning based alternative method to capture the behavioural aspect of the neutron count observations compared with the real soil moisture measurements.The aim was to develop a virtual calibration method, cross validated against an external ground truth data source, independent of CosmOz network.The Australian Water Availability Project (AWAP) data base [12] has been used for this purpose.

III. EXPERIMENTAL DATA SETS DESIGN
The AWAP soil moisture data base [12] was used as an external data source, independent of CosmOz network to develop the virtual calibration method [2].The AWAP database monitors the state and trend of the terrestrial water balance of the Australian continent, using model-data fusion methods to combine both measurements and modeling.The AWAP database provides 16 environmental attributes from which Radiation (MJ/m2), Max Temperature (degC), Min Temperature (degC), Rainfall (mm), Soil Evaporation (mm), Local Discharge (Runoff+Drainage) (mm), Surface Runoff (mm), Open Water Evaporation ('pan' equiv) (mm), Deep Drainage (mm), Sensible Heat Flux (MJ/m2) and Latent Heat Flux (MJ/m2).These attributes were used as part of the training and testing input for the machine learning algorithms.Selection of these inputs from the whole AWAP data set was based on expert domain knowledge and principal component analysis (PCA) as feature selection method, where eleven least correlated attributes (covering 99% of data variance) were selected for the experimental design.Figure 5 shows the training and testing inputs from AWAP.The pressure corrected fast neutron count time series from CosmOz was also considered as part of the whole training and testing input sets (Figure 4).On the other hand Upper Layer Soil Moisture (%) and Lower Layer Soil Moisture (%) were used as the training target for the data experiments (Figure 5).An AWAP data adaptor was developed and used to download and unzip all www.ijacsa.thesai.orgAWAP folders, process the sequential NetCDF gridded data files, and to extract all the time series.Various training and testing data sets were formed based on a randomized incremental optimization algorithm to evaluate the generalization capability of the proposed individual and ensemble machine learning architectures.Combinations of % training data and % testing data were varied from {10%-90%} to {50%-50%} to identify the best possible training-testing data balance to achieve maximum prediction accuracy with highest possible sensitivity and specificity.[22][23] and Learning Vector Quantization Neural Network (LVQN) [24][25] were selected for this study.In previous work related to this topic S-ANFIS was used for soil moisture estimations [1].

A. Rationale
The main rationale behind selecting these four supervised estimators was to conduct a comparative study on significantly varied neural network architectures predicting soil moisture based on the same fast neutron counts to identify a better architecture for this calibration purpose [26][27].In the later stage of this paper, an ensemble approach has been proposed for better soil moisture estimation and dynamic cosmic ray probe calibration.In an ensemble approach selection of widely varied supervised estimators was another essential rationale to achieve much better prediction generalization and higher calibration accuracy.Ultimate goal was to have a parallel processing of the neutron count data using multiple ANNs to capture significant data behavioural variance in relation to the soil moisture and also in the predicted time series; so that ensemble generalization can perform better than one individual ANN, hence they could complement each other on a dynamic range.The MATLAB programming environment was used to train and test these estimators.Individual performances were evaluated based on the common training and testing paradigms at any given time.

B. Performance Evaluation
Performance assessment was conducted using a point to point comparative study between the soil moisture time series output from the trained estimators during testing and the soil moisture time series output obtained from the existing field calibration methodology.Higher percentage similarity between these two time series was essential to justify the effectiveness of the individual estimator based alternative method before further improvement could be achieved.Each point on the both time series was representing a single day, so point to point comparison provided a daily comparison.
A predictive performance estimation mechanism based on time series cross correlation and auto correlation was applied to measure percentage accuracies of the predictions.High correlation between expected soil moisture profile and the predicted one represented better prediction performances.Finally, performances of these estimators were quantified by prediction accuracy ((TP + TN) / (TP + FN + FP + TN) where true positives =TP, true negatives =TN, false positives = FP, false negatives = FN).The evaluation process also included sensitivity (TP / (TP + FN)); specificity (TN / (FP + TN) calculations to justify the estimation correctness.As in hydrology ± 3% tolerance limit is acceptable in soil moisture measurements, point to point comparison between two time series provided us TP, TN, FP, and FN estimates [4][5][6].

C. S-ANFIS Estimator
S-ANFIS is a neural network method based on the Takagi-Sugeno fuzzy inference system.Since it integrates both neural networks and fuzzy logic principles, it has the potential to capture the benefits of both in a single framework [17].S-ANFIS uses only differentiable functions thus standard learning procedures from neural network theory can easily be used.The parameters are propagated again, and in this epoch back-propagation is used to modify the antecedent parameters or the membership functions, while the consequent parameters remain fixed [18].The generation of the rule base is unsupervised followed by supervised learning to update the rule parameters.In this study the supervised part of the S-ANFIS estimator was a multi-layered perceptron network (MLPN).A sigmoid activation function in the form of a hyperbolic tangent has been used in this estimator.

D. CFNN Estimator
CFNN is similar to feed-forward networks, but include a connection from the input and every previous layer to the www.ijacsa.thesai.orgfollowing layers.As with feed-forward networks, a two-or more layered cascade-network can learn any finite input-output relationship arbitrarily and predict time series sequences well; provided it has enough hidden neurons [17, [19][20][21].Sigmoid activation function with normalization between -1 and 1 has been used.

E. ENN Estimator
The ENN is a simple recurrent neural network consisting of an input layer, a hidden layer, and an output layer.In this way it resembles a three layer feedforward neural network.Elman neural networks are very useful for predicting time series sequences, since they have a limited short-term memory [17, [22][23].Short-term memory provides a unique capability to the ENN, by storing the previous learning step which could be used to influence the next learning step.At each time step, the input is propagated in a standard feed-forward fashion, and then a learning rule is applied.The fixed back connections result in the context units always maintaining a copy of the previous values of the hidden units (since they propagate over the connections before the learning rule is applied).Thus the network can maintain a sort of state, allowing it to perform such tasks as sequence-prediction that is beyond the power of a standard multilayer perceptron.

F. LVQN Estimator
A LVQNN consists of two layers competitive layer and linear layer.The first layer maps input vectors into clusters that are found by the network during training.The second layer merges groups of first layer clusters into the classes defined by the target data.The total number of first layer clusters is determined by the number of hidden neurons.The larger the hidden layer the more clusters the first layer can learn, and the more complex mapping of input to target classes can be made [17, [24][25].

V. GENERALISATION RESULTS AND DISCUSSION
The CFNN network required typically 2,000 training iterations, ENN required only 750 training iterations and LVQNN needed only 1400 training iterations.The CFNN (with learning rate equal to 0.42 and a momentum term equal to 0.5) with eleven inputs, ten hidden and one output neuron was able to reach a best success rate of 84.3% (rates varied between 67% -84.3% for various training-testing paradigms as describe in section 3) in correct soil moisture prediction while using {75% training -25% testing} paradigm.The ENN with same architecture (with an additional recurrent layer with tap delay 1:5, where hidden later size was 50, and 'trainlm' as training function) was able to reach a best success rate of 86% (rate varied between 75% -86%) while using {70% training -30% testing} paradigm.In the LVQNN, neurons were added to the network until the sum-squared error (SSE) falls beneath an error goal (0.001), or a maximum number (172) of internal neurons was reached.It was important that the spread parameter was large enough so that the hidden neurons respond to overlapping regions of the input space, but not so large that all the neurons respond in essentially the same manner.The spread parameter was set to 0.79.The LVQNN was able to achieve a maximum of 80% prediction accuracy using {65% training -35% testing} paradigm.S-ANFIS training was fastest among all of the estimators.S-ANFIS was able to predict bulk soil moisture at an accuracy of 87% while using {80% training -20% testing} paradigm (accuracy varied from 72% -87%).S-ANFIS was the overall best performer compared to the other three estimators whereas LVQNN was able achieves maximum prediction accuracy with least amount of data being used from training.TABLE 1 summarizes all the generalization results using all four SML estimators in terms of correct percentages of soil moisture prediction.Results are presented for four different experimental scenarios with different combination of training-testing, where individual supervised estimator had maximum prediction accuracy.It was evident that although prediction accuracies were significantly high, but there was no best architecture as individual estimator's performance sensitivity and specificity were quite mixed with no clear winner.Ultimately supervised neural network based alternative virtual calibration could only be useful if a better prediction accuracy along with higher sensitivity and specificity could be achieved than what is recorded in the At this point it was a natural progression to apply an ensemble learning paradigm where several supervised estimators could be jointly used to calibrate cosmic ray sensors.In the next section two layered ensemble approach was applied to combine and complement these four estimators in order to explore any possible improvement in soil moisture estimation and virtual calibration of these cosmic ray sensors.www.ijacsa.thesai.org

A. Two Layered Ensemble Methodology
Two layered ensemble approach has been proposed and developed in this study to explore the possibility of overall improvement.In the first layer global training was performed on all four supervised estimators (S-ANFIS, CFNN, ENN and LVQNN) using a common training and target set.Once trained all the estimators were simulated (tested) using the same training inputs, but without the targets to generate four individual predictions.The next layer of this proposed approach was constructed using a self-organizing map network (SOMN) and weighted averaging block.A self-organizing map (SOMN) is a type of artificial neural network that is trained using unsupervised learning to produce a low-dimensional (typically twodimensional), discretized representation of the input space of the training samples, and called a map.SOMN may be considered a nonlinear generalization of PCA, where cross correlation among various inputs get projected on an empirical orthogonal plane based proportion of data variance captured along different components.SOMN was selected to decorrelate the predictions from the ANNs as it has more generalization capability to cover data variance than PCA or any cross correlation based methods, so less amount of data variance remains unexplained [33][34].Individual ANN is providing single dimensional prediction.Four predicted time series from four different estimators are combined as inputs to the SOM.
Once trained initial SOM grid (the 100 neurons with network size 10 X 10) was re-distributed among the data points, distributed concentration of the trained neurons formed natural separated clusters.Two clusters were highly correlated if they were comparatively closely positioned -hence they were selected.Least correlated ones were thrown out.An objective function based on intra cluster distance measure was used to determine the cluster correlations (Figure 7).SOM decorrelate these inputs to create individual pattern maps (and clusters) for individual input time series based on training weights (Fig. 8).Based on the SOMN natural clustering (or natural grouping of the predicted values) on the predictions from the layer 1 and the training targets, highly correlated ones were selected as they had statistically similar fluctuations.Euclidian distance among the SOM clusters were used to define the number of similar clusters hence highly correlated.Data points which belonged to the similarly positioned clusters were then marked and traced back to flag the corresponding original supervised estimators together as highly correlated.Data points belong to those selected clusters www.ijacsa.thesai.orgwere then traced back to the corresponding ANN methods, and predictions from those methods were selected for averaging to form the final one dimensional prediction.Initial prediction values proposition in an initialized SOMN are depicted in Figure 8.The average of the output of several different models could be called as an ensemble model which will take the form of The idea of averaging different outputs from different models was developed in the neural network community [28][29].Later it was also established by Krogh et al. [30] that the generalization error of the ensemble could be lower than the mean of the generalization error of the single ensemble members.It has also been reported in the literature that the generalization error of an ensemble model could be improved if the predictors on which averaging is done disagree and if their fluctuations are uncorrelated [31][32].Development of the ensemble approach was primarily focused on establishing a mechanism where decomposition of several predictions could be done in order to establish the highly correlated ones and throw away the uncorrelated ones.
Selected predictions were then averaged to form the final prediction.Sorted corresponding cross correlation coefficients were used to perform the weighted average.Based on the SOM weight maps correlation coefficient between [-1 and +1] were generated.Ensembles with higher coefficients are given higher weighting in the prediction.This additional processing was used to refine the final outcome in more realistic way.First two highly correlated predictions were used to perform the weighted average to form the final outcome.Based on the nature of dynamically available time series different estimators were selected based on SOM based de-correlation and selection processes.

B. Ensemble Performance Evaluation
Performance of this newly proposed ensemble architecture was evaluated using rigorous testing paradigms.As described in section 3 amount of data used in testing was varied from 10% to 50%.Evaluation of the generalization performance concluded that the best bulk soil moisture prediction was achieved while 30% data was used for testing with 70% being used for training.Selected ANNs during ensemble training phase were then used in the testing phase as it was obvious that they had the best generalization capabilities for that particular training-testing paradigm instance.Accuracies were calculated based on the final testing prediction and the ground truth testing targets.Overall accuracy was 94.4% with 91% sensitivity, 90% specificity, 2% false positive and 95% true negative.Results based on ensemble show a clear improvement from the prediction point of view, which also proved the intended effectiveness of the proposed ensemble approach.Figure 9 shows the prediction performance based on ensemble approach.

C. Ensemble Calibration Evaluation
Based on ensemble performance evaluation results it was evident that ensemble approach could be able to provide a unique platform for calibrating cosmic ray sensors on a dynamic basis.Two layered ensemble approach could be used as frequently as possible to train and capture new neutron counts data to estimate soil moisture profile.Higher performance accuracy with very low false positive results during testing phase shows the high level of reliability on this approach.Ensemble based approach is a virtual approach with high degree of flexibility which offers high frequency remote sensor calibration compared to the expensive field soil sampling method.This dynamic machine learning based virtual calibration could be a benchmarking methodology for calibrating cosmic ray sensors measuring area wise bulk soil moisture.

VII. CONCLUSION
This study concluded the ensemble of supervised machine learning algorithms could be an effective alternative calibration method for remote area wise estimation of bulk soil moisture using the cosmic ray sensor's fast neutron count readings.Using the AWAP database it was possible to train the ensemble supervised estimator with historical ground truth soil moisture data, which provided better generalization capability to predict accurate soil moisture from the cosmic neutron counts.Prediction results were very encouraging.Potentially this could help us to develop a web based remote virtual sensor calibration mechanism.This way the cosmic ray sensor could be monitored and calibrated virtually and continuously.

Fig. 3 .
Fig. 3. Schematic flow diagram of the fundamental principal behind this cosmic ray probes.

Fig. 4 .
Fig. 4. CosmOz Data from the period July 2011 -May 2013 were used for testing and validation for this research study, (a) Daily rain fall, (b) Pressure corrected neutron counts, (c) Estimated bulk soil moisture profile for Tullochgorum siteconverted from pressure corrected neutron count data generated by the Hydroinnova CRS-1000 cosmic ray soil moisture probe..

Fig. 5 .
Fig. 5. Individual time series were extracted from the AWAP gridded maps based on pixel values corresponding to the latitude-longitude information of Tullochgorum site in Tasmania, Australia.Various training and testing data sets were formed based on a randomized incrementing optimization algorithm.IV.SUPERVISED ESTIMATORS Four supervised estimators, namely Sugano type Adaptive Neuro Fuzzy Inference System (S-ANFIS) [17-18], Cascade Forward Neural Network (CFNN) [19-21], Elman Neural Network (ENN)[22][23] and Learning Vector Quantization Neural Network (LVQN)[24][25] were selected for this study.In previous work related to this topic S-ANFIS was used for soil moisture estimations[1].

Fig. 6 .
Fig. 6.(a) The schematic design for the layer 1 of the proposed ensemble supervised machine learning approach for cosmic ray sensor calibration, where individual supervised estimators are being trained in parallel; (b) Second layer of the ensemble approach, where SOMN is applied to decorrelate the training predictions against the training targets to select the highly correlated ones, and throw away the least correlated predictions.

Fig. 7 .
Fig. 7.The representation of initial cluster positioning of the predicted time series from the four different supervised estimators.SOMN was applied to decorrelate these clusters to select fewer highly correlated clusters (typically first two in most of the cases).

Fig. 8 .
Fig. 8. Example of trained SOMN based de-correlated individual weight maps for the four supervised estimators.In this instance predictions from ANFIS and LVQNN have been selected for final stage of prediction estimation, while 80% data is used for training.

Fig. 9 .
Fig. 9. Ensemble approach based soil moisture prediction performance, (red curves while blue represents testing target ground truth).