Deep Learning Approach for Forecasting Water Quality in IoT Systems

—Global climate change and water pollution effects have caused many problems to the farmers in ﬁsh/shrimp raising, for example, the shrimps/ﬁshes had early died before harvest. How to monitor and manage quality of the water to help the farmers tackling this problem is very necessary. Water quality monitoring is important when developing IoT systems, especially for aquaculture and ﬁsheries. By monitoring the real-time sensor data indicators (such as indicators of salinity, temperature, pH, and dissolved oxygen - DO) and forecasting them to get early warning, we can manage the quality of the water, thus collecting both quality and quantity in shrimp/ﬁsh raising. In this work, we introduce an architecture with a forecasting model for the IoT systems to monitor water quality in aquaculture and ﬁsheries. Since these indicators are collected every day, they becomes sequential/time series data, we propose to use deep learning with Long-Short Term Memory (LSTM) algorithm for forecasting these indicators. Experimental results on several data sets show that the proposed approach works well and can be applied for the real systems.


I. INTRODUCTION
Global climate change and Global Warming effects have caused many problems to the farmers and the managers. Climate change not only causes great impacts on all regions, resources, environment and socio-economic, but also affects water resources, agriculture and rural development, health and the coastal areas. Economic losses due to the impact of climate change combined with the costs of recovering from the damage reduce the economic growth of many countries and globally. Especially, Agricultural/Aquaculture countries have been suffering great losses because of Global climate change. For example, fishery is one of the key economic sectors of Vietnam's agriculture, with billions of dollars in exports. It also creates jobs for millions of workers, contributing to stabilizing social security and developing the country. Shrimp is a seafood product in the top export of billion US dollars. However, shrimp nursery is too difficult because they are particularly sensitive to water. Polluted water environment is a great concern for numerous fishermen and Agricultural managers. However, these farmers do not have enough technique and media to forecast environment factors, so they cannot manage risks which they are facing. Thus, how to manage and monitor the water quality, especially in aquaculture and fisheries such as fish raising management is really necessary.
Recently, techniques in the Internet of Things (IoT) can help users to build the monitoring systems. IoT refers to the millions or even billions of physical devices around the world currently connected to the internet, collecting and sharing data together. With advancements in internal processors and wireless networks, we now can make everything more proactive and intelligent [1]. There are many applications of IoT such as automatic door systems (smart-home), aircraft and self-driving cars (smart-city), etc. which have become a common part of IoT. This adds a level of digital intelligence to passive devices, allowing us to communicate real-time data without human involvement, effectively integrating the digital and physical world. When something is connected to the internet, it is able to send or receive information, or both. Leveraging IoT techniques, the ability to send or receive this information makes everything smarter, and that is also our target.
This work 1 proposes an architecture for the IoT systems to monitor and forecast the water quality (such as indicators of salinity, temperature, pH, and dissolved oxygen -DO) in aquaculture and fisheries. These data are collected every day and ordered by time, they can be considered as time series or sequential data. Previous works show that deep learning with LSTM (Long-Short Term Memory) could be an appropriate method for time series data [2], thus, we propose deep learning with LSTM models for forecasting these water quality indicators.
The remaining of this study is presented as follows. We exhibit some state-of-the-art approaches in sensor data processing in Section II. Section III introduces the proposed architectures for the forecast problems. In Section IV, we present the data sets used in the experiments and do some investigations, comparisons, explain the experimental results. Finally, we discuss and summarize our work in Section V.

II. RELATED WORK
A complete IoT system relates to several sides such as sensor networks, hardware circuits, energy, etc., and software for analysis. One of important tasks in the environmental monitoring IoT system is the data analyzing and forecasting methods. In this work, we focus on reviewing the literature which relates to the forecasting methods. There are several researches in this field.
The authors in [3] introduced a method to develop a service oriented architecture for the weather information systems which forecast weather using these data mining techniques (e.g., Artificial Neural Network and Decision tree). The study in [4] proposed a deep neural network for weather forecasting with a set of weather-related variables. The study in [5] used recurrent neural networks (RNN) with its Long Short-Term Memory (LSTM) architecture to predict the ambient temperature (TA). The prediction is based on meteorological data (e.g., temperature, humidity) retrieved from IoT stations and send them to the basic station with LoRa protocol. They formulate the TA prediction problem as a time series regression problem. The proposed approach uses two types of hidden layers: LSTM layer and full connected dense layer. Experimental results show that the model work well.
In [6], the water, temperature, pH and DO levels are measured and integrated with aerating and water supply pumps using Arduino. The user could receive information at predetermined intervals on preferred communication. The authors tested on a sample of two days measurements of temperature, pH and DO levels. Results show that with this integration system, farmer need not hire worker at their site, consequently drive down operating costs and improve efficiency.
The authors in [7] introduced an online water quality monitoring system for intensive fish culture in China. Based on historical data, the system forecasts the water quality using artificial neural networks (ANNs) and control the water quality in time to reduce catastrophic losses. Their results demonstrate that multi-parametric, long-distance and online monitoring for water quality information can be accurately acquired and predicted.
The authors in [9] proposed the detection and monitoring of emotion applications. The authors introduce a deep learning approach for emotion classification through an iterative process by adding and removing large number of sensor signals from different modalities. They uses Convolutional Neural Network and Long Short-term Memory Recurrent Neural Network on the raw sensor data. The results show that their proposed approach is effective in human emotion classification when large number of sensors input is utilised and the hybrid models outperform traditional fully connected deep neural network.
In [10], the authors proposed a deep learning and imagebased model for air quality estimation. The model extracts feature information from scene images captured by camera equipment and then classifies them to estimate air quality levels. The authors tested on several methods such as Support Vector Machines and Deep Residual Network on a high-quality outdoor air quality data set. Experimental results show that the proposed method produces more accurate results than other methods.
The authors in [11] used deep-learning technologies in the environmental field to predict the status of pro-environmental consumption. The authors predicted the pro-environmental consumption index based on Google search query data, using a recurrent neural network (RNN) model. They compared the prediction accuracy of the RNN model with that of the ordinary least square and artificial neural network models. The RNN model predicts the pro-environmental consumption index better than any other model.
In [12], the authors presented the Public agencies aiming to enforce environmental regulation have limited resources to achieve their objectives. They demonstrate how machinelearning methods can inform the efficient use of these limited resources while accounting for real-world concerns, such as gaming the system and institutional constraints. They predict the likelihood of a facility failing a water-pollution inspection and propose alternative inspection allocations that would target high-risk facilities.
The authors of [13] developed a web-based system which monitor the pH and salinity values. The workings of the monitoring and control system send data on pH and salinity sent by the sensor and then stored and in database hosting. They said that for the value of a dangerous salt content worth less than 160 and more than 210, if for a pH of less than 6.5 and more than 7.5.
Darmalim et al. in [14] proposed an IoT solution to automatically monitor the environmental factors. The system had five sensors to measure each parameter. It is developed using a Pythonframework with a web application to present information from the IoT device. The work in [16] presented a feasible model for the daily average temperature on the area of Zhengzhou and apply it to weather derivatives pricing. Other researches can be found in [8], [17], [18], [19], [20].
Although many researches have been done in the topic of environmental forecasting, using LSTM for forecasting water quality indicators is not seen recently and this problem still has room for improvements. In this work, we propose an architecture for the IoT systems to monitor water quality (such as indicators of salinity, temperature, pH, and DO). Since these data are collected every day (ordered by time), we propose deep learning with Long-Short Term Memory (LSTM) models for forecasting these time series data.

A. System Architecture
The system architecture is presented in Fig. 1. In this system, we have implemented the sensors for monitoring water quality indicators such as salinity, temperature, pH, and dissolved oxygen (DO) in the fish/shrimp ponds. Data from sensors is daily transferred to a cloud database (in this work, we have used the cloud database provided by the DigitalOcean 2 ). This data is automatically transferred to a server for analyzing and forecasting.
Using this system, the users can visualize all of the indicators via mobile devices or pc/laptop. The system automatically sends the messages to the users (farmers) when the value of an indicator exceeds the limit (threshold) value. Moreover, based on historical data, the system can forecast the values of each indicator for the next dates (or other selected time).
The system infrastructure is implemented by another group in our team. In this paper, our task is to develop a forecasting model which can provide the forecast values to send early warning to the users. Thus, in the rest of this work, we focus on developing the forecasting models, especially Deep Learning with LSTM for forecasting time series data. LSTM (Long Short-Term Memory) is well-known deep learning method which widely-used in forecast models and can be used to avoid the long-term dependency problem [2], [15].
We propose the LSTM architecture as presented in Fig.  2. For testing purposes, the proposed model includes a single LSTM layer with four units followed by a Fully Connected layer using ReLU as the activation function (the leftside of Fig. 2) and another network containing two recurrent hidden states with 16 units for the first LSTM layer and 4 units for the second LSTM layer which we call stacked LSTM (the rightside of Fig. 2). The input to these models are a sequence values (based on timestamp) of each indicator (e.g., salinity, pH, DO, and Temperature) and the output are their forecast values. We implemented the LSTM with Adam optimizer [21] as the optimized function. The learning rate used is initiated at 0.001. In the experiments (will be presented in Section IV-C) we will validate to choose which LSTM model is appropriate for the kind of sensor data.
When we look inside the LSTM (shown in Fig. 3 as an example), its major and important part of a LSTM network is cells that provide a bit of memory to the LSTM so it can remember the past. LSTM usually contains three types of gates including Input Gate, Output Gate and Forget Gate. Gates in LSTM consist of sigmoid activation functions which produce output ranging from 0 to 1. In most of the cases, the value is either 0 or 1. The sigmoid function (Equation 1) is used commonly for gates because we want a gate to give only positive values and should be able to give us a clear cut answer whether, we need to keep a particular feature or we need to discard that feature.
Where: • i t denotes input gate.
• f t reveals output gate.
• o t represents forget gate.
• w m shows weights for the respective gate(m).
• h t−1 denotes the output of the preceded LSTM block at the timestamp t-1.
• x t is the input at the current timestamp.
• b m exhibits the biases of the respective gate(m).
Equation 2 reveals what new information will be stored in the cell state while Equation 4 aims to provide the activation to the final output of the LSTM block at timestamp t. The forget gate at Equation 3 give us the information on which information to throw away from the cell state.

A. Data Sets for the Experiments
In the proposed architecture in Fig. 1, the sensors' implementation is deployed by another group in our team. However, this system is just started and we do not have enough data for training the model. Luckily, there are several published data sets (presented in Table I) which have the structures similar to our data structure, so we have used them to test the proposed forecasting approach.
The first data set is published by Australian Government, which is the sensor data of Temperature, pH and salinity at New South Wales Estuary.
The second data set is published by the BCO-DMO, located at the Woods Hole Oceanographic Institution, US. This data set relates to water quality parameters (e.g., pH, salinity, temperature, and dissolved oxygen -DO) in Tomales Bay, CA (38.148N, 122.898W) which is a monthly time series through sampling every 2 km.
The third data set is published by the University of Melbourne, Australia. This data set relates to an IoT system which provides the Sensor data including of temperature, light level, humidity every 5 minutes at 8 locations from 2014 to 2015.
The fourth and fifth data sets are published by the Government of Canada. These data relate to the Surface temperature and salinity measured along the track of commercial ships, mostly between Montreal (Quebec) and St. John's (Newfoundland) from 1999 to 2018. We have just used the data of 2017 and 2018 for testing.
The sixth data set is published by the European Multidisciplinary Seafloor and water column Observatory. This dataset contains temperature and dissolved oxygen concentrations and the associated sensor raw data acquired between July 2017 and August 2018 on EMSO-Azores observatory by the EGIM.
After pre-processing by removing missing data, the remaining number of observations are presented in the column #Records in Table II.

B. Evaluation Metrics and Baseline
Root mean squared error (RMSE) is used to evaluate the models. The RMSE for n observations is calculated by equation (5).
where, p i is the true value, andp i is the forecasted value.
We have used regression version of the Support Vector Machines (SVM) as a baseline for comparison. In the first group of experiments, a single layer of the LSTM (model in the left side of Fig. 2) is performed to select the number of hidden neurons. Results show that, on these data sets, the RMSE is not much different when using the number of hidden neurons from 4 to 60. An example is presented in Fig. 4. Thus, we have selected four hidden neurons for faster training.  The second group of experiments is performed to choose the number of optimizing epochs. Results show that the RMSE is the best with 10 epochs for small data sets (data set 1 and data set 2) as presented in Fig. 5. However, the number of epochs should be increased for larger data sets (data sets 3,4,5, and 6), i.e., from 60 epochs to 80 epochs as an example in Fig. 6.

C. Experimental Results
In the third group of experiments, we have revised the single LSTM model by adding more layers, which is called stacked LSTM (e.g., model in the right side of Fig. 2). Results show that the RMSE can be improved when using stacked LSTM as presented in Fig. 7. However, the RMSE is not significantly improved by adding more than 2 layers to the LSTM model, thus we have used the model with 2 layers as proposed in the right side of Fig. 2 for the rest experiments. The final results are presented in Table II. There are four considered indicators which include salinity, pH, temperature, and dissolved oxygen (DO). As shown from Table II, most of the cases, the LSTM has lower error (RMSE) than the baseline SVM, thus it can produce better forecasting results. These results show that the forecast values of the LSTM model are nearly approached to the true values, thus the proposed model could be appropriate for sensor data with time series, and it could be applied to the real world. These results also consist with other works in this domain [5], [7], [9], [11].

V. CONCLUSION
In this work, we proposed an IoT system for monitoring water quality in aquaculture and fisheries, especially the model for forecasting the quality indicators such as salinity, pH, temperature, and DO. Experimental results on the considered data sets with four indicators show that the proposed approach can be applied for the real system. By monitoring these realtime indicators and getting early warning, the system can help the farmers to manage the quality of the water, thus collecting both quality and quantity in shrimp/fish raising.
In the future, we will collect the data provided in our system to retrain the model for implementing to the real world. In this work, we deployed a simple LSTM to perform the forecasting tasks. Some sophisticated models should be taken into account to improve the forecast performance in further researches. The LSTM model in this work uses one variable (the indicator) for forecasting, however, by utilizing other features/variables in the dataset to adapt the LSTM to Multi-Variate LSTM forecasting models could be an approach for improving the forecasting performance.

ACKNOWLEDGMENT
This study is funded in part by the Can Tho University Improvement Project VN14-P6 supported by a Japanese ODA loan.