Real-Time Wildfire Detection and Alerting with a Novel Machine Learning Approach A New Systematic Approach of Using Convolutional Neural Network (CNN) to Achieve Higher Accuracy in Automation

—Up until the end of July 2022, there have been over 38k wildfires in the US alone, decimating over 5.6 million acres. Wildfires significantly contribute to carbon emission, which is the root cause of global warming. Research has shown that artificial intelligence already plays a very important role in wildfire management, from detection to remediation. In this investigation a novel machine learning approach has been defined for spot wildfire detection in real time with high accuracy. The research compared and examined two different Convolutional Neural Network (CNN) approaches. The first approach; a novel machine learning method, a model server framework is used to serve convolutional neural network models trained for daytime and nighttime to validate and feed wildfire images sorted by different times of day. In the second approach that has been covered by existing research, one big CNN model is described as training all wildfire images regardless of daytime or nighttime. With the first approach, a higher detection precision of 98% has been achieved, which is almost 8% higher than the result from the second approach. The novel machine learning approach can be integrated with social media channels and available forest response systems via API’s for alerting to create an automated wildfire detection system in real time. This research result can be extended by fine tuning the CNN network model to build wildfire detection systems for different regions and locations. With the rapid development of network coverage such as Starlink and drone surveillance, real time image capturing can be combined with this research to fight the increasing risk of wildfires with real time wildfires detection and alerting in automation.


I. INTRODUCTION
The frequency of wildfires is increasing globally, with wildfires occurring this year in unprecedented locations, such as Europe and Yellowstone and Yosemite in the United States. Wildfires cause great property damage and result in numerous injuries and deaths each year. In 2021, there was a record breaking 58,985 wildfires, which ravaged a total of 7.1 million acres [1]. Compared to the 18,229 wildfires and 1.3 million acres lost in 1983 [1], the year when official record-keeping began, this is a sizable increase of 223% [3]. In 2020, California wildfires emitted more than 91 million metric tons of CO 2 , that is about 25% of the state"s annual fossil fuel emissions and this percentage is forecasted to keep increasing over the next few years [4].
Although wildfires can occur naturally and do provide some beneficial effects like soil nourishment, they need to be controlled in order to mitigate the high levels of CO 2 emission and prevent property loss and casualties. The earlier wildfires can be detected, the better the chance of reducing CO 2 emissions, property damages and life casualties. Fig. 1 breaks down the causes of wildfires [5].
Evidently, from Fig. 1 at least 69% of wildfires stem from human causes, and according to the U.S. Department of Interior, the actual percentage is even closer to 85%. Establishing an automatic wildfire detection and prevention system should be a focal point in reducing the volume of wildfires in the future. With the continuous construction of power lines across the world, a lot of drone investments should be made for surveillance purposes in order to reduce the possibility of man-induced wildfires [7].
To further understand the factors and variables that should be considered for early wildfire detection, In this research paper the top 20 largest California wildfires were examined from information on Inciweb [8], the government"s incident information system that displays all present and past cases of wildfires [9]. Wildfires can happen anytime and there is a clear increase in nighttime wildfire intensity due to global warming. Globally, night wildfires have become 7.2% more intense from 2003 to 2020 [10].  From the above Fig. 2, ϕ N and ϕ D represent the proportion of total fire radiative power detected at nighttime and that detected at daytime respectively. Fig. 2 shows that the increase in nighttime fire activity across the US has been outpacing its daytime counterpart"s [11]. So in this research, the images captured at nighttime and the importance of the corresponding network model were important aspects of the research.
The early detection of wildfires is essential to controlling wildfires expeditiously. Academia and industry have come together to solve this increasingly urgent and important problem. Companies, namely soar.earth and pano.ai etc., use the latest technologies such as real time satellite imagery, drone surveillance, and remote network connectivity with Starlink to perform real time image and video capturing, which is a much more accessible and immediate way to detect wildfires. Currently, wildfire spot detection and response times take too long, and in many cases, even a few seconds are important and precious to contain wildfire damages. An automated wildfire detection and alerting system is needed to notify response systems and the public in real time to provide better containment and prevention measures.
This research concluded the convolutional neural network (CNN) paired with the AI model server framework can rapidly and successfully identify wildfires with high accuracy, even source images from daytime versus those from nighttime carry significantly different characteristics. Additionally, the learning models can be extended to different locations and regions with model fine tuning.

II. BACKGROUND STUDY
Over the years, many research efforts have delved into the application of artificial intelligence, particularly the use of image recognition and deep learning techniques, to the field of early stage wildfire detection and management. University of California San Diego"s WIFIRE center [12] and Piyush and a group of scientists published a review of ML applications to six problem domains: (i) fire detection and mapping; (ii) fire weather and climate change; (iii) fire occurrence, susceptibility and risk; (iv) fire behavior prediction; (v) fire effects (vi) fire management [2].
Other researchers have explored using classification machine learning models with color features combined with texture classification on superpixel regions of still images [13]. The algorithm uses an RGB color model to detect the color of the fire [13]. Researchers have employed Artificial Neural Networks (ANNs) for fire detection [14] and extended ANNs to wireless sensor networks to create a fire detection system [15]. Various ML methods used in fire detection systems include Support Vector Machines (SVM) to automatically detect wildfires from video frames [16], Genetic Algorithm (GA) for multi-objective optimization of a LiDAR-based fire detection system [17], Bayesian Network (BN) in a vision-based early fire detection system [18], Adaptive Neuro Fuzzy Inference System (NFIS) [19], and Kmeans Clustering (KM) and fuzzy logic [20].
In the last few years, Academia and industry have come together to find solutions with wildfire detection. Researchers and scientists have found that approaches based on deep convolutional neural networks (CNN) tend to yield the best results for wildfire detection [21]. Tao proposed training CNN models end to end, from raw pixel values to image classifier outputs, and Sharma recommended using imbalanced datasets as inputs to these networks to simulate real life scenarios [22]. In 2019, an adaptive pooling approach of conventional image processing techniques and convolutional neural networks provided even higher accuracy and reliability [23]. Recently a group of researchers at Shanghai University have used CNN and satellite images for wildfires detection [24].
From the above research papers the following conclusions can be drawn: 1) So far, all investigations are based on smoke detection. However, smoke detection using wildfire images taken during the night is not effective, especially with smaller datasets. There has been no research thus far in creating separate learning models for nighttime and daytime wildfires.
2) The wildfire images from nighttime carry large variations in color, texture and shapes. No research papers have talked about those significant variations against smoke based detection.
3) How a CNN Network model can be generic and flexible across various locations and regions for wildfires detection.
4) A self-learning and automated process is imperative to detect wildfires very early.
In this research paper, a novel systematic approach for automatically detecting wildfires and alerting response systems were proposed and implemented with the following three major advances: 1) Google Cloud Platform (GCP) was used in this research to build Convolutional Neural Network (CNN) learning models for daytime and nighttime wildfires.
2) A modern model server architecture to serve the models with input images to achieve high accuracy regardless the time of those images were taken.
3) To make this work more generic so that it can be leveraged at different locations, Convolutional Neural Network Fine tuning is explored to adapt and enhance the network models to have the same high accuracy across locations. www.ijacsa.thesai.org

A. Data and Data Preparation
For this research project, a combination of several different sources of wildfire image data were used. The images are taken from Kaggle, open source projects, and Google images. Most of the images have been properly labeled with wildfire and non-wildfire, but a few are unlabeled, so a few of hours work were spent on manually labeling those images for the purposes of this research. In real life, it is worth noting that image labeling can be achieved by crowdsourcing and having data sourcing companies, such as keymakr.ai, scale.ai, etc, provide us with lots of labeled data. The images were split into daytime and nighttime to build two separate learning models with the same setup. The number of images in the training, validation, and test sets used for the wildfire smoke detection model can be seen in Table I. The following pre-processing steps were performed: 1) Combine all images into one big data set.
2) Filter out images of wildfires in black and white and those with questionable smoke and flare.
3) Classify images into daytime and nighttime wildfire sets 4) Ensure all images are properly labeled, especially for the training and test datasets Then the following transformations were performed during data loading to improve the performance of the models. The images were resized and cropped to the empirically determined size of 1040 × 1856 pixels to improve training and inference speed. This operation also enables us to evenly divide the image into overlapping 224 × 224 tiles. Finally, normalization of the images to 0.5 mean and 0.5 standard deviation was conducted, as expected by the deep learning package used (pytorch vision).

B. Implementation of CNN and Alerting Mechanism
Pytorch was used as the underlying framework for the Convolutional Neural Network in this research. Fig. 3 illustrates the overall architecture. Using the Fast R-CNN package, based on the idea of running the CNN just once per image and then finding a way to share that computation across ~1000 proposals, each wildfire image was fed once to the underlying CNN and then selective search was run as usual to generate region proposals. Then, each proposal is projected onto the feature maps generated by the CNN. Fast R-CNN offers an exponential increase in terms of speed [25] over traditional CNN"s.
In this research, a model server framework was used. Currently, there are multiple implementations of model server for serving purposes, but the original idea of a low latency and high throughput model server came from the research from UC Berkeley Rise Lab Clipper framework [26]. As of now in the market, there are multiple implementations of model server for serving purposes. TorchServe was used as the implementation framework and combined the API from sunrise-sunset.org [27] to get the trigger point for model serving. Fig. 3 illustrates the model server architecture. The characteristics for wildfires detection in daytime and nighttime can be very different. In the daytime, early detection is heavily based on smoke, so fog, mist, etc. are the primary factors that affect the model"s accuracy. At night, early detection focuses more on glares, so lights, fog, etc. are the main factors that could skew accuracy. It is important to treat them differently. After the pre-processing described in section A was completed, three models were trained -(i) one big model with the datasets from both daytime and nighttime combined, (ii) one model for just daytime, and (iii) one model for just nighttime. The models for daytime and nighttime together were aggregated by using the model server framework for the result. The comparison of the results from one big model vs the aggregation of daytime model and nighttime model is discussed in section C. Fig. 4 illustrates the overall systematic approach.
After developing a highly accurate spot wildfire detection system, an alerting and notification mechanism can be established. The higher the detection accuracy is, the lower is the count of false positive alerts. In this investigation, the alerting system was divided into two parts -1) social channels (Twitter, Facebook, Instagram, etc.) with different social handlers, and 2) SMS text or automated phone calls to police and fire departments -to alert responders to act fast. To elaborate on the first part, a social handler responsible for monitoring over a specific area can publish wildfire hazard alerts to the people subscribed to it across various social media platforms. This can be very effective in informing people who live around that specific area to prepare themselves for immediate evacuation upon the start of a wildfire. To expand on the second point, cloud services like Twilio can be leveraged to send SMS messages to a given area code for wildfire notifications.

C. Key Metrics and Results
True positive (TP), false positives (FP), true negatives (TN), and false negatives (FN) are calculated between the model predictions and the ground truth labels. For all the experiments, the following two key metrics were used typically for binary classification problems. It is the ratio of TPs over all detections made by the model, namely:  Recall -Measure of the ability of the model to detect all ground-truths instances.
It is the ratio of TPs over all the ground-truths, namely:  Accuracy -Measure the fraction of all instances that are correctly categorized; it is the ratio of the number of correct classifications to the total number of correct or incorrect classifications A model is said to be a good model if it has high precision and high recall. A perfect model has zero FNs and zero FPs (that is, precision=1 and recall=1). However, it usually is not feasible to attain a perfect model.
In this research, after the model server was employed, the accuracies were calculated by the means of the daytime and nighttime models. Table II shows the comparison of  precision, recall and accuracy from different models and  approaches at stage training and validation.   TABLE II. RESULTS OF PRECISION, RECALL AND ACCURACY.

Model P (%) R (%) A (%)
Train big model 90. From Table II, it shows that the approach with the model server framework to serve the network models from daytime and nighttime has much higher precision and accuracy of 97.8% and 89.5% vs. one model approach of 89.5% and 83.7% respectively at image validation. Clearly the approach with the model server architecture offers much higher precision and accuracy.

D. Alerting and Notification
In this investigation, a test dataset was used to trigger the process of posting tweets to Twitter. When a test image was fed to the network model, if it is classified as fire, the Twitter API is triggered to generate a message that will be broadcasted to all the subscribers of a Twitter handler automatically in real time. The same mechanism can be implemented to send SMS messages or even automated phone calls.

E. Additional Observations and Future Plans
The results in this research show that spot wildfire can be detected with high precision and accuracy based on the AI Convolutional Neural Network (CNN) learning model with live data stream from monitoring stations, drones or satellites. The cost perspective of the approach from a commercialization perspective will not be discussed in this paper. The computing power for digesting live data and powering the AI model in addition to data storage has a significant cost impact. Soon, real time image capturing will not be an issue in feeding data through a pipeline. Potential edging computing or embedded systems with the AI learning models can be used or deployed to reduce the costs for spot wildfires detection. At the same time, alerting and notification to the public and officials can be established in an automated way. In this research the sunset and sunrise API [27] was used to invoke different models, this may not always be accurate. Wildfire characteristics at dawn and dusk time may get blue, but with more training data, classification accuracies at dawn or dusk will surely increase.
In the future, it would be interesting to do an investigation on the impact of time series images to the complexity of the Alerting & notification www.ijacsa.thesai.org network models. Since real life images would be captured via forest monitoring high towers, drones or satellites, by nature they are time series images, this may make the network models even less complex and higher accuracy, but more research and validation are needed. The impact for the time series images to the learning correlations needs further investigation.
In this paper, the dataset used is not very big, the more data that is fed into the CNN, the more powerful the model could become when it gets trained. It would also be an interesting research topic to find a way to automate continuous improvement mechanisms for fine tuning CNN models so that learning models can be generic enough across different locations and regions. Finally, the model server architecture can be further refined and used for different segmentation of conditions with more drastic differences in order to make detection even more robust.

IV. CONCLUSION
In this research, a novel machine learning convolutional neural network (CNN) with a combination of model server was used to aggregate models for daytime and nighttime to have a higher accuracy. The model server serving different models with different time ranges (approach 1) vs. one big model that did not distinguish daytime and nighttime (approach 2) was compared for accuracy and feasibility. With approach 1, a higher precision of ~98% was achieved vs. approach 2 of ~90%, and with a shorter training time. Approach 1 carries more implementation complexity. This research result shows that wildfires detection accuracy can be improved significantly by considering different models for images from different time intervals and combining them using a model server architecture.
In this research, an alerting and notification system is discussed and can be built to integrate with social media and wildfire responding systems to automate the entire detection and alerting process to have wildfires under control to save lives and reduce property damages.
This investigation is based on the datasets that were collected from the Internet (Kaggle, Google Images and GitHub). In real life, real time videos/images capturing will be more accessible with the development of satellite monitoring, drones and many cases of monitoring towers set up. Those datasets would be more time series and unbalanced data with less variation. The datasets used in this research carry more variations of wildfire scenarios for training the models. A preprocessing with the data sets collected was performed before image feeding to the training process of CNN. With the real life time series images, the preprocessing also could be simplified. The models trained can be used at various places across the globe with CNN fine tuning for spot wildfire detection.
Lastly, the key in wildfire detection, prediction and prevention is to achieve automation, once network models are trained and deployed in the cloud, they can be continuously refined automatically. Live data from real time capturing from various tools can be fed to the models for wildfires detection without much human intervention. A process, feeding of the live data, refining the model and automatically redeploying the model in the cloud will greatly help our societies to fight the increasing risks of wildfires across the globe, and also will help to reduce the carbon emissions resulting from wildfires.