Opportunistic use of Spectral Holes in Karachi using Convolutional Neural Networks

—Wireless services appearing in the next generation wireless standard i.e. 6G include Internet of Everything (IoE), Holographic communications, smart transportation and smart cities require exponential rise in the bandwidth in addition to other requirements. The current static spectrum allocation policy does not allow any new entrant to exploit already grid-locked Radio Frequency (RF) spectrum. Hence, quest for larger bandwidth can be fulfilled through other technologies. These include exploiting sub-Terahertz band, Visible Light Communication and Cognitive Radio scheme or exploiting of RF bands in opportunistic fashion. Cognitive Radio is one of those engines to exploit the RF spectrum in secondary style. Cognitive Radio can use artificial intelligence driven algorithms to complete the task. Several intelligent algorithms can be used for better forecasting of spectral holes. Convolutional Neural Network (CNN) is a Deep Learning algorithm that can be used to predict the presence of a spectral hole that can be opportunistically exploited for efficient utilization of RF spectrum in secondary fashion. This paper investigates the performance of CNN for metropolitan Karachi city of Pakistan so that the users can be provided with uninterrupted access to the network even under busy hours. Dataset for the proposed setup is collected for 1805 MHz frequency band through NI 2901 Universal Software Radio Peripheral (USRP) devices. The root mean square error (RMSE) for the predicted results using CNN appears to be 81.02 at epoch of 200 and mini-batch loss of 3281.8. Based on the predicted results, it was concluded that CNN can be useful for investigating the possible opportunistic usage of RF spectrum; however, further investigation is required with different datasets.


I. INTRODUCTION
6G wireless communication standard and services discussion and investigation is already initiated. The proposed network initiatives promises to provide traffic capacity in the range of 1-10 Gbps/m3 as compared to 10 Gbps/m3 availability in 5G networks. It is assumed that the technologies that will play the role of enablers may include sub-Terahertz, visible light communication and Artificial Intelligence enabled cognitive communications. Additionally, greater use of multi-RAT and multi-link schemes will also be required to rectify the issues arising from higher frequency propagation and providing higher reliability communication links [1].
The higher capacity requirement of the future technologies is a great challenge towards ubiquitous connectivity of wireless devices. Cognitive Radio is a possible option to play key role towards successful exploitation of Radio Frequency (RF) spectrum in opportunistic fashion. Thus, producing enough bandwidth to provide connectivity to wireless devices even in the congestion time. Cognitive Radio is a novel concept to incorporate artificial intelligence enabled techniques to exploit spectrum in opportunistic fashion. The major requirement towards a successful exploitation of RF band is to use primary bands in such a fashion that the usage does not produce harmful interference to the primary users. The cognitive radio is implemented through cognitive cycle [2]. The process of cognitive cycle starts with gathering RF spectrum monitoring [3]- [5]. This can be done through spectrum sensing process [6] and geo-location databases [7]. Spectrum monitoring combined with Radio Environmental Maps (REM) can also produce a better method of identifying unused Spectral bands [8]. Additionally, in [8] it is presented that the geo-location based database scheme typically results in an underutilized spectrum sharing scenario hence, better spectrum measurement scheme shall be incorporated that includes joint spectrum monitoring network and REM schemes [8] [9].
In the Artificial Intelligence enabled spectrum sensing radios, radios measure the Signal to Noise Ratio (SNR) and Energy levels of the primary radio transmission and this data is fed to the training algorithms. These algorithms predict the future nature of the RF spectrum bands regarding their usage in future i.e. empty or occupied. For this activity, different algorithms can be involved. As the spectrum assignment is a completely random assignment so a linear algorithm like Logistic regression may not produce useful results in all the channel conditions with different available active and secondary users [10]. While in most of the nonlinear environments, different algorithms use many other algorithms due to their specific use such as Support Vector Machines, K-Nearest Neighbor (KNN), Linear discriminant analysis (LDA) and Decision Trees [11]. In comparison to the aforementioned machine learning algorithms [12]- [14], deep learning algorithms typically perform much better.
The efficiency of Deep Learning algorithms is far better than the machine learning algorithms in general. However, this efficiency is resulted on the basis of available dataset and the nonlinear nature of the outputs. For example in case of linear functions, the logistic regression is preferred over the nonlinear www.ijacsa.thesai.org algorithms because not only the regression produces simpler results but also predicts the given function in perfect fashion. However, when the available function is nonlinear and we have luxury to collect the 5000 samples of the data set we choose deep learning based algorithms in comparison to machine learning algorithms. Typical applications of deep learning algorithms include the real time test cases when the logistic regression don't produce the required level of accuracy. Additionally, there are complex patterns that can only better identified through these complex algorithms. Typically, audio and video data come into the picture of the said category [15].
Convolutional Neural Networks (CNNs) find their way to the area of deep learning based problems at enormously higher rate. These algorithms find their applications into the domain of machine translation, sentence classification, and sentiment analysis. Similar to the classic case of slide window algorithms, these algorithms slide over the given sequence of characters [15].
In the current work, CNN is chosen to predict the data for the opportunistic use of spectral holes in a large metropolitan city of Karachi Pakistan. For this purpose, a dataset is trained using RMSprop, which is a fast learning algorithm. The CNN has been used as a high performance classifier. The operation of convolution strengthens and reduces interference with the original signal features and have better tolerance to noise. The training parameters in CNN are less than in a fully connected network [16]. Two factors including Root Mean Square Error (RMSE) and loss function have been used to interpret the results.
The rest of the paper is divided in such a way that Section-II describes the related work. It is followed by Section-III which provides details of the proposed system. Section-IV explains the results obtained. Section-V is the last section providing Conclusion and future directions.

II. RELATED WORK
In the literature, Deep Neural Network based prediction algorithms are used by various authors to produce the data of spectral holes so that maximum number of users could be accommodated in available RF spectral bands [17]. Authors in [8] have presented a novel framework for spectrum monitoring purpose. The authors add CNN based algorithm to the spectrum measurement devices i.e. spectrum detectors that help the devices to identify the presence or absence of primary user signals (in this case, Radar signals are considered as primary users) even if the signals are found overlapped with other secondary users (in this case, WLAN and LTE are considered as secondary users). Furthermore, a large dataset containing various signal waveforms such as Radar pulses, LTE, WLAN and thermal noise waveforms is also developed that can be utilized by other researchers. A novel pre-processing scheme is also implemented that produces samples of amplitudes and phase shift of the collected waveforms. The results produced by the proposed scheme are excellent i.e. 99.6% accuracy is achieved when the proposed algorithm is run on testing dataset. This is in addition to the robustness of the proposed algorithm to noise. The proposed algorithm is also run on various SNR regimes that shows an improved performance of the selected algorithm in comparison to other models such as spectrogrambased CNN algorithms.
In [18], authors model the spectrum sensing issue as a classification problem by using CNN based processing. The received signal samples are initially normalized to overcome the effect of noise and uncertainties embedded in the received signal samples. The proposed model utilizes following different signaling schemes along with their variants i.e. PSK, FSK, QAM and AM. This encompasses reasonably wide variety of signals that are tested under the scenario considered in the paper. The maximum possible number of real world scenarios is considered for the training setup of proposed scheme. This feature will help the proposed scheme to perform better in the undefined wireless environments such as realworld wireless channels. The performance of proposed model is also compared with two algorithms maximum-minimum Eigen Value Ratio and frequency-domain entropy based methods. The results show better performance against its competitors even in the presence of colored noise. That shows the proposed scheme may find better use in the real-world environments. Additionally, to tackle the real world wireless issues, the proposed schemes is added with transfer learning technique to improve the performance. The data utilized in the proposed scheme is simulated except the experimentation into the real-world wireless environment testing.
Detection of RF signals under low SNR regime poses a great problem towards successful utilization of RF spectrum by opportunistic users [19]. In [16], authors attempt to solve the pressing issue (of detection under low SNR) by using CNN algorithm. The data collection for the proposed experimentation is performed by using Cyclostationary feature detector (An algorithm that converts the received signal into its signatures) and energy detection (An algorithm that measures the energy of the received signals). The performance of the proposed scheme is also compared with classic cyclostationary feature based detection schemes. The results show an improvement.
In classic spectrum sensing algorithms, typically model based approach is applied to sense the wireless environment i.e. energy and other methods, however, to devise better results the authors in [20] recommend a data driven approach to solve the problem of spectrum sensing through Deep convolutional network based spectrum monitoring approach. The proposed algorithm uses data to train itself. The sample covariance matrix is used as an input to the CNN algorithm. Maximum Aposterior Probability (MAP) based technique is used to devise the cost function. In offline method, the results of the proposed scheme show excellent performance by showing improved performance even under the correlated samples of primary user with correlation coefficient of 0.7 and uncorrelated primary user. Additionally, the proposed scheme works well than the conventional scheme i.e. Eigen value detection algorithm by approximately 7.5 times at an SNR level of -14 dB.
CNN is a branch of deep neural network in deep learning which is becoming popular tool for analyzing visual images. It includes image recognition, classification, and detection tasks [21]. They are referred to as convolutional neural networks www.ijacsa.thesai.org because they use mathematical operation of convolution which is a specific type of linear operation [22]. In terms of Neural Networks, CNNs fall under category of generalized multilayer perceptron (MLPs). The major difference between CNN and MLP is that in MLP each input element is connected to each neuron in the hidden layer causing full connectivity whereas in CNN only a limited number of input elements called the receptive field is connected to only part of the hidden layer. CNN reduces the risk of over fitting of data in this way as compared to MLPs [23].
CNN is a type of deep learning which has been found to be very successful in areas of objects and image recognition, detection and segmentation challenges. CNNs applications in the various fields are on the rise. A number of reviews on applications of CNN have been done in the literature. A few recent reviews are shown in Table I. It can be seen that CNN applications are found in health informatics and in other image classification schemes.
A generalized CNN consists of input layer, an output layer and a number of hidden layers [8]. A generic CNN architecture has been shown in Fig. 1. CNN algorithm uses three layers to convert an input signal to output. The layers are: input, feature extraction e.g. learning and output layer. The input layer passes the data to a series of kernels i.e. convolutional layers with filters, pooling and fully connected (FC) layers. Then the Softmax function is used to decide the output data with a value of probability. Thus, it can be deduced that the convolutional layers works as an engine to extract features of input signal. It is also represented through mathematical operation of convolution. Convolutional layer works in conjunction with ReLU function. It is also called Rectified Linear Unit. There are many nonlinear functions that can be utilized such as Sigmoid; however, performance of ReLU under real environments is better than its competitors. The output of the layer is connected with pooling function. Pooling function is used to reduce the layer number of input parameters. Hence, it is also known as subsampling or down sampling unit. It is used to reduce the dimensionality of the input map; however, the significant information is always retained to make it highly useful function of the defined algorithm. It is generally implemented through max, average and sum pooling.
In this paper, CNN is used to predict the RF Usage in Karachi so that the available bands can be identified for further opportunistic usage. For this purpose, a dataset is also developed by using NI 2901 Universal Software Radio Peripheral (USRP) devices. CNN is applied to this dataset for prediction of the available RF bands. The RF band selected for the purpose is 1805 MHz. The RMSE for the proposed case comes out to be 81.02 at 200 epoch. The next section explains the details of the experimentation and analysis of the simulation results. Fig. 2 shows the structure of proposed system. In system, NI USRP 2901 is used to sense the presence of useful signals under real channel. A dataset is s developed using spectrum sensing algorithm. The Labview graphs are ported onto the Excel and Matlab to train the given algorithm. RMSprop is used for training purpose for the selected CNN Algorithm. Performance of RMSprop is closer to Momentum Gradient Descent Algorithm. It is used to control the learning rate thus taking larger steps produces convergence more quickly in horizontal direction. Two parameters including RMSE and Loss Function were used for investigation of the results. RMSE can be defined as standard deviation of residuals (prediction errors). In other words, it shows the difference between the regression line and spread of the prediction errors. Machines work typically through loss function parameter. Loss function is used to determine the output of the given algorithm and the given target value. However, under circumstances when predicted values deviate too much from loss functions, the estimation error algorithms uses optimization functions.

III. PROPOSED SYSTEM
IV. RESULTS Fig. 3 shows two parameters for the proposed setup. These include RMSE and Loss function. Total time to generate the results from proposed setup is 4 minutes and 40 seconds. One iteration is assumed for each epoch. Learning Rate decreases in larger amount. It means that chances of reaching a global optimal solution are high but there is also risk of getting stuck at a sub optimal solution. The training statistics for the experiments conducted is also shown in Table II for 200 iterations as till that time the result becomes consistent in terms of RMSE and loss function. Results indicate that for the dataset of Karachi, after few iterations, RMSE and Loss function become consistent, however rapid decrease in learning rate is an area which needs to be further investigated. It is worth mentioning that the proposed setup is implemented using a single CPU. Hence, for large amount of data, GPUs are recommended to use.  In this work, deep learning using CNN has been used to predict the spectral spaces for Karachi city. The algorithm takes longer time to train as compared to machine learning algorithms. The proposed setup is implemented using a single CPU machine. The results show that the best possible RMSE and Loss function are achieved at 50 th iterations. Even after increasing the iterations, no significant progress on these parameters is found. However, the Base Learning Rate decreases significantly after running the machine for 200 iterations and needs to be further investigated and compared with other deep learning algorithms. This work was based on data set for metropolitan city of Karachi. In future, we aim at investigating the rural area datasets to find out any significant differences while changing datasets with CNN and other deep learning methods for investigating opportunistic use of spectral holes with the aid of artificial intelligence techniques.