Comparison of Multilevel Wavelet Packet Entropy using Various Entropy Measurement for Lung Sound Classification

Wavelet Entropy (WE) is one of the entropy measurement methods by means of the discrete wavelet transform (DWT) subband. Some of the developments of WE are wavelet packet entropy (WPE), wavelet time entropy. WPE has several variations such as the Shannon entropy calculation on each subband of WPD that produces 2N entropy or WPE, which yields an entropy value. One of the WPE improvements is multilevel wavelet packet entropy (MWPE), which yields entropy value as much as N decomposition level. In a previous research, MWPE was calculated using Shannon method; hence, in this research MWPE calculation was done using Renyi and Tsallis method. The results showed that MWPE using Shannon calculation could yield the highest accuracy of 97.98% for N = 4 decomposition level. On the other hand, MWPE using Renyi entropy yielded the highest accuracy of 93.94% and the one using Tsallis entropy yielded 57.58% accuracy. Here, the test was performed on five lung sound data classes using multilayer perceptron as the classifier. Keywords—Wavelet packet entropy; lung sound; Shannon entropy; Renyi entropy; Tsallis entropy


I. INTRODUCTION
Abnormalities that occur in the respiratory system can be observed from the sound generated during the respiratory process.This breathing sound commonly is heard by a doctor using a stethoscope, also known as auscultation.The respiratory or pulmonary sound analysis is one of the most interesting research topics in the field of medical signal processing.Various methods have been developed for the extraction of pulmonary sound features for automatic classification.One of the most commonly used methods is the entropy analysis.Entropy is a measure of signal or system irregularity.It is frequently used to measure signal complexity as in biological signals.
Various entropy calculation methods have been used in lung sound analysis in which Sample entropy was used as a feature for detecting pulmonary sound status using morphological complexities [1].Meanwhile, Tsallis entropy was used for lung sound analysis in [2] and [3].Multiscale entropy was reported to be better in distinguishing lung sounds in alveolitis patients rather than spectral or statistical methods [4].Another entropy measurement method is the wavelet entropy (WE).It uses Shannon entropy calculations on the subband of discrete wavelet transform (DWT) [5].The improvement of the wavelet entropy is the wavelet packet entropy (WPE) that uses the wavelet packet decomposition (WPD) subband [6].WE produced low accuracy when used as a feature for pulmonary sound classification as reported in [7].
In five classes of pulmonary data, WE only resulted in an accuracy of 43%.For that reason, the development of WE method is needed to improve the accuracy of pulmonary sound classification.
Previous research has proposed a multilevel wavelet packet entropy (MWPE) method for pulmonary sound feature extraction [8].Entropy was calculated on the subband of WPD at several levels using the Shannon entropy method.In addition to the Shannon method, there are other several methods of calculating entropy such as Renyi entropy and Tsallis entropy.Renyi entropy is a common form of Shannon entropy [9]; while Tsallis entropy is a generalization form of entropy with the generalization parameter of q [10].In this study, MWPE was calculated using Renyi entropy and Tsallis entropy to observe the resulted accuracy.The results obtained was then compared with the MWPE resulted in previous studies using five classes of lung sound data.This paper is presented as follows.Section 2 describes some previous studies using wavelet entropy and wavelet packet entropy.Section 3 describes the detailed methods used in this study including data, wavelet decomposition, and classifier.Results and discussion are presented in Section 4 and the conclusions and prospects for future research are presented in Section 5.

II. RELATED WORKS
Wavelet entropy (WE) is widely used for complex signal analysis such as for biological signals.It is entropy calculation using subband of DWT.In [5], WE was used for brain signals analysis in short durations.Compared to spectral entropy (SE), WE was found better to detect non-stationary signals.It was also used for a ventricular beat suppression analysis in the cases of atrial fibrillation [11].A number of differences in WE values showed some different levels of suppression in the ventricular beat.If WE was calculated on the subband of the DWT, then some researchers used subband results from WPD. Entropy in the subband of WPD results were used for analysis of murmurs in heart sound in [12].Not all subbands were used for entropy calculation but they could be calculated based on the frequency range, noise frequency, and energy threshold [12].www.ijacsa.thesai.orgAnother variation of WE was the different entropy calculations on wavelet subband.Sample entropy on DWT subband used for EEG signal analysis was presented in [13].Cen and Li used the Tsallis wavelet entropy for power signal analysis [14].Normalized Shannon wavelet entropy, meanwhile, was calculated on wavelet coefficient for epileptic EEG analysis [15].
Another method based on wavelet entropy is the wavelet packet entropy (WPE).Some variations of WPE have been proposed by several researchers.In [6], entropy was calculated using crest energy in each subband of WPD results.Meanwhile, Shannon method was used in WPD subband for bearing inspection in [16].The number of generated features was 2 N , where N refers to the signal decomposition level.In another study, multilevel wavelet packet entropy (MWPE) was proposed for lung sound analysis [8].If in [16], WPE was produced by calculating the Shannon entropy on each WPD subband, then in [8], WPE was generated from Shannon entropy calculations from the subband relative energy such as WE calculation in [5].So, each decomposition level would produce an entropy value.Since WPE was calculated on multilevel, for N level decomposition it will produce N entropy values as the signal feature.The experiments reported 97.98% accuracy using Db8 at the level of decomposition N = 4 [8].The results were obtained for five classes of lung sound data.
In previous research, MWPE used Shannon entropy calculation to compute entropy on WPD subband.In this study, MWPE was tested using Renyi entropy (RE) and Tsallis entropy (TE).Renyi entropy is a common form of the entropy equation [9].Meanwhile, Tsallis entropy, commonly called as non-extensive entropy, is often used for non-additive signal analysis [17].A comparison among ShEN, RE, and TE for MWPE calculation is expected to be a recommendation of the selection of entropy calculation methods on MWPE for biological signal analysis, especially for lung sound analysis.

III. MATERIAL AND METHODS
Fig. 1 shows a block diagram of the process conducted in this paper.First, the preprocessing of the pulmonary sound signal was done for amplitude normalization and to uniform the mean of the signal.WPD was then performed from level 1 to level N.At each level of decomposition it performed WPE calculations using the Shannon, Renyi, and Tsallis method.In the next stage, the classification was done using MLP and Nfold cross-validation.A further explanation is provided in the following subsections.

A. Lung Sound Data
In this study, we used lung sound recording data obtained from several sources [18] [19].The same data was used in previous studies [8] [20].Each pulmonary sound data consisted of single breathing cycle, inspiration, and expiration.Using a sampling frequency of 8000Hz, the length of one data was then ranged from 20000-30000 samples.The data consisted of five classes of normal bronchial data representing normal lung sounds, wheeze, stridor, crackle, and plural rub representing pathological lung sound [21].In the data normalization process was carried out as follows.Zero-mean was done as in Equation (1).
with x(n) is the input signal and y(n) is the output signal, which has mean = 0. Furthermore, the normalization of amplitude was presented as in Equation ( 2) in order to obtain a signal in the range of -1 to +1.
In the next step, we did wavelet decomposition to calculate the WPD as in the following section.

B. Wavelet Packet Entropy
Wavelet packet decomposition (WPD) on signal S(t) was defined as in equation (3).
with S(t) is an original signal, j is the scale, n and k are the band and surge parameter respectively.From Equation (3) we could calculate the energy of each subband as in Equation (4).where j, n, k represent the scale, band, and surge parameter, respectively.The total energy of WPD is: Relative energy for each subband in scale j can be expressed as: Wavelet packet entropy (WPE) is expressed as: The N notation is used to denote the level of decomposition used in WPD.www.ijacsa.thesai.org In the previous study, we used one WPE value as a feature for signal analysis.In this paper, it is proposed to use N WPE value for the feature extraction of the pulmonary sound signal to improve accuracy in pulmonary sound classification.The characteristics used in this study are as in (8).
Equation ( 7) uses the Shannon method to calculate WPE.This equation can be modified using Renyi entropy or Tsallis entropy.The calculation of Renyi entropy for WPE can be expressed by Equation (9).
where r is the notation in which WPE was calculated using Renyi entropy (RE), N is the decomposition level, and q is the order.Practically, we used the order of RE q = 2.
Meanwhile, WPE with Tsallis Entropy method can be calculated using equation ( 10) Where, t is the notation in which WPE was calculated using the Tsallis entropy (TE) method, N is the decomposition level, and q is the order.Here, we used TE order q = 2.
MWPE would be calculated at the decomposition level N = 1-7.Based on the results of previous research, a higher level of decomposition of N> 7 will not improve accuracy [8].The mother wavelets tested were Haar, Db2, Db8, Bior1.5, and Bior2.8 as used in [8].

C. Classifier and Validation
In this study, we used multilayer perceptron (MLP) as the classifier and N-fold cross-validation (Nfold CV) for validation.MLP and Nfold CV were selected as the classifiers in which the results obtained would be compared with previous studies that used the same classifier and validation.MLP was chosen because of its simple architecture and its ability to solve non-linear problems.MLP does not require a large amount of training data to learn [22].MLP consists of the input layer, hidden layer, and the output layer.The number of features determine the number of node in input layer; while the number of nodes in the output layer corresponds to the number of data classes.Meanwhile, the number of hidden layers and the number of nodes in the hidden layer were determined by trial and error.Basic configuration of MLP is displayed in Fig. 2. For MLP parameters we used learning rate 0.3, momentum 0.2, epochs 500, and sigmoid as activation function.We did not choose the best parameter for MLP because we wanted to focus on the effect of MWPE as features.In this study, we used 3fold CV; the overall data was divided into three datasets with one dataset used as test data and two datasets used as training data.Testing was done three times so that all dataset ever used once as test data.We chose 3FCV because the least amount of data in one class is 18 so that at least each data set will consist of six data.

IV. RESULTS AND DISCUSSION
The result of wavelet packet decomposition up to level 2 for wheeze sound can be seen in Fig. 3.It appears the information of the signal concentrated at low frequency.Subband AA2 occupied the band 0-1000Hz indicating that the most of the lung sound energies lied in the frequency <1000 Hz.Thus, a decomposition level N > 2 was required to view information from the lung sound.In this research, we used the level of decomposition N = 7 so that the subband can be as wide as 31.75Hz.Fig. 4 shows the MWPE results for each entropy calculation.Fig. 4(a) shows MWPE using Shannon method.Shannon method generated WPE values that increased along with increasing levels of decomposition.This was because the energy in each subband spread more evenly, especially for the frequencies below 1000 Hz.The WPE value of each decomposition level for each class was also relatively far apart, so there was a difference between each type of lung noise.Stridor produced the highest WPE value, while the pleural rub produced the lowest one.Stridor had a more evenly distributed signal spectrum <1000 Hz while the pleural rub had a spectrum that tended to be concentrated at one frequency.Comparison between Stridor and pleural rub can be seen in Fig. 5 and 6.Crackle and stridor had the highest value, and relative coincide while wheeze, normal, and pleural rub resulted in lower values.The MWPEr value tended to increase as the N value rose but not so high.The minus sign was generated from the factor (1q) with q = 2.As shown in Fig. 3, some MWPEr values coincided at some N levels, causing the possibility of relatively high classification errors.MWPEt had the same pattern as MWPEr because it had the same form of the equation and was calculated in a same order.MWPEt had a higher magnitude compared to MWPEr.
The accuracy of lung sound classification using MWPE, MWPEr, and MPWEt is presented in Table 1-Table 3.In Table 1, the highest accuracy using MWPE was 97.98% using Db8 with decomposition level N = 4 [8].The 97.98% accuracy was also achieved when N = 5-7 but N = 4 was taken as the best parameter for producing the fewest features.It is generally seen that the accuracy increases when the N value rises and then stable over N = 4.At N = 4, 16 subbands (24) would be formed with a width of 250Hz on each.Because of the most significant information lying in the frequency <1000 Hz, then four subbands of 250 Hz were sufficient as a differentiator between data classes.It can be seen in Fig. 2 that for N = 4 there is a considerable difference between the data classes.
MWPEr produced the highest accuracy of 93.94% using Bior2.8 and the decomposition level N = 4.For higher N, the accuracy value was unchanged.This result was similar with a result on MWPE where a higher level of decomposition did not produce a higher accuracy.MWPEt only produced the highest accuracy of 57.58% for Bior1.5 with N = 3.The higher decomposition rate did not improve the accuracy.This low accuracy was due to very high WPEt values and tended to spread.www.ijacsa.thesai.orgFrom the simulation results obtained that MWPE using Shannon entropy yielded the highest accuracy of 97.98% using Db8 and N = 4.This result was better than MWPEr and MWPEt which produced the highest accuracy up to 93.94.The use of MWPE was better than the use of WPE at one decomposition level as in [8] which produced an accuracy of up to 70.71% at N = 1 using Bior2.8.In the previous study, the use of WE as a feature only resulted in 43.43% accuracy using DWT Db2 level 7; while, the combination of six entropies yielded an accuracy of 94.95% [7].
In this study, all WPD subband results were used to calculate WPE.In another study, the subband selection of WPD results was performed to estimate the signal features.In [12], the information on each node became the basis for selecting the subband to be used as a feature.Meanwhile, in another paper, the distribution of data on the frequency spectrum was used as the basis for the gradual take up of subband [23].The study of the best subband selection for MWPE calculations will be a promising topic in subsequent research.

V. CONCLUSION
This paper describes the variation of MWPE calculations using Shannon entropy, Renyi entropy, and Tsallis entropy.Tests using five lung sound data classes showed that MPWE using Shannon entropy yielded higher accuracy compared to other two methods.This indicates that the distribution of energy in each subband is adequate to be the basis for the feature extraction of lung sound.MWPE is open for use in other biological sciences such as ECG, EEG, and heart sound.

Fig. 4 (
Fig. 4(b) shows MWPEr in five classes of lung sound data.Crackle and stridor had the highest value, and relative coincide while wheeze, normal, and pleural rub resulted in lower values.The MWPEr value tended to increase as the N value rose but not so high.The minus sign was generated from the factor (1q) with q = 2.As shown in Fig.3, some MWPEr values coincided at some N levels, causing the possibility of relatively high classification errors.

Fig. 4 .
Fig. 4. Wavelet Packet Entropy using (a) Shannon Entropy for N Level Decomposition (b) Renyi Entropy for N Level Decomposition (c) Tsallis Entropy for N Level Decomposition.

Fig. 4 (
Fig. 4(c) shows MWPEt in five classes of lung sound data.The results showed the same pattern as MWPEr where crackle and stridor had relatively close values while normal, wheeze, and pleural rub had lower values with the same pattern.MWPEt had the same pattern as MWPEr because it had the same form of the equation and was calculated in a same order.MWPEt had a higher magnitude compared to MWPEr.

TABLE III .
THE ACCURACY OF MWPET (%) FOR VARIOUS MOTHER WAVELETS AND DECOMPOSITION LEVEL N