Signal Reconstruction with Adaptive Multi-Rate Signal Processing Algorithms

Multi-rate digital signal processing techniques have been developed in recent years for a wide range of applications, such as speech and image compression, statistical and adaptive signal processing and digital audio. Multi-rate statistical and adaptive signal processing methods provide solution to original signal reconstruction, using observation signals sampled at different rates. In this study, a signal reconstruction process by using the observation signals which are sampled at different sampling rates is presented. The results are compared with the least mean squares (LMS) and the normalized least mean squares (NLMS) methods. As the results indicate, the signal estimation obtained is much more efficient than the previous studies. Designed multi-rate scheme provides significant advantages in terms of error and estimation accuracy. Keywords—LMS; Multi-Rate Systems; NLMS; Statistical Signal Processing


INTRODUCTION
Multi-rate signal processing is an integral part of the signal processing technique and has been developing rapidly during the last decade.Multi-rate signal processing methods focus on systems which include signals that are sampled at different rates.In many communication and signal processing systems there are two main fields of study: signals sampled at different rates and signals with variable sampling rates.Multi-rate signal processing techniques solve these problems efficiently.Recently multi-rate signal processing techniques are used in image and speech compression, digital audio coding, statistical and adaptive signal processing, discrete-time multidimensional signal processing, high-resolution image acquisition.In particular, when the developments in the last decade are examined, the study of [1] is highlighted.The authors utilize two signals with one having twice the sampling rate of the other to detect the coefficients of the random process.They show that the optimal filter is a linear filter whose coefficients change periodically for this particular problem.The authors in [2] study multi-rate signal processing and analyze the fundamental themes of cyclic signal processing systems.In [3], information measure is determined for multirate linear systems.[4] investigates stationary concept under variable rates and multi-rate Wiener filtering.Random signals with different sampling rates which involve observations taken from several observers are estimated in [5].The convergence analyses of the multi-rate systems are presented in [6].It is observed that if significant increase in rate does not occur than the convergence rate can increase.When the multi-rate observation sequences are used, an adaptive filtering is achieved by LMS algorithm in [7].
Authors in [8] found out a solution to the problem of reconstructing a high resolution signal from two low-rate sensors with time delay by using multi-rate measurements.An adaptive filtering is achieved by the help of multi-rate observations and LMS algorithm in [9].Also multi-rate signal modeling for target recognition in radar monitor is investigated in [10].Optimum filtering problem for multi channels is solved in [11] for the first time.
According to [12], if the real signal does not exist, the low resolution observations of the signal can be used for estimating the power spectral density of the stationary random signal.In 2005, multi rate sensor arrays are also developed in [13].High sample rate signal reconstruction by the use of statistical techniques in the presence of low rate sampled noise is investigated in [14].The authors in [15], outline multi-rate filters and study progressive sampling rate transformations and multi-level filtering.Different possibilities of down-sampling and up-sampling are investigated in [16].They also obtain interesting graphical results.
In 2008, authors develop an algorithm which updates adaptive filter coefficients faster in [17].Their algorithm arranges updating speed automatically and relates non-linear relevance between minimum error and updating speed.The output feedback control of the multi-rate sampled systems with output estimator is studied in [18].An approach which uses suitable low resolution samples to estimate power spectral density of wide sense stationary random signal is developed in [19].However, in literature combination of adaptive signal processing methods with multi-rate schemes is an open issue.In this study, I propose to combine adaptive signal processing methods with multi-rate signal processing techniques to provide more efficient signal reconstruction.My approach provides lower mean-square error (MSE) and better estimation performance.
The rest of the paper is organized as follows.Section 2 presents and summarizes multi-rate systems.The multi-rate LMS algorithm and multi-rate NLMS is presented in Section 3 and 4 respectively.Section 5 describes the proposed estimation methods for different input signals.The proposed system and the problem statement are presented in Section 6.The simulation parameters and simulation environment are described in Section 7. Finally, Section 8 discusses the results and concludes the article.www.ijacsa.thesai.org

II. MULTI-RATE SYSTEMS
The observation signals are sampled at different rates in some signal processing applications.These signals should be processed together for detection, prediction and classification.To solve the problems of multi-rate systems, the single rate signal theory should be extended to multi-rate signal theory.This theory should be implemented to single-channel, singlerate or multi-channel, multi-rate problems.In this section, the theory which is developed for multi-rate systems is explained and the basic processes in multi-rate systems are presented.
The changing of the sampling frequency caused problems in many digital signal processing systems.For example, CD players, digital audio tapes and digital broadcasting have different sampling frequencies.Especially, sampling rates of many voice signals should be convertible to each other.Also in some systems, the discrete-time signals with different sampling rates should be made compatible with each other.Separation of wide-band digital signal for transmitting in narrow-band channels is an example for multi-rate systems.
The method of multi-rate signal processing includes decimation and inter leaver.Decimation which includes filtering and down-sampling decreases sampling rate of the signal.Inter leaver which includes up-sampling and filtering increases sampling rate of the signal.There is also transformation of sampling rate process which includes cascade connection of decimation and inter leaver.
For optimal filtering, the estimated signal and observed signal are considered wide-sense-stationary.The sampling rates are equal for both signals and the filter is linear timeinvariant filter (LTI).LTI filter preserves stationarity.However for multi-rate systems, the situation is different.The periodicity is discussed in multi-rate systems because down and up sampling processes vary with time and they do not preserve stationarity.So wide-sense-stationarity becomes crucial.

III. MULTI-RATE LEAST MEAN SQUARES (LMS) ALGORITHM
The least mean square optimum filtering is related to observed data.Desired data sequence and observed data sequence are measured, saved and used for designing the filter in this method.The criterion in least mean square (LMS) algorithm is to minimize the sum of the squares of error function.Multi-rate least mean square filter is designed in [7].By this filter, using two observation sequences provide lower mean square error than using one high-rated or low-rated observation sequences.
Multi-rate LMS algorithm is designed for several input signals with different sampling rates.The equations are more complex than traditional LMS algorithm.( 1) and (2) show the high-rated observation vector and low-rated observation vector respectively.

IV. MULTI-RATE NORMALIZED LEAST MEAN SQUARES (NLMS) ALGORITHM
Multi-rate NLMS algorithm is designed for several input signals with different sampling rates.The equations are more complex than traditional NLMS algorithm.( 5) and ( 6) show the high-rated observation vector and low-rated observation vector respectively.
The filter coefficients are periodic and coefficient vectors are updated in each iteration in multi-rate NLMS algorithm.The filter coefficient updates is expressed in (7) and (8).Note that in here α>0 coefficient is used for preventing divide by zero error.

V. PROPOSED ESTIMATION METHODS FOR INPUT SIGNALS
In this study, the input signals are derived from first-order auto regressive process.It can be defined as in (9).

 x[n]c+αx[n-1] +u[n] 
In the above equation, x [n] shows the value at n th moment, x[n-1] corresponds to the value at a previous moment, c is a constant, α is used for model parameter and u[n] corresponds to White Gaussian Noise.The u[n] is assumed as zero mean and having σ 2 u[n] variance.For |α|<1 the process becomes wide-sense stationary (WSS).If α=1, x[n] has infinite variance and becomes not WSS.For c=0 the process becomes zero mean process.The signal-to-noise ratio can be defined as in (10).
SNR defines the ratio between signal and noise.In here, P signal is the average power of the signal.Both signal and noise power should be measured at the same points in the system.Traditionally in many applications, SNR is used in logarithmic decibel scale.It can be defined as in (11).The proposed system is shown in Fig. 1.The random input signal is derived from first order autoregressive process.Then this input signal is passed through two different filters which are a low pass filter (LPF) and a band pass filter (BPF).The obtained signals are passed through a down sampler after the filtering process.The measurement noise is added to the observation signals and finally this noisy signal is passed through up sampler.After these processes, the observation signals are compared with input signal by the use of LMS and NLMS algorithms.The mean square error (MSE) is minimized and thus the reconstruction of the input signal is completed.
The second input signal is a stereo voice signal.This signal is recorded along 2.02 seconds, it is sampled at 22.05 kHz sampling frequency.Stereo signal has two channels and the component of one of the channels is taken as input signal.This voice signal approximately has 100,000 components, thus instead of processing single bit sequence, the data is processed in terms of data blocks.Then the above mentioned processes are applied to the voice signal and the input signal reconstruction for this signal is obtained.

VII. SIMULATION PARAMETERS
The random signal is obtained from the first order auto regressive process which is given below.

 x[n]0.97x[n-1] +u[n] 
Here, u[n] is selected as a white noise signal which has zero mean and has a variance of 0.0591.The input signal is selected with zero mean and unit variance.To provide 10 dB SNR, the noise variance is taken 0.1 since the input signal has unit variance.In simulations, the 20 dB SNR is also taken into consideration and results are also obtained for this value.Note that, to achieve 20 dB SNR, the noise variance is taken 0.01.The second input signal can be seen in Fig. 2.
According to the calculations in MATLAB, the variance of voice signal is 0.0416.Thus, it is determined that the noise variance should be 0.00416 to achieve 10 dB SNR.Then the filtering process is realized by using below filters ( 13) and ( 14) which show the coefficients of LPF and BPF respectively.In here filtering causes bandwidth restriction.Using two different filters ensure diversity.As shown in Fig. 3, X 2 [n] shows the observation signal which is output of LPF and X 3 [n] shows the observation signal which is output of BPF.These signals are passed through down samplers which have orders 2 and 3 respectively.Then measurement noise is added to these signals.Finally, to obtain estimator signals, up sampling process is implemented.Note that, the main goal of down sampling is limiting sampling frequency.Up sampling also provides index mapping in simulations.In adaptive filtering scheme, the estimator signals are multiplied by filter coefficients in terms of sixtet blocks because the least common multiple of down sampling ratio is equal to six.In both LMS and NLMS algorithms, the initial values of filter coefficients are chosen zero at the beginning of the simulations.To provide best estimation, the LMS and NLMS coefficients should vary periodically in time.The estimation error is calculated using (15).

 e[n]x[n]-x e [n] 
The adaptive filter coefficients of LMS and NLMS algorithms are updated using step-size parameter μ for each step.The step-size parameter is chosen experimentally in algorithms.Hundred iterations are realized to obtain MSE alteration graphic.Note that, for random input signal, in each iteration, the input signal is generated and error between generated signal and estimated signal is calculated.Finally the sum of these error values is divided by iteration number to calculate MSE.www.ijacsa.thesai.org

VIII. RESULTS AND DISCUSSION
The simulations are realized by using MATLAB.Results are obtained for two different input signals by adding different measurement noises.The convergence of LMS algorithm is obtained approximately at iteration 200 as seen from figure 3.At the minimum MSE value, stable learning curve can be obtained.This figure is attained when step size parameter μ=0.005.To prevent instability, the step-size parameter (SSP) is chosen big enough.If SSP becomes very small, each step causes small changes on coefficient vector thus algorithm will work slower.If we choose SSP as very high, then the algorithm may become instable.
Fig. 4 shows the success of the prediction.The estimated signal follows the original signal as close as possible.In here, down-sampling prevents full prediction because after downsampling, the number of samples of data sequence decreases, in other words down-sampling causes data loss.Additive noise also affects the performance of prediction negatively.The convergence of NLMS algorithm is obtained approximately at iteration 100 as seen from Fig. 5.At the minimum MSE value, stable learning curve can be obtained.This figure is attained when step size parameter μ=0.5.Fig. 6 shows the success of the prediction.The estimated signal follows the original signal much closer.Also in here, down-sampling prevents full prediction since after downsampling, the number of samples of data sequence decreases, in other words down-sampling causes data loss.Additive noise also affects negatively the performance of prediction.In addition, when we increase the SNR value to 10 dB, the MSE value increases and the prediction performance decreases for both LMS and NLMS algorithms.The second input signal is stereo voice signal.The LMS and NLMS results are obtained separately.At first, voice signal is turned into single data sequence and it is applied as input to the system.Then the voice signal is separated into data blocks to prevent instability since data size is very high.
The results of MSE and estimation performance of LMS algorithm for complete data sequence of voice signal is shown in Fig. 8.   Approximately after iteration 100, convergence is achieved which is clearly seen from Fig. 10.This result is obtained when the step size parameter μ=0.13 which is chosen experimentally to prevent instability.There are many research topics and open issues in statistical signal processing field.For future work, other methods or algorithms can be explored to achieve better filtering and estimation.Also, my future scope is to implement Recursive Least Squares (RLS) method to the same problem and to compare performance of RLS to that of LMS and NLMS in multi-rate fashion.In addition, the non-integer sampling rates can be used for down sampling and up sampling processes for performance comparisons.These approaches can also be applied for two dimensional signal processing field.Finally, nowadays multi-rate filtering is applied to finiteimpulse-response (FIR) filters, it can be also implemented to infinite-impulse-response (IIR) filters and this is also an open issue in multi rate signal processing literature.

Fig. 3 .
Fig. 3.The MSE of LMS algorithm in SNR=20 dB for the random input signal

Fig. 4 .
Fig. 4. The LMS estimation graph of random input signal at SNR=20 dB

Fig. 5 .
Fig. 5.The MSE of NLMS algorithm in SNR=20 dB for the random input signal

Fig. 6 .
Fig. 6.The NLMS estimation graph of random input signal at SNR=20 dB Fig. 7 shows the joint MSE results of LMS and NLMS algorithms for the random input signal.It is clearly seen from the figure, NLMS converges faster than the LMS algorithm.NLMS has lower MSE than LMS.The main reason for this achievement is that LMS has slow convergence when eigen value spread of input signal is fast.NLMS solves the slow convergence problem of LMS because in NLMS the value of

Fig. 7 .
Fig. 7.The Comparison of LMS and NLMS algorithm in SNR=20 dB for the random input signal

Fig. 8 .
Fig. 8.The LMS estimation graph of voice signal at SNR=10 dB Fig. 9 shows the results of NLMS algorithm for same voice signal.When we examine Fig. 10 and Fig. 11, it is clearly observed that NLMS outperforms LMS in terms of MSE and estimation performance for voice signal.In MSE graphics, first values are very high because at the beginning, the data is unstable.

Fig. 9 .
Fig. 9.The NLMS estimation graph of voice signal at SNR=10 dB To achieve more stable MSE, we should split data into blocks.Data partition also provides easier data processing.

Fig. 10 .
Fig. 10.The MSE of LMS algorithm in SNR=10 dB for voice signal (When data partition exists)

Fig. 11
Fig. 11 shows the LMS estimation performance of partitoned voice signal.Estimation performance increases when we use data partition.The performance comparison of LMS and NLMS algorithms in terms of MSE for voice signal is showed in Fig. 12.It is clearly seen from the figure that, NLMS converges faster than LMS.NLMS has lower MSE than LMS.The main reason for this success is that LMS has slow convergence when eigen value spread of input signal is fast.NLMS solves the slow convergence problem of LMS because in NLMS the value of SSP is normalized by input signal power.Consequently, the dependence of convergence on the input signal is removed in NLMS algorithm and NLMS is superior than LMS in terms of convergence rate and MSE.

Fig. 11 .
Fig. 11.The LMS estimation graphic of voice signal at SNR=10 dB (When data partition exists)