Quality of Service Provisioning in Biosensor Networks

Biosensor networks are wireless networks consisting of tiny biological sensors (biosensors, for short) that can be implanted inside the body of human and animal subjects. Biosensors can measure various biological processes that occur inside the body of the subject under test. Applications of biosensor networks include automated drug delivery, heart beat rate monitoring, and temperature sensing. Since biosensor networks employ wireless transmission, heat is generated in the tissues surrounding the implanted biosensors. Human and animal tissues are very sensitive to temperature increase. Therefore, the generated heat is mitigated by the natural thermoregulatory system. However, excessive transmissions can cause a significant increase in temperature and thus tissue damage. Hence, there is a need for a mechanism to control the rate of wireless transmissions. Of course, controlling the rate of wireless transmissions will lead to Quality-of-Service (QoS) issues like the required minimum delay and throughput. In this paper, we are going to investigate the above issues using the framework of Markov Decision Processes (MDPs). We are going to develop several MDP models that will enable us to study the different trade-offs involved in QoS provisioning in biosensor networks. The optimal policies computed using the proposed MDP models are compared with greedy policies to show their vigilant behavior and viable performance. Keywords—Biosensor networks; Quality of service; Markov decision processes


I. INTRODUCTION
Biosensors can be implanted inside the body of human and animal subjects to form a biosensor network that can be used for monitoring and observing various biological processes and detect anomalies.No processing is done on the biosensors.Therefore, measurements are transmitted to a Base Station (BS) for processing and recommendation of necessary actions.Biosensor networks can be used in daily medical tasks like sensing body temperature, calculating heart beat rate and automated drug delivery.Biosensor networks are powered by either rechargeable batteries or by continuously transmitting energy to them via electromagnetic waves.
Biosensor networks have the same technical challenges introduced by traditional wireless sensor networks.In addition, they introduce new challenges that are unique to them.For example, a major challenge to realizing the full potential of biosensor networks is the heat they generate as a result of power dissipation and wireless communication.Every wireless transmission generates heat.This heat increases the temperature of the tissues that surround the biosensor.The effect of the generated heat is balanced by the human thermoregulatory system.However, excessive transmissions may result in heat that is greater than what can be drained by the thermoregulatory system.If the temperature increase exceeds a certain threshold, the tissues may be damaged.In such a case, the biosensor should be shut down in order for the tissues to cool down and attain the normal body temperature.
As a consequence, the maximum safe temperature level that human tissues can withstand becomes an important factor while operating biosensor networks.Hence, there is a need for intelligent thermal management techniques to mitigate the thermal effect on human tissues.Such techniques, for example, would enable long-term monitoring and measurement to be performed.Furthermore, there is a need for a mechanism to optimize the transmission schedule of biosensors to prevent the potential damage to the human tissues and respect the required QoS.All these contradicting challenges need to be carefully and intelligently addressed.
Very little work has been done in the area of QoS provisioning in biosensor networks.The main focus has been to minimize the average temperature increase of the system with no consideration for QoS [1], [2], [3].On the other hand, QoS issues such as data loss and late delivery are not studied in the context of temperature-sensitive environments like the ones in which biosesnor networks operate.These two specific QoS issues are studied in this paper using a new model that includes the state of the buffer inside a biosensor.In this way, a more accurate picture of the operation of biosensor networks can be painted.
The rest of the paper is organized as follows.Section II provides a survey of the relevant literature.Then, section III describes the newly proposed model.After that, section IV presents the numerical results and several insights.Finally, section V concludes the paper and provides directions for further research.

II. RELATED WORK
The goal of this paper is to extend the models presented in [1], [2], [3] to include some QoS metrics.The current models consider only power and energy constraints with no regard for the effect of traffic and finite buffer size on the performance of biosensor networks.Hence, in this section, we are going to critique the current models and discuss their shortcomings.For more details about the problem and its context, the reader www.ijacsa.thesai.org is encouraged to read our previous papers and the references therein.
Support for various QoS requirements like low packet loss and delay is essential in the development of future wireless networks that employ tiny sensing devices.Several cross-layer optimization techniques have been proposed in the literature to tackle QoS-related issues.For example, the authors in [4] handle the issue of the time-varying nature of the wireless channel by constraining different system parameters like data rate, modulation schemes, and transmission power.The tradeoffs between the average transmission power and average packet dropping probability and the average buffer delay are studied in [5].The authors consider a system with a finite transmission buffer and a time-varying wireless channel.The system is formulated as both a constrained and unconstrained MDP with an average cost criterion.
The heating issue in biosensor networks is addressed in [1], [2].The authors optimize the network lifetime under strict temperature constraints by considering different amounts of initial energy.The system consists of biosensor nodes whose wireless transmission affects the temperature level of the surrounding tissues.The system is modeled as a discrete time MDP that grows in discrete time steps.During each time slot, the scheduled sensor undergoes a change in its energy and temperature in accordance with its action.Temperature of the unaffected biosensors is assumed to decrease by a constant value.However, temperature of the affected biosensors increases according to a direct relationship with the biosensor scheduled for transmission and the state of its wireless channel with the base station.The system is solved to obtain an optimal operating policy that maximizes the network lifetime while keeping the system in a safe temperature zone to avoid tissue damages.The results obtained indicate that the optimal policy performs better when compared to several heuristic policies.Figure 1 shows the system used in the study.
Optimization of biosensor networks by increasing the number of transmitted samples is addressed in [3].Three actions are considered as shown in Figure 2. The control signals are initiated by the base station which also controls the power source.The model is also formulated as a discrete time MDP Fig. 2: A biosensor can be rechargeable.Recharging biosensors can increase their lifetime but it also increases the temperature of the tissues around them.A biosensor can be put to sleep to cool down.whose state includes the current energy, transmission power, and temperature.The temperature is also used as a strict (i.e., global) constraint.The authors evaluate an optimal policy by solving the system using the value iteration algorithm with an average reward criterion.The obtained optimal policy maximizes the samples which can be transmitted by the biosensor network when compared with greedy and heuristic policies.

III. SYSTEM MODEL
Figure 3 shows the layout of the system studied in this paper.Only one biosensor node is shown.Each biosensor has its own state.Multiple biosensor nodes share a common wireless channel that connect them to the base station.Each biosensor node contains a finite size buffer for storing the samples generated by the biosensing elements.These arriving samples may experience delay and loss while traveling to the base station .We assume that each biosensor node knows the state of the wireless channel and the size of its buffer.Hence, the state of the biosensor node is made up of three state variables: wireless channel, buffer size, and temperature.Based on the state of the biosensor, the controller should determine an efficient policy that optimizes certain QoS metrics.Basically, in each time slot, the controller decides whether to make a transmission or put the transmitter to sleep.Next, the details of the system model are given.

A. Wireless Channel Model
We consider a slotted Rayleigh fading channel with Additive White Gaussian Noise (AWGN) N o and channel bandwidth W .The Rayleigh fading channel is assumed to be slowly varying so that the received Signal to Noise Ratio (SNR) remains constant during a single time slot.It is also assumed that transitions are only allowed to current or adjacent states.This slowly varying discrete time Rayleigh fading process can be represented by a Finite State Markov Chain (FSMC) which Fig. 3: System model for a biosensor node with a finite buffer and controller.has K channel states [6].The channel states are numbered from 0 to K − 1.The channel gain for each state c, where c ∈ (0, ...., K − 1), is represented by θ c .The probability distribution for the next channel state during a time slot n is given by P C (c, c ) can be calculated by partitioning the range of channel gains into a finite number of intervals.The information about the fading process given in [7] is used.Further, we assume that the channel state transition probabilities for all channel states are available [8].

B. Buffer State Model
Samples generated by the on-board sensing elements are stored in a finite buffer of size β.Let σ n indicate the number of arriving samples at the beginning of time slot n.Samples arriving in time slot n can only be transmitted in the next time slot n + 1. Sample arrivals are Poisson distributed with an average arrival rate equal to λ.They are also independent of the channel fading process.A truncated Poisson process is considered since the number of on-board sensors is finite.This necessitates an upper bound, represented by Z, on the number of samples.It is assumed that the length of each time slot is equal to one time unit.Hence, the truncated Poisson process can be approximated as follows: It should be pointed out that p(σ n = Z) has a large probability due to truncation.In our model, this means that the likelihood that all sensors generate samples in one time slot is high.
Let B n be a state variable indicating the number of samples in the buffer at the beginning of time slot n.Then, the number of samples in the buffer in time slot n + 1 is given by where A n is the number of samples transmitted in time slot n.

C. Transmission Model
The number of samples transmitted in a time slot n is equal to A n which takes values from the set {0, 1, 2, ..., α}.The transmitter is responsible for taking certain number of samples from the buffer and transmit them over the correlated faded channel.Let A = a 0 , a 1 , a 2 , ..., a A indicate the set of actions performed by the transmitter where a 1 indicates one sample is transmitted , a 2 indicates two samples are transmitted and so on.a 0 represents the sleep action;i.e., no sample is transmitted by the biosensor node in this state.
Let P (C n , A n ) represent the power required to make action A n in time slot n while the channel state is C n .Power required to take a certain action in slot t must belong to P t (c, a) ∈ P op , where P op indicates the set of power levels supported by the transmitter.Furthermore, we enforce a fixed Bit Error Rate (BER) constraint on all the transmissions done by the transmitter.Assuming an adaptive M-ary Quadrature Amplitude Modulation (MQAM) modulation scheme with ideal coherent phase detection, the power required to satisfy a particular BER can be evaluated by using the following equation from [8]: In ( 5) N o represents the channel noise, E b represents the fixed BER constraint that is satisfied assuming coherent phase detection, θ c represents the channel gain when the channel state is c and W represents the bandwidth of wireless transmission.If the required power is less than that described in (5) , it means that action is not feasible.Power calculated in (5) give a pessimistic estimate of the power required to achieve a certain BER for different channel states and actions.
In each time-slot the biosensor node's rate of transmission can be calculated by where Φ represents the number of bits per symbol used for transmission of A n samples during F channel uses.G represents the size of incoming samples in terms of bits.
If we set G = F , the rate will be equal to Φ.We can transmit different number of samples by changing the number of bits per symbol.If we set number of bits per symbol equal to number of samples transmitted in a time slot, then Φ(A n ) = A n ; i.e., the transmission rate becomes equal to the action suggested by the optimal policy.

IV. MDP FORMULATION
The global state of the system, denoted by S, consists of three variables and the state space is given by where T is the temperature state variable.The size of the state space is thus the product of the number of channel states, number of buffer states, and number of temperature levels.In this section, two MDP formulations are given.They differ in whether the temperature is part of the global system state or a constraint.
An important element of any MDP formulation is the sytem state transition probability matrix.This matrix describes how the system transitions from one state to another.We assume that state variables are independent.Thus, the state transition probability matrix of the system can be calculated by simple multiplication of the transition probabilities of the channel and buffer state variables.The temperature state variable plays no role in the computation of the state transition probability matrix of the system.This is because it is not random.
Hence, the following equation gives the state transition probability matrix of the system.
where s, c and b represent the current state of the system, wireless channel and buffer, respectively.On the other hand, s , c and b represent the next state of the system, wireless channel and buffer when action a is performed.The next state of the wireless channel is independent of the current action.
The current action a determines the next state of the buffer only.
The solution of an MDP formulation is referred to as a policy which is a mapping from the system state space to action space.That is, a policy determines the best action that should be performed in each possible state of the system.An optimal policy guarantees an optimal behavior of the system.
Two objectives are considered.The first one is to minimize the expected long-term average transmission power.
where π(s i ) represents the action suggested by policy π and P (s i , π(s i )) is the instantaneous transmission power.
The second objective, however, is to maximize the expected long-term average transmission rate.
where R(s i , π(s i )) represents the instantaneous transmission rate.
An important performance metric is the average loss rate which represents the expected number of samples that are dropped due to buffer overflow.The following equation shows how the number of samples lost in time slot n is computed for a specific state s and action a.
The average number of lost samples can be computed using the first moment as follows.
The instantaneous delay during a time slot n can be computed as follows.
where b n is the instantaneous buffer size during time slot n.
The expected long-term average delay is the following.
Finally, our thermal model is discussed.In this model, the increase in temperature is directly proportional to the magnitude of the action.For example, transmitting one sample during the best channel state (C n = 0) will increase the temperature by one unit.The following equation is used for computing the instantaneous temperature increase.
The long-term average temperature is mathematically expressed as follows.
Notice that the expectation operator is dropped since temperature is not a random variable.
Next, the details of the MDP models are given.First, in the average thermal increment model, the problem is formulated as a constrained MDP model where a particular objective function is optimized while putting various constraints on other QoS metrics .The first MDP formulation maximizes the system transmission rate while keeping the average power, delay, thermal increment and loss rate within given bounds.The second MDP model, on the other hand, optimizes the system power consumption while respecting a minimum transmission rate and keeping the biosensor network in a safe operating zone.Let x(s, a) indicate the decision variable in solving the MDP models obtained in previous section.x(s, a) represents the steady state probability distribution when the system is in state s and action a is performed.Based on different rewards and depending on the QoS parameters, we want to optimize x(s, a) to obtain an optimal policy which describes what action to take when the system is in state s.The MDP model proposed is solved using the LP algorithms in MATLAB [11] to obtain optimal operating policies for correlated wireless channel.The default mode for LP solver is to minimize the reward function.
Since the problem is formulated as an average cost constrained MDP, there are certain basic constraints that must applied for each implementation.
The first constraint ensures that the x(s, a) is a probability distribution with its sum over all pairs of system states and actions equal to one.The second constraint ensures that we are solving an average cost constrained MDP.The third constraint enforces that the decision variable x(s, a) is always positive.These basic constraints are common to all the LP models given in this paper.
The first LP model is about the maximization of the transmission rate (i.e., throughput).The details of the model are as follows.In the next LP model, the objective is to minimize the average transmission power and use the other metrics as constraints.The following are the details of the model.
The first constraint ensures that there is a minimum average throughput.The remaining two constrains put an upper limit on the loss rate and delay, respectively.

B. LP Formulation for the Strict Temperature Model
The LP formulation of the strict temperature model is similar to that of the thermal increment model discussed above.However, the reader is reminded that the system state now includes the temperature as a state variable.This represents a global constraint.Thus, there will be no explicit constraint on the temperature increase like in the previous LP models.The following are the details of the new LP model.

C. Finding the Optimal Policy
After solving the above LP models, a probabilistic distribution over the state-action space is obtained.We would would like to find a policy that tells us what action should be performed in each system state with a probability of one.This can be achieved as follows.π * (s, a) = x * (s, a)

V. RESULTS AND DISCUSSION
In this section we numerically solve the model proposed in the previous section to obtain the optimal policies and then simulate them.In our simulation, we are going to analyze the effect of various QoS constraints on the optimal policies.Then, we study the different optimal policies obtained by solving the average thermal increment and strict temperature models.The thermal behavior of the obtained policies is also discussed.The following system parameters are used in the model formulation and simulation.They are also described in Table II.Arrivals at the buffer input are assumed to be Poisson with an average arrival rate of three.Buffer size is set to eight samples.Eight channel states are considered.The state zero is assumed to be the worst with a very small gain.There are eight possible actions in each state of the system;i.e., transmitting from one up to seven samples or no transmission.Based on these system parameters, the MDP model is formulated as a linear program and solved using MATLAB.The slowly varying Rayleigh model is described in Table I.It has an average power gain of 0.8 and a Doppler frequency of 10 Hz.

B. Analysis and Insights
For the purpose of analyzing the effect of various constraints on the optimization of average transmission rate and average power consumption, we vary the magnitude of the constraints on the average loss rate, delay and thermal increments to study their effects on the objective function.Values of the input parameters are also varied and their effects on both the constraints and objective function are studied.
First, the LP model expressed by equations (25)-28 is studied.Figure 4 shows the effect of varying L O .It can be seen that the average transmission power decreases as the average loss rate increases.Since more samples are allowed to drop when the loss rate constraint is increased, the optimal policy will use the least amount of power possible for transmission.Also, increasing the arrival rate increases the average power consumption of the system.This is because there will be more samples in the buffer which need to be transmitted.Figure 5 shows the effect of varying the delay (i.e., D O ).It can be seen that the value of the optimal average transmission power decreases as the average delay constraint is increased.This indicates that as the constraint on the average delay is increased, samples are allowed to experience more delays which results in a lesser average power consumption.
The effect of changing the average arrival rate λ on the  Fig. 6: The increase in the minimum average transmission rate constraint (R O ) causes an increase in the optimal average transmission power utilized.average delay constraint is studied next.Figure 5 shows the variations in the average delay and optimal average transmission power due to different arrival rates.The delay and average arrival rate have an inverse relationship.For example, for a fixed D O , the left side of the delay constraint in equation 28 will be reduced if we increase the average arrival rate.This in turn should increase the optimal average power consumption in order to achieve the same delay constraint.By contrast, the behavior observed in Figure 5 is the opposite.This can be explained by the fact that the delay is directly proportional to the buffer occupancy while it is inversely proportional to the average arrival rate.So, based on the insights obtained from Figure 5, we can conclude that the effect of the increased delay dominates the reduction achieved by increasing the average arrival rate which in turns reduces the average power consumption.
We next study the effect of having a minimum average transmission rate requirement on the optimization of average power.The behavior obtained after applying the minimum average transmission rate constraint in equation ( 26) is shown in Figure 6.It can be seen that as the value of the constraint increases, the optimal average power consumption increases.This happens because the increase in the minimum average transmission rate constraint requires that the biosensor node transmits more samples.As a result, the optimal value of average power consumption increases.
Next, the LP model expressed by equations (20)-( 24) is studied.In the same way, the value of P O is varied.The results are then plotted in Figure 6.It can be seen that the optimal average transmission rate increases as the average transmission power P O increases.This indicates that as the constraint on average power is increased, more power is available which can then be used to transmit a larger number of samples.Of course, this will result in higher transmission rates.
The effect of increasing the arrival rate on average transmission rate is depicted in Figure 7.It can be seen that as Fig. 7: The effect of increasing the average arrival rate (λ) on the optimal average transmission rate as the average power constraint (P O ) increases.
the average arrival rate increases the average transmission rate decreases.This is due to the fact that any increase in the average arrival rate causes an increase in the loss rate which in turns reduces the average transmission rate of the biosensor.
Maximization of the average transmission rate can cause the temperature of the system to increase by a large amount.The minimization of the average transmission power indirectly minimizes the system's thermal state increment by minimizing the power consumption.However, for the maximization of the average transmission rate, we need to explicitly include a constraint that controls the increase in the thermal state of the system at symbol level.In order to study the effect of the constraint in equation ( 23), the value of T h is varied to obtain various optimal policies.The results are then used to calculate the optimal average transmission rates.Figure 8 shows that the average transmission rate increases as the average thermal increment increases.This is at the cost of damaging the tissues, of course.So, we should try to keep the thermal increase constraint as small.
It should be pointed out that a change in the average delay constraint does not affect the average transmission rate.The reason for such behavior is that the delay depends on the buffer state and the average arrival rate.If we keep the average arrival rate constant, the delay becomes directly related to the state of the buffer.But, changes in the buffer state also cause similar changes in the transmission rate.As a result, the optimal average transmission rate stays constant as the average delay constraint is varied.However, if we increase the arrival rate at the input of the buffer, the average loss rate and the delay both increase.This will cause a reduction in the optimal average transmission rate as shown in Figure 9.

C. Optimal Policies for the Thermal Increment Model
In this section, we study the thermal increment model and how the thermal increment constraint affects the optimal 0.5 Fig. 9: Increasing the value of the average delay constraint does not have any effect on the average transmission rate.However, it decreases as the average arrival rate increases.

policies.
The optimal policy that results from solving the LP model in equations ( 25)-( 28) is plotted in Figure 10.The minimum average transmission rate constraint R O is set to 0.07, average delay constraint D O is set to 10 msec and average loss rate constraint L O is set to 2 Samples.The 3D plot indicates that as the channel state improves, the policy suggests to make a transmission.Similarly, an increased number of samples in the buffer also indicates that the transmitter should start sending more samples to the base station.However, since the objective is to minimize the average power consumption and the minimum average transmission rate constraint is quite small, a maximum of one sample is transmitted even in the best channel state.This has the advantage of reducing the temperature increase of the biosensor node.However, if we increase the minimum average transmission constraint to 0.35,  it can be seen in Figure 11 that the number of samples transmitted as the buffer state improves is increasing.
The optimal policies obtained from the different LP models have unique behaviors.They are observed to be monotonically increasing in the channel and buffer state of the system.This means that as the channel state improves or the buffer state increases, the optimal policy also increases monotonically.
When embedding these policies into an actual hardware, we can define the actions in terms of increasing values of channel and buffer state information.The controller can make an easy decision based on these thresholds defined by the optimal policy.This behavior can thus help in the practical implementation www.ijacsa.thesai.org of these optimal policies on biosensor hardware.
The optimal policies computed in the previous section are simulated using MATLAB and the results are compared with a greedy policy.In the case of the transmission rate maximization, the greedy policy works on the principle that it always tries to transmit the maximum number samples that are allowed under the given system state without exceeding the constraints of the average loss rate, average thermal increment, average delay and average transmission power.As for the transmission power minimization, the greedy policy works by transmitting the least number of samples possible without violating the required average transmission rate.Each data point is the result of running the simulation five times.Both policies are simulated for a different number of time slots and their results are compared.The performance of the average transmission rate maximization policy against the greedy policy is shown in Figure 12.Clearly, this Figure indicates that the optimal policy outperforms the greedy policy in terms of the total number of transmitted samples.

D. Strict Temperature Model
In this section, we are going to study the LP model expressed by equations ( 29)-(32).The obtained optimal policy is simulated and the temperature variations are observed.Similar to the previous approach, a comparison is performed with a greedy policy.The conclusion is that the optimal policy provides better performance.
We choose four temperature levels to represent the temperature states in the model proposed for the strict temperature model.The lower and upper bounds on the temperature are set to 37 o C and 40 o C. The number of channel and buffer states are set to eight, respectively.The average arrival rate at the input of buffer is set to three.The optimal policy allows transmissions only when the temperature is in state one.For higher temperature states, the policy chooses the sleep action to keep the thermal state of the system within the provided constraints.Again, the behavior of the optimal policy is observed to be monotonic in the channel and buffer states.The policy ensures that more samples are transmitted as the state of the wireless channel and buffer improves.The average temperature and power constraints are also kept within bounds.It is also observed that when the temperature is in its worst state, the policy suggests not to transmit any samples in order to save the biosensor from going into the highest thermal state.Therefore, the optimal policy is also monotonic in terms of the temperature states.
The optimal policy computed for the transmission rate maximization problem is also compared with a greedy policy that satisfies the constraints given in the model.The greedy policy always tries to transmit the maximum possible number of samples while respecting the QoS constraints.A running average for all the constraints is used to make the decision in each time slot.The simulation is run five times for each number of slots and the average results are calculated.Figure 13 shows the results obtained by running the simulation for up to 10000 time slots.The results indicate that the optimal policy again outperforms the greedy policy in terms of the total number of transmitted samples.However, the difference between the two is small as compared to the optimal policy for the previous average thermal increment model.

VI. CONCLUSION
In this paper, the problem of QoS provisioning in biosensor netoworks has been studied using the framework of MDPs.The newly proposed model captures the interaction between the wireless channel and buffer at a biosensor node.The obtained policies maximizes network throughput and lifetime under several QoS constraints.They are also monotonic which means that they can be easily realized.Further, the simulation of the thermal behavior of the optimal policies indicate that the strict temperature model provides a better control over temperature increase when compared to the average thermal

Fig. 1 :
Fig. 1: Biosensors are implanted inside the body of a human to collect physiological measurements and transmit them over a wireless channel to an access point for further processing.
, a) × P (s, a) ≤ P O (21) s∈S a∈A x(s, a) × L(s, a) ≤ L O (22) s∈S a∈A x(s, a) × T (s, a) ≤ T h (23) s∈S a∈A x(s, a) × D(s, a) ≤ D O (24) The constraints in (21)-(25) makes sure that the average values of power consumption P (s, a), loss rate L(s, a), thermal increment T (s, a) and delay D(s, a) do not exceed their thresholds P O , L O ,T h and D O , respectively. min a) × R(s, a) ≥ R O (26) s∈S a∈A x(s, a) × L(s, a) ≤ L O (27) s∈S a∈A x(s, a) × D(s, a) ≤ D O max a) × P (s, a) ≤ P O (30) s∈S a∈A x(s, a) × L(s, a) ≤ L O (31) s∈S a∈A x(s, a) × D(s, a) ≤ D O (32) where x(s, a) represents the decision variable for the optimization of average transmission rate.P O , L O , T h and D O represent the thresholds on the average transmission power, loss rate and delay, respectively.

Fig. 4 :
Fig.4: Reduction in the optimal average transmission power as the average loss rate constraint (L O ) is varied.

Fig. 5 :
Fig. 5: The optimal average transmission power decreases as the average delay constraint (D O ) increases.

Fig. 8 :
Fig.8: The optimal average transmission rate increases as the average thermal increment (T h ) constraint increases.

Fig. 10 :
Fig. 10: Optimal policy for minimizing average power consumption with R O = 0.07, D O = 10 msec and L O = 2 Samples.

Fig. 11 :
Fig. 11: Increase in the minimum average transmission rate constraint (R O = 0.35) results in an increased number of samples transmissions in the optimal policy

Fig. 12 :
Fig. 12: Comparison of sample transmissions for different policies with a varying number of time slots.
Fig. 13: Comparison of sample transmissions for different policies achieved by the average transmission rate maximization.

TABLE II :
Simulation parameters.

TABLE I :
Channel states and transition probabilities.