A Sleep Monitoring System with Sleep-Promoting Functions in Noise Detection and Sound Generation

Recently, there has been a growing demand and interest in developing sleep-promoting systems for improving sleep condition. Because sleep environments are various, and sensitivity to noise differs individually, it is difficult for current sleep-promoting systems to provide an adoptable solution. This paper develops a non-invasive sleep monitoring system with adaptive sleep-promoting sound according to sleep environments and sleep habits. For people who fall asleep in a quiet environment, a constant sound playing probably affects their sleep. The proposal is designed to distinguish the noise disturbances, and a sleep-promoting sound is triggered automatically. A device with multiple sensors: an infrared depth sensor, a RGB camera, and a four-microphone array, is used to detect sleep disturbances. When a noise is detected, an ambient sound is playing to cover the noise automatically. Besides, it also applies to people who are used to sleep with sound by providing additional sound playing from the beginning of their sleep. Moreover, from the input of depth signals and color images, the scores are calculated from the sleep information, and are record for sleep quality evaluation. An overnight experiment was carried out, and the results show the efficiency of the proposed system in diverse sleep environments. The adaptable method is feasible for individuals, and it is also convenient and cost-effective to be used in home context. Keywords—ambient sound; image sequence analysis; noise detection; non-invasive sleep monitoring; sleep promotion


INTRODUCTION
This paper develops a system for generating sleeppromoting sound automatically based on noise detection in sleep monitoring by using a device with multiple sensors.
Currently, sleep-promoting system are popular in modern society, such as smartphone applications and commercial products.The numerous sleep promotion systems show the need of people to create good sleep condition.To better understand an ideal sleep environment, some researches in social science has indicated the importance of tranquility to most of people [1][2] [3].Furthermore, darkness is also an important sleep promoter that low light condition endogenous melatonin production [4].However, it is hard for current sleeppromoting technology to take whole control of the lighting factor in home context.Considering other condition, the comfortable temperatures of people for sleep are not all the same, and a suitable firmness bed is also different by shapes and sizes of people [5] [6].Besides, the effect of smells on sleep experience of people has not been extensively study.Soundbased sleep promotion has become the majority in current sleep-promoting systems, for the purpose of creating a relaxing atmosphere.Hence, sound condition will be addressed in this study.
A sound-based sleep promoting system is absolutely necessary for people to create good sleep condition.The sleeppromoting sound is also closely related to sleep environments and sleep habits.To our best knowledge, this problem has been seldom investigated in the literature.Noises from outside traffic, or people talking in the street, that causes difficult sleeping.But a snoring partner who sleeps in the same room is a more serious noise problem [7].On the contrary, some people probably have difficult to fall asleep, because it is too quiet.They prefer sleeping with a sleep company, such as turning on a radio or television, raining and wind whistling form outside of window, or playing a soft music in the background.From the observation, the sensitivity to noise differs individually.Because sound is the only sensory system that remains truly active during sleep, and a brain naturally craves sensory input [8].A quiet night also enhances the stimulus from the sleep environment, because any random noises probably active the restless brain.In both cases, the evidence has shown that white noise and rhythmic sound may help to mask other noises in a noisy room [9].Besides, it also gives a brain a tonic signal that dampens its own internal system in a quiet room.From the mechanism of a brain working, sound is the major factor to interrupt sleep.On the contrary, it also can be used to mask environmental noises [10].People who fall asleep easily in a quiet room only need sleep-promoting sound when noises happening, that a constant sound playing probably causes negative effect to their sleep.However, to those usually sleep with sound, the sleep-promoting sound should play continuously until they fall asleep.Besides, they also need the sound mask when noises happening during sleep.Moreover, these sleep-promoting sound should be adjust to moderate volume in sleep condition.Some researches has shown that overly loud white noise significantly reduces REM (Rapid Eye Movement) sleep in a variety of animals [8].Hence, the soundbased sleep promotion helps to improve sleep condition of people.Besides, for the variety of sleep environments and the diversity of sensitivity to noise of people, an adaptive method is in an urgent need.
It is possible to provide adaptive sleep-promoting sound if noise disturbances can be detected.Then when the noises happen, they are covered by sleep-promoting sound immediately.Currently, there is not any automatic method in sleep promotion systems to detect sleep disturbances.www.ijacsa.thesai.orgIn previous work, a method has been proposed for distinguishing occurrences of disturbances from a sleep environment [11].Furthermore, the method can be used to identify the appropriate time for sleep sound promotion.
In this paper, a sleep monitoring system with adaptive sleep-promoting sound by an epoch approach method has been developed to detect the occurrence of sound events.To our best knowledge, the proposed system is the first adaptive soundbased sleep promotion system, which is triggered by the time estimation from the noise detection in sleep monitoring.Given a stream of audio signals, the audio signals are scored in a constant interval by an epoch approach method [11].When the noise is detected, an ambient sound is playing to mask the noise automatically.Meanwhile, the motion behavior and lighting changes are scored, and the depth images and color images are recorded simultaneously.Finally, an interface with sleep diagram is provided for sleep examination, which specific section can be expanded to browse the corresponding video clips showing in Fig. 1.For people who fall asleep in quiet sleep condition, the proposal is designed to distinguish noise disturbances, instead of constant sound playing which may affects their sleep.Besides, it also applies to people who are used to sleep with sound by providing additional sound playing from the beginning of their sleep.The results of an overnight experiment show the efficiency and flexibility of the system.This paper is organized as follows.Section II reviews the related works.Section III presents the proposed adaptive sleeppromoting method.The experiment and evaluation are discussed in section IV.Section V provides the conclusion and future work.

II. RELATED RESEARCH
In the section, the related research is reviewed.Objective of this research is to propose a method for sleep promotion and sleep quality improvement toward heathy people or some with sleep problems.Sleep disorder is one of the biggest modern plague in the twenty-one century, and is harm to health and quality of life of people.Not only people who with sleep disorders, but also heathy people may have difficulties falling asleep occasionally [12] [13].Under the promise that total sleep time is not reduced, even a mild sleep disturbance suppress deep sleep that is sufficient to affect memory performance of healthy people [14].Therefore, the World Sleep Day has been named on March 14 by WASM (World Association of Sleep Medicine) to arise the attention from public to sleep issues.In spite of the medicine treatment is still widely used, a common nonpharmaceutical treatment is using sound as sleep promoter.Because sound is the only sensory system that remains active in sleep [8].Hence, it is feasible to provide sound-based sleep promotion system for sleep experiences improvement.

A. Sound-based Applications in Sleep Promotion
A large number of mobile applications with sleeppromoting sound had been proposed, and sounds from nature environmental materials are widely used in current systems.Sleepmaker Rain collected real rain sound recording from World Heritage Listed Forests in Tasmania, Australia.It allows users to choose their favorite rain sound to sleep with, such as drizzling rain, pouring rain, or thunder rain.Nature Sound Relax and Sleep collected more all natural sounds without any music composition, like song of birds and winter snowing.aSleep considered both natural sounds and background music with adjustable speed.Moreover, Relax Melodies, and Relax and Sleep provided users with large amount of ambient sounds to organize their own playing lists.Some researches started to focus on the correlation between binaural beats frequencies and brainwaves.The evidence show that binaural beats frequencies slow brainwaves, and that help people to fall asleep [15].From this point, both Relax Melodies and Pzizz considered binaural beats frequencies.In addition, Pzizz generated new harmonious soundtrack through neuro-linguistic programming with every time use.Proactive Sleep Alarm Clock expanded sound database by using the iTunes songs of users as sleep promoter, and also provided a sleep report for sleep examination.Besides, it not only considered an objective measurement to monitor body movements during sleep through an accelerator of a cellphone.But also a subjective measurement to record daytime activities of people for proactive prediction.In addition, all these sleep promotion applications can be used for meditation or napping at daytime, or for relaxing sleeping at night.www.ijacsa.thesai.org

B. Sound-based Commercial Products in Sleep Promotion
Some commercial products using multimedia as sleep promoter has been announced.SleepPhones used a soft headband with removable speakers, and the firm fabric helps to block the noises from an environment (http://www.sleepphones.com/).The headphone can be plugged in devices of users to play their own music, and the devices can be hidden under their pillows.In addition, a commercial CD with ambient sound selling on their website also provided a good option.This attached product needs people to wear to bed, some people may have uncomfortable sleeping.Moreover, Philips Wake-up Light created a bedside lamp that combined with natural sounds and lighting control to stimulate an environment with nature sunlight (https://www.usa.philips.com/c-p/HF3520_60/wake-up-light).The evidence has shown the correlation between lighting and melatonin production, and that is related to sleep qualities of people [4].It gradually decreases sound and light for providing a relaxing sleep environment, then gradually turns into a bright warm light to wake up users naturally in the morning.Both products are convenient for people to use at home for sleep promotion.

C. Disturbances Detection in Sleep Monitoring
Many studies had aimed on an activity-based technique for monitoring a subject in bed.However, only few researches emphasized environmental factors in sleep condition.In previous work, a sleep monitoring system has been developed to identify three kinds of events in a sleep environment: sound events, lighting events, and motion events [11].Another monitoring system also considered the temperature and humidity changes in a room, and sleep postures of a subject, but noises factor was disregarded [16].The major problem of the system is that the providing posture detection is based on the skeleton recognition from Microsoft Kinect, it is hard to detect a correct posture if a subject sleep with quilt covering.That is hard to apply to real sleeping.
Table I compares the functions of related works and the proposed system in the paper.These works used sound as sleep promoter to improve sleep experiences of people.The providing sound for sleep promotion can be classified into three types: natural sounds, background music, and synthesized soundtracks.Except SleepPhones is a wearable product, the rest of them do not need to attach on the bodies of people during sleep.In addition, most of them have an alarm to end up sleeping.Because the purpose of the proposed system is to record normal sleep of users, it is not offering an alarm to interrupt their sleeping.Only sleep monitoring systems considered the influences of environmental factors on people.Besides, both the two sleep monitoring systems and Proactive Sleep Alarm Clock provided a sleep report for sleep examination.From the comparison on Table I, the proposed system in this paper is superior to these sound-based sleep promotion systems by providing an adaptive promotion solution.
In this section, the related research has been reviewed.From the discussion, various sound-based system had been developed for sleep promotion.However, they did not well meet users need.In this paper, both the technologies of monitoring and promotion in sleep condition are considered, to identify the appropriate time for sleep promotion.Moreover, a sleep report with sleep diagram and sleep quality evaluation is also provided for users to examine their sleep.www.ijacsa.thesai.orgIn this section, the system flow of the sleep monitoring system with adaptive sleep-promoting sound will be described.This section reviews the core components of the proposed system, the method for promotion estimation, the automatic sleep-promoting sound, and the sleep quality evaluation.Fig. 2 shows the flow chart of the proposed system.In the following, an epoch approach method is introduced to estimate appropriate time of sleep promotion, which is based on noise detection in sleep monitoring.Then a sleep-promoting sound is triggered automatically to cover noises in an environment according to the estimation.Finally, a sleep/wake identification method is adopted to calculate motion information from the sleep recording for sleep quality evaluation.

A. An Epoch Approah Method for Promotion Estimation
This section explains the method for promotion estimation that can be used to identify appropriate time for sleep promotion during sleep.Given a stream of audio signals, the audio signals are scored and normalized to a scale from 0 to 1.In preliminary stage, the empirical threshold was set from audio sensor data.An epoch approach method was proposed to determine a continuous event at a constant interval [11].Using an epoch length of 3 seconds, once an audio score in the interval is over the empirical threshold, a noise is detected.
There are two different aspects in sleep promotion and sleep monitoring , and two empirical thresholds are set in audio signals to classify degree of noises: audio promotion threshold and audio detection threshold.The audio promotion threshold (0.15) is used to detect sounds over 40 dB.The table from ASHA (American Speech-Language-Hearing association) declares the decibel levels of common environmental sound (http://www.asha.org/public/hearing/Noise/).The 40dB sounds is in moderate degree, such as talking in low voice.In general, a sleep environment under 40dB sounds is quiet that the sound do not wake up people.However, people may have turning behaviors, small twitches during sleeping under the feeling of noise disturbances.Therefore, the audio promotion threshold is used to estimate the appropriate promotion time when a noise over 40dB is detected.Another threshold (0.6) is used to detect remarkable sound over 60dB.The 60dB sound is about the volume of normal conversation voices and that has already may startled people from sleep.
The audio detection threshold is used to identify and record specific events for sleep examination.Once a sound event is triggered, the audio signals and the corresponding depth images and color images are recorded simultaneously to generate video clips.The length of the event is at least with 4 units of epoch lengths (12 sec.)showing in Fig. 3.The continuous event completely record the occurrence of an event.Besides, the report from ASHA also mentioned the long-term effect of noise problem in sleep condition, that people easily have cardiovascular disease sleeping under 50dB sound.Moreover, to sleep under 70dB sound will increase 30% incidence of MI (Myocardial infarction).Hence, the effect of noise disturbances should not be ignored in a sleep environment.
To consider other disturbances in a sleep environment, the epoch approach method is also applied to determine a motion event and a lighting event.It iteratively aggregates data in a constant interval in (1) [11].

{
(1) [11] where , , = the activity count of the preceding epoch, the current epoch, and the following epoch.The empirical thresholds from depth sensor and color sensor data was set in the preliminary stage.Given streams of depth signals and color images, one GMM background model is established in depth signals, and another GMM background model is established in RGB images [17].After morphological smoothing, the noises of segmented foreground objects are reduced, then the area of foreground objects are scored and normalized in the same scale.The depth score and color score are considered as magnitude of movements, and magnitude of lighting changes respectively.After that, the epoch approach method iteratively calculates number of scores over the empirical thresholds.Then recording of motion events and lighting events are triggered when activity count of current epoch is bigger than activity count of the previous epoch.Through this way, the sparse score value is discarded for that is not able to be considered as an event.Hence, a sleep diagram is provided with various sleep information for sleep examination.www.ijacsa.thesai.org

B. Automatic Sleep-promoting Sound
This section explains two modes of sleep-promoting sound and the providing sound materials.From the promotion estimation, the appropriate time for sleep-promoting sound playing has been identified.Moreover, for people have different sensitivity to noises, two modes are provided for general people and some who prefer to sleep with sound showing in Fig. 4.

1) People sleeping with quiet:
The sleep-promoting sound is only playing according to the estimation time from the poposed epoch approch method.The sleep-promoting sound plays 5 minutes according to the time index recording to cover and smooth noises.
2) People sleeping with sound: To people who are used to sleep with sound, the steps of sound promotion as follows:  At the beginning of sleep, the sleep-promoting sound plays 20 minutes to help them fall asleep.
 During sleep, the promoting sound plays 5 minutes according to the time index recording from the promotion estimation method to cover and smooth noises.
Besides, ambient sounds from natural materials are provided by default, such as calm sea waves, dripping rains, pouring rain or thunderstorms, and users are also allowed to sleep with their own soundtracks.Hence, the adaptive sleep promotion system is realized through the method of promotion estimation by both considering sound-based sleep promotion and sound detection in sleep monitoring.

C. Motion Information for Sleep Quality Evaluation
A historical activity-based method, actigraph, had been proposed by Tryon for sleep quality evaluation [18].It is a statistical model from sampling activity data to identify sleep or wakefulness state.At first, a device based on the method, actigraphy, is a wearable device with accelerator and light sensors to monitor activities of people during sleeping.The correlation between motions and sleep states has been validated in clinical field.There is a strong correlation of 0.81 between measurement of total sleep time from actigraphy and that from PSG (Polysomnography) [19].
The three major methods of sleep/wake identification are listed in Table II, and they can be compared with each other for they all have similar parameters in their equations, and two of them using the same adjustment rules [20] [21] [22].The accuracy of Webster algorithm determined by comparison sleep/wake scores with EEG (Electroencephalography) data, and the accuracies of Cole algorithm and Sadeh algorithm agreed with PSG data.Nowadays, many wearable devices use the sleep/wake identification algorithm for sleep quality evaluation.Only the sampling data of Webster had been eliminated for the improvement of device, the algorithms of Cole and Sadeh are still popular used in clinical diagnosis.In this paper, Sadeh algorithm is adopted to evaluate sleep quality for the highest accuracy and the simplest rescoring rule.
One rescoring rule with 1-minute scoring interval is used to improve the accuracy.There are two 30 seconds epochs in the interval of one minute.If both sleep state and wake state present in the same minute, then the minute is rescored as wake state.There is high possibility to be wakefulness if wake state occurs in this short period, and that helps to avoid the false wake problem.Moreover, the algorithm has been verified the tolerance of device sensitivity by adding a constant value or multiplying a constant ratio with the activity count value, and that did not affect the accuracy of the result.Hence, the method can be adopted with a totally different device.From the motion information of the device, the sleep or wakefulness state of sleeping is identified, then the SE (sleep efficiency, total sleep time/total time in bed) is calculated for sleep quality evaluation.

D. Summary
In summary, this section described the method to realize adaptive sleep promotion in the proposed sleep monitoring system.Moreover, an activity-based method to calculate sleep efficiency is applied to evaluate sleep quality.In next section,

A. Experinment Settings
The samples of 5 subjects includes 4 men and 1 woman, ranging in age between 34 to 40 years (mean age=35.6;SD=2.88).The samples includes 5 normal adults and 1 patient who has been diagnosed with sleep apnea (age: 40).All participants were volunteers to participate in the study, for access the effects of an adaptive sleep promotions and discover the disturbances in their sleep environments.
Experiment space was set in home scenario that all participants were asked to sleep on their own beds.It avoids the effect from an unfamiliar environment which usually causes trouble sleeping, and also helps us to collect various sleep information from different sleep environments.All participants were allowed to keep their sleep habits, such as turning on a small lamb.There is no fixed sleep time in the experiment setting, and they were asked to get to bed and wake up as usual.For testing the effect of the adaptive sleeppromoting sound, three conditions were tested in the experiment as follow: 1) Participants sleeping with original sleep condition: All participants sleep in their usual sleep envionmtents without any interventions by deault.
2) Participants sleeping with sleep-promoting sound: All participant sleep with sleep-promoting sound when the noises are detected in their sleep enviornments.
3) Participants sleeping with sleep-promoting sound and noise interuptions: All paritciapnt sleep with sleep-promotion sound when the noises are detected in their sleep environment.In addiotn, noise interruptions is setting in a constant interval during their sleeping to test the effect of sleep-promotiong www.ijacsa.thesai.orgsound, especially for the participants whose rooms are usually quiet with few noise problems.Four kinds of noise intruptions with a runinig truck, going down stairs, road-works constructing, and a howling ambulance were playing in one hour, one and half an hours, two hours, and two and half an hours respectively counting from the start of their sleeping.
The experiment was carried out with blind test.All participants did not know the applying conditions above.The sysetm implemented a program with one of the condtions randomly during the experinment period.In addtion, the time index of sleep sound promotion were recored.During their sleeping, all the participants were asked to answer a question from the system to record the reason of their arousing once they woke up.The exepriment period is from May 20 to July 24 in 2015.There are 36 nights of recording in the experiment. 1 subject spent 9 nights, 2 subjects spent 8 nights, 1 subejct spent 7 nights, and 1 subject spent 4 nights.The subjects slept between 11:22 P.M. to 03:45 P.M..A total of 13,573 minutes data were collected.

B. Evaluation Results and Analysis 1) Participants sleeping with original sleep condition:
Firstly, the original sleep condition of the participants will be discussed.Table III (Condition: O is original condition, P is promotion sound, and PN is promotion sound and noise interruptions) shows the influences of noise disturbances in their sleep environments, and according to it their sleep environments can be classified into a quit sleep environment or a noisy sleep environment.From the percentage of sound events, participant A, participant C, and participant D had quiet sleep environments with few noise disturbances.But the noise problems existed in both participant B and participant E.
After the sleep experiment, an interview was held with each participant to understand more specific details about the noises sources.From the information, participant B lived near a big street where in the middle of a city, and there existed serious traffic noises even in the midnight.The source of noises were coming from a pump in the basement where participant E lived in the first floor of an apartment.From the sleep diagrams showing in Fig. 6, there are minor waves of sound event curves with participant A, participant C, and participant D. But there are obvious strong sound event curves showing on sleep diagrams of participant B and participant E.More evidences shows on the Table III.From the number of sleep promotion on the Table III, it also reveals the high frequencies of noise problems with participant B and participant E, and that results in the high quantity of sleep-promoting sound triggering.
They were also asked to answer a question to the system when they aroused in the middle of sleep, and the system recorded the answer time and answer code (1: toilet, 2: insomnia, 3: noises, 4: sleep partners, 5: alarm, 6: others).From the wakening reason on the Table III, all participants woke up from the middle of sleep occasionally about only one or two days a week, except participant A. Because participant A is a light sleeper, and she usually got up to went to the toilet in the midnight.Besides, she slept with her cat and she also got up to get the door for her cat occasionally (6: others).In addition, participant C was used to sleep in his living room without a curtain, and that causes the obvious waves of lighting events showing in Fig. 6.For the reason, the percentage of lighting events with participant C is also obvious higher than the others.The locations of sleep environments of the rest users are listed as follows: participant A lived with her family and owned her room, participant C rented an apartment near an alley, and participant D slept in a student dorm with a roommate.
A sleep diagram is provided for all the participants to see what exactly happened during their sleep showing in Fig. 1, because they may not remembered it.Especially, when they waken by noises, they may missed the major reason of wakening and thought they just needed to go to the toilet.The circumstance occasionally happens with participant B and participant E, and they both slept in noisy sleep environments.After browsing their sleep diagrams, they found out that the noise problems were not absent with the interruptions of their sleeping.From the experimental results, all participants agreed that the monitoring system is robust to detect and to record sleep events, and the sleep diagram helps them to recognize disturbances during their sleep efficiently.

2) Participants sleeping with sleep-promoting sound:
In this section, the condition of sleep-promoting sound will be further discussed.Only participant E chose rain sound to sleep with for three days and sound of sea wave to sleep with for four days in the experiment, and the rest participants chose dripping rain sound to sleep with.Besides, they all chose the same promotion mode with the sound playing from the beginning of their sleep.That means they were aware of the sleep-promoting sound during the experiment period, even the condition of sleep-promoting sound was chosen randomly by the system.They thought that the sleep-promoting sound was provided for helping them fall asleep, but they all did not know the additional function of the adaptive promotion sound for covering noises disturbances during their sleeping.Participant A, participant C, and participant D heard the sleep-promoting sound few times during their sleeping, because their rooms were really quiet.From the number of sleep promotion showing on Table III, there are obvious few times of sleeppromoting sound playing with them.The system only plays the sleep-promoting sound when a noise is detected.However, they all did not notice the occurrences of noises problems in their rooms, and that means they did not find out the mechanism of adaptive sleep-promoting function.Therefore, the sleep-promotion sound well covered and smoothed the noises in their rooms.
From the time index recording of sleep promotion, the special circumstance happened on both participant B and participant E. For their sleep environments were too noisy, the system played the sleep-promoting sound constantly during their whole night sleep.They both heard the playing sound sometimes during their sleeping, but it is impossible for them to find the mechanism of automatic playing sound for the constant playing.They may thought the sleep-promoting sound was originally set for whole night playing, and their sleep environments were already too noisy that they barely noticed the sleep-promoting sound playing.From the average of sleep efficiency showing on Table III, the sleep qualities in condition of sleep-promoting sound improves slightly with participant B and participant E; and the sleep qualities between the original www.ijacsa.thesai.orgsleep condition and the sleep-promoting sound condition are similar with participant A and participant C. Because the effect of adaptive sleep-promoting sound is not obvious in a quiet sleep environment for seldom noise disturbances.Participant D was not considered in the discussion for few data collection.From the observation, both participant B and participant E slept in noisy rooms, and the adaptive sleep-promoting sound worked well to cover the noises disturbances, and that also improves their sleep qualities.Finally, the condition of sleep-promoting sound and noise interruptions will be discussed.All participants were not aware of the noise interruptions setting.Almost all participants were occasionally aroused in the sleep condition, except participant C who was a heavy sleeper without interruption of his sleeping during the whole experiment period.From the examination of the time index between the question reply from the participants and the noise interruptions setting, and there are no overlapping with their arousing time and the noise interruptions playing time.In addition, the function of adaptive sleep-promoting sound were also applied in the condition, and the function covering noise disturbances also helps to improve sleep qualities.The correlation between sleep quality and the sleeppromoting sound is further discussed.From the average of sleep efficiency showing on the Table III, the effect of sleeppromoting sound is not obvious with participant B. Because the interruption of his sleeping both happened in the two days sleeping of this experiment condition.However, there are better performances of sleep qualities with the rest participants.From the observation, the adaptive sleep-promoting sound helps to improve their sleep qualities efficiently to bring better sleep experiences.

V. CONCLUSION
This paper developed a system that provides adaptive sleep-promoting sound based on the sound detection in sleep monitoring.An estimation method was proposed to determine the appropriate time for sleep promotion.According to the time index from the estimation method, the adaptive promotion sound is applied to cover the noise disturbances in a sleep environment.From the experiment result, it has clearly demonstrated the efficiency of the adaptive sleep promotion for different individuals, and the reliability of diverse sleep environments.Hence, the adaptive system helps to improve life qualities of people from the improvements of better sleep experiences.Besides, the non-invasive system do not cause uncomfortable sleeping, and the device is also cost-effective and convenient to be used in home context.www.ijacsa.thesai.orgFor the improvement of portable device, a smartphone with depth camera has announced recently and that is more convenient for users to use the system at home without additional device.It brings the possibilities to develop the system in mobile application.In addition, a sleep posture classification based on image analysis will be developed in the future work.

Fig. 1 .
Fig. 1.Interface of sleep diagram for sleep examination.The black marks on the top indicate index of sleep disturbances recording.Users select the specific region with marks to browse the corresponding video clips.Meanwhile, the selected region is expanded to show more details

Fig. 2 .
Fig. 2. Flow chart of development of the adaptive sound promotion system

Fig. 4 .
Fig. 4. Two playing modes of automatic sleep-promoting soundAlgorithm 1 shows the steps of promotion estimation and event detection.Given a set of scores from audio signals stream, if the audio signal is over the sound promotion threshold, a promotion time is marked.In addition, an sound event is triggered if the audio signal is also over the sound detection threshold.In other word, the appropriate time for sleep promotion includes the time index of sound events happening.Meanwhile, give a set of scores from depth signals stream, and a set of scores from color images stream, if the activity count of current epoch is bigger than that of the previous epoch, an event is triggered and starts a recording.The recording of the event continues while the activity count of the following epoch is equal or bigger than the activity count of the current epoch, otherwise the recording of the event is ended.
two different kinds of sleep conditions are discussed: a quiet sleep environment and a noisy sleep environment.The calculation of sleep quality is used to examine the effect of sound-based sleep promotion in the overnight experiment.IV.EXPERINMENT This section evaluates the proposed method with a realistic scenario.A device (Kinect for Windows) with a depth sensor, a RGB camera, and a microphone was used in the experiment, with 8-bit VGA resolution (640 × 480 pixels) of RGB images, 11-bit VGA resolution (640 × 480 pixels) of depth signals, and 24-bit resolution of audio signals.Both of depth signals and RGB images are at a frame rate of 30 Hz, and the sampling rate of audio signals is 16 kHz.In addition, a region-of-interest (ROI) (320 × 350 pixels) region is selected on both depth signals and color images for reducing signal noises.The device was placed at a distance of 0.2 meter and in a height above the bed of 1 meter showing in Fig. 5.It set in front of a bed at a depression angle of 27° that the camera view is in the center of a bed covering the entire body of a participant.

Fig. 6 .
Fig. 6.Sample Sleep diagrams from participants.(a), (b), (c), (d), (e) were the sleep recording of participant A, B, C, D, E, respectively.The graph consists of red curve of sound events, green curve of lighting events, and blue curve of motion events.(a), (c), (d) have quiet rooms, and both sleeping of (a), (b) indicates the noise problems with these participants 3) Participants sleeping with sleep-promoting sound and noise interuptions:Finally, the condition of sleep-promoting sound and noise interruptions will be discussed.All participants were not aware of the noise interruptions setting.Almost all participants were occasionally aroused in the sleep condition, except participant C who was a heavy sleeper without interruption of his sleeping during the whole experiment period.From the examination of the time index between the question reply from the participants and the noise interruptions setting, and there are no overlapping with their arousing time and the noise interruptions playing time.In addition, the function of adaptive sleep-promoting sound were also applied in the condition, and the function covering noise disturbances also helps to improve sleep qualities.The correlation between sleep quality and the sleeppromoting sound is further discussed.From the average of sleep efficiency showing on the TableIII, the effect of sleeppromoting sound is not obvious with participant B. Because the interruption of his sleeping both happened in the two days sleeping of this experiment condition.However, there are better performances of sleep qualities with the rest participants.From the observation, the adaptive sleep-promoting sound helps to improve their sleep qualities efficiently to bring better sleep experiences.

TABLE I .
COMPARISON OF SOUND-BASED SLEEP PROMOTION SYSTEMS

TABLE III .
SLEEP PARAMETERS OF EACH PARTICIPANT