An Interval-Based Context Reasoning Approach

Context-aware computing is an emerging computing paradigm that provides intelligent context-aware application. Context reasoning is an important aspect in context awareness, by which high level context can be derived from low-level context data. In this paper, we focus on the situation in mobile workspace, where a worker performs a set of activities to archive defined goals. The main part of being aware is to be able to answer the question of "what is going on". Therefore high level context we need to derive is current activity and its state. The approach we propose is knowledge-driven technique. Temporal relations as well as semantic relations are integrated into the context model of activity, and the recognition is performed based on the model. We first define the context model of activity, and then we analyze the characteristics of context change and propose a method of context reasoning.


INTRODUCTION
Context-aware computing is an emerging computing paradigm that provides intelligent context-aware application.According to well-known definition proposed by Dey, context is any information that can be used to characterize the situation of an entity.An entity is a person, place or object that is considered relevant to the interaction between a user and an application, including the user and application themselves [1].Context-awareness means exploiting context information to provide adaptive information or service, or reducing the interaction between user and application.
Context-awareness is related to the manipulation of context information pertaining to certain entities.Information from physical sensors are called low-level context.High level context, also called situation sometimes, can be derived from low-level context by proper interpreting.This process is context reasoning, or context interpretation.Situations are semantic abstractions from low-level contexts cues.So the relationship between low level context and situation must be integrated into a context model, which represents human knowledge about the world [2].This can either be done by specification, i.e. human defines situations and their relationship based on his /her knowledge, or the model are learned automatically using machine learning techniques [2].The two approaches are also called knowledgedriven approach and data-driven approach [3].
Data-driven techniques are based on the machine learning methods and are well suited for recognizing simple activities and gestures from raw sensor data or video data [4].A wide range of algorithms and models include Hidden Markov Models [5], dynamic and naïve Bayes networks [6], and decision trees [7] and so on.Knowledge-driven techniques, concerning knowledge representation as well as reasoning with them, is closely related with classical topics in artificial intelligence [8].Ontology-based method is one of frequently used technique.For example, Chen has proposed a technique to recognize activities through ontological reasoning [9].Knowledge-driven techniques can also be used to recognizing complex situations based on the recognized simple context [10].
In this paper, we focus on the situation in mobile workspace, where a worker performs a set of activities to archive defined goals.The main part of being aware is to be able to answer the question of "what is going on".Therefore the situation, or high level context, we need to derive is activity and its state.
Determining the activity and its state cannot only be completed by observing the context but also need to draw conclusion from the observations.Knowledge about the relation of context and activity should be integrated into the model.Compared with most of knowledge-driven techniques, the approach we propose considers the temporal relation between activities and contexts.
Activity is seemed as process with duration, and context may change in the process.Each activity context model is a set of semantic relations and temporal constraints with respect to the activity and the context.If some observed contexts match the defined patterns, or if their times of occurrence meet the specified constraints, then an instance of this situation occurs.
We have developed an activity recognition approach taking the temporal relation into account in previous work [11].This paper considers more temporal relations.Moreover semantic relations are also integrated into the model.By this means, more types of situation can be recognized.
The rest of the paper is organized as follows.Section 2 provides the context model of activity.Section 3 proposes the activity recognition approach.Case study is described in section 4 and section 5 concludes the paper.

II. CONTEXT MODEL OF ACTIVITY
Suppose the application is related to a set of contexts which are denoted as c 1 ,c 2 ,...c n , and the domain of c i is D i .At given time t, the value of context c i is denoted as ) is called the negative pattern of (c i ,d j (i) ), denoted as ).
We define a special context pattern time-elapse(t s ,T),   Context pattern p is a condition of conducting activity a.If it is not satisfied as its corresponding temporal pattern, the activity is abnormally conducted.

Before(p,a), During(a,p), Overlap(p,a)
PREM(p,a) Context pattern p is a premise of conducting activity a.If it is not satisfied as its corresponding temporal pattern, the activity is not started or interrupted.For real world problem, some semantic relations are not easily to be distinguished.For example, condition and premise are very similar.Here the difference between condition context pattern and premise context pattern is that the condition context pattern can be controlled by human, while the premise is objective.That is if a condition context pattern does not hold in proper time as defined, the activity can be performed but not normally.However, if a premise context pattern does not hold, it can be deduced that the activity has not been started.
The difference between consequence and accompanying context pattern is that the consequence is related with the goal of the activity, and the pattern holding means the activity will end or has ended, while the accompanying context pattern holding may means the activity has been started and if it does not occurred as defined, there may be interruption or abnormal situation.When context interpretation is performed, the occurring context is compared with the model, and current high level context, i.e. the activity and its state is deduced.

III. RECOGNIZING ACTIVITY BY CONTEXT REASON
Generally in context-aware system, sensors (either virtual sensors or environmental sensors) are used to acquire the raw context.The acquired low-level context data are dealt with by many context-aware middleware or infrastructure and higher level context information are available and represented in formal format.Our work is to derive context in much higher level based on these context information, i.e. recognize the activity and its current state.
The input of our work is sequence of context values in different time, c(t 1 ),c(t 2 ),…c(t n ).Assume that every meaningful context change can be detected, and the values are omitted if there is no change happening.So if we have c(t i ) and c(t i+1 ), for any t, t i t< t i+1 , c(t)=c(t i ) .We need to identify the activity and its state according to these context values and activity model.
There are two opposite strategies of processing context data.The first one analyzes previously received context, matching them with related temporal pattern on the basis of their semantic meanings and give the result.This strategy requires less memory but may introduce a higher processing delay.The second approach processes data incrementally, recognizing and stores partial consequence in the form of automata as soon as they are detected.This may need more memory (memorizing different state) but speed up the processing.We adopt the second strategy.

A. State and internal state
In activity models, temporal relations define the rules of context pattern occurring.Combined with their semantic meanings, these rules can be used to interpret "what is going on".Let e-pattern(a,t)={ppe-pattern(a)˄hold_at(p,t)}, apparently e-pattern(a,t) is a subset of e-pattern(a), which can be used to measure how close to the end for the activity process.If e-pattern(a,t i-1 ) e-pattern(a,t i ), the state at time t i is closer to activity end than last moment.This is useful when the activity is judged to be ended, but meaningless when the activity is still proceeding normally.
When an activity is being conducted without interruption, its accompanying context pattern should always hold if its temporal relations with the activity is equal, so should the premise context pattern if the relation is during (i.e.activity during context pattern).If for an accompanying context pattern p, finish(p,a), or overlap(a,p), p should be hold once it occurs until or after the activity end.Therefore, when the activity is being conducted without interruption, the conditions that the context should satisfy are changing.Since the above proposition is easy to be proved according to the definitions, proof is omitted here.However, context value at t satisfies the start pattern or end pattern, i.e. all patterns in s-pattern or e-pattern hold, does not mean the activity starts or ends normally.There are other condition should be checked.Similarly, if the context values over the interval satisfy the on patterns of an activity does not mean the activity is performed normally, even if no interruption happens.We will illustrate this afterwards.
There are 4 types of state to be recognized, called output state.The states are: waiting (for activity beginning), on (one activity is conducted), suspend (one activity is interrupted), abnormal.The recognizing result is denoted as 2tuple:(activity name, state).The state transition is shown in Figure 1.In addition to above states, we define other 9 states which are used for system control, called internal states.The concept of internal state is actually state of automate, which has process memory itself when it is reached.The states are: waiting, starting, on, suspend, ending, end, start abnormal, process abnormal, end abnormal.
Among them, waiting, on, suspend have similar meanings with output states of same name.State starting means stage that all premise conditions are satisfied and part of accompanying context patterns have occurred but the start pattern is not satisfied.State ending means consequence context has occurred, or for the activity without consequence context pattern interruption lasts too long, hypothesis of activity ending need to be proved.If end pattern is satisfied, or time is longer enough, system converts to end state and check the condition to determine if it is normal.Therefore state end is actually a checking point without duration and there is no corresponding output state.Three abnormal states corresponds abnormal output state.Internal state starting is start when output.Since mostly there is no a time point in which all the start patterns are satisfied at same time, setting an internal state starting can avoid taking this situation as abnormal.Internal state transition is shown in Figure 2. Since internal state has more process information, analyzing previous context is avoided by this mean.This can be shown below.Internal state is related with the recognizing process, so we focus on the internal state in this paper and waiting (on, suspend) means internal state unless it is specifically explained.

B. Memorizing occurred context
For some of temporal relation, such as Before and During, determining whether they are satisfied depends on the previous context information.For example, if During(p,a), and p does not hold at time t, we can't determine whether it is normal.www.ijarai.thesai.orgOnly when the activity ends and p has never occurred, it is determined that activity is not performed normally.In order to avoid checking previous context which may introduces high processing delay, we use some flag to memorize some special occurred context.
We use a set b-COND(a) to record the context patterns which need to hold before activity a is started.The initial value of b-COND(a) is b-COND(a) ={ppP, before(p,a) and COND(p,a)}.From the very beginning, whenever a context value input, the patterns in b-COND(a) are checked and holding patterns are deleted from the set.Normally the set is empty when the activity is stated.So the when activity a is recognized to start, it is abnormal if b-COND(a) is not empty.
Similarly, we use a set d-ACCOMP(a) to record the context patterns which need to hold during the time that activity a is performed.Let d-ACCOMP(a)={ppP, during(p,a) and ACCOMPA(p,a)}.This is the initial value of d-ACCOMP(a).When a is recognized to be started, the patterns in the set are checked and holding patterns are deleted.When activity a is recognized to end, it is determined to be an abnormal end if d-ACCOMP is not empty.

C. Recognizing activity and its state by state transition
The recognizing process depends on the state and state transition.The basic idea is as following.
If all the premise conditions are satisfied and accompanying context patterns start to hold, it is recognized that the activity has started.Considering there may be more than one accompanying context should hold at the beginning (Equal or Start relation) and there is no real time point that these context patterns change to hold, we use state starting to solve the problem.When all the premise context patterns hold and at least one accompanying pattern hold, the state is converted to starting from waiting.When the start pattern is satisfied, condition checking is performed and the state is converted to on or abnormal start.
As the activity is conducted, condition is checked when any context changes and the state is converted to abnormal process when any of them are not satisfied.Moreover, comparing current on pattern with last moment can determine whether the activity is interrupted.If interruption occurred, the state is converted to suspend.
Activity end can be judged by consequence context.If consequence context patterns occurred, the state is converted to ending and on pattern is no longer to be checked.If there is no consequence context pattern for an activity, activity end can be judged by premise and accompanying context pattern.If some of premise and accompanying context pattern are not satisfied, state will change to suspend and if the state lasts long enough, it will be converted to ending for the activity with no consequence context pattern, and to abnormal process for others.
In ending state, if e-pattern is satisfied, the state is converted to end, or else, e-pattern(a,t)is calculated according to the current context c(t).e-pattern(a,t) is compared with epattern(a,t-1) to determine whether the activity is close to end.If more context patterns in end pattern occurred, i.e. e-pattern(a,t-1)e-pattern(a,t), initialize time-elapse context pattern to make the state lasting for longer time.If time-elapse is true, it means that the state lasts too long and state is converted to abnormal process.
In end state, d-ACCOMP(a) is checked and if the set is empty, the state is converted to waiting(for next activity).Otherwise there exists not happened context pattern that should hold during the activity process, the state is converted to abnormal end.
For three types of abnormal, the state transition rule is waiting for a certain time and converted to waiting, on respectively.
TABLE III shows the transition rules in every state.

IV. CASE STUDY
We use the approaches presented above in single-crystal X-ray diffraction experiment support.We focus on the context reason part, i.e. recognize the activity and its state.This is important in support environment, because the reaction of the support system, i.e. provide alerting and guiding information depends on the correctly recognized the situation.
The single-crystal X-ray diffraction procedure consists of three phases, in which activities are conducted in three places, such as Room 101, Room 102 and Room 103 in our example.The three phases are: selecting a crystal which is carried out in Room 101, analyzing the crystal which are conducted in Room 103 and structural determination which is accomplished in Room 103.There are eight activities in these phases and they should be performed as a certain sequence.
To perform every activity, different tools are needed.The location of user, location of tools as well as their states can be sensed and detected, which are used to determine the activity.There are 19 types of contexts related to the activities in experiment.TABLE IV is the context model of every activity.
Huang et al has designed same kind of system and analyze its effect in education [13].However technology is not an important issue in that paper and activity recognition is simply realized by rule-based reason, only considering context at one time point.
Our work focuses on the technical aspects.We have developed an activity model taking the temporal logic into account, as in [11].However, in [11], there are only equal and during relation, and some of abnormal situation cannot be recognized.These problems are partly solved in this paper.

V. CONCLUSION
Context interpretation in most of research is based on the assumption that high level context may result in different sensor readings or low level context, where the relation and knowledge is integrated into context model.Therefore the context interpretation is actually the hypothesis of "cause" from "consequence" which is regarded as true unless it is denied.The condition of deducing high level context is generally not sufficient.Our method is also based on this assumption and more complex because of composition of temporal relation with semantic relations.The underlying www.ijarai.thesai.orgThe approach proposed in this paper is interval-based, considering the context change over a time period.This can recognized more situation than the method based on the context data in time point, but introduce high complexity in process.We define 6 temporal relations and 4 semantic relations; these are not enough to represent the relation in real world.The approach need to be further studied in future research work.
Overlap(p,a), During(a,p), Finish(a,p) ACCOMP(p,a) Context pattern p will occur as its corresponding temporal pattern if the activity a is conducted normally Start(p,a), Equal(p,a), During(p,a), Finish(p,a), Overlap(a,p) CONSEQ(p,a) Context pattern p is a consequence of activity a. Overlap(a,p) If COND(p,a), p is called a condition context pattern of a.In same way, for PREM(p,a), ACCOMP(p,a), CONSEQ(p,a), p are called premise context pattern, accompanying context pattern and consequence context pattern respectively.www.ijarai.thesai.orgC. Modeling activity with semantic relation and temporal relation We can model the relation of activity and context pattern with semantic relation and temporal relation.These relations show the context changing rules when the activity is performed normally without interruption.

Definition 1 Definition 2
Let aA, s-pattern(a) is called start pattern of activity a, if and only if: s-pattern(a)={pPREM(p,a)(ACCOMP(p,a)˄ (start(p,a) equal(p,a) start(a,p))}  Starting pattern of an activity is a context pattern set, the pattern in which should hold when the activity is started.Therefore according to generally underlying assumption of context reason, it can also be used to determine the start of the activity.When an activity a ends normally, its consequence context pattern should hold.Otherwise, if there is an accompanying or premise context pattern p, equal(p,a), or finish(p,a), p should cease holding, i.e. p holds, when the activity ends.Let aA, e-pattern(a) is called end pattern of activity a, if and only if: e-pattern(a)={pCONSEQ(p,a)(PREM(p,a) ˄ finish(a, p))(ACCOMP(p,a)˄(finish(p,a) equal(p,a)))} 

Definition 3
Let aA, o-pattern(a) is called an on pattern of a, if and only if : o-pattern(a) {p (PREM(p,a)˄During(a,p)) (ACCOMP(p,a)˄(Equal(p,a)Finish(p,a) Overlap(a,p)))} and o-pattern(a){p(PREM(p,a)˄During(a,p)) (ACCOMP(p,a)˄Equal(p,a))}  An activity may have more than one on patterns.Among them, if: o-pattern(a)={p(PREM(p,a)˄During(a,p)) (ACCOMP(p,a)˄(Equal(p,a)Finish(p,a) Overlap(a,p)))}  Any on pattern of a is the subset of o-pattern(a), denoted as max-on-pattern(a).

TABLE I .
TEMPORAL RELATION OF CONTEXT PATTERN AND ACTIVITYBesides temporal relations, there are also semantic relations existing between context patterns and activities.COND, CONSEQ, PREM, ACCOMP are semantic relations defined to illustrate logic connection in context patterns and activities.The meaning and corresponding temporal relations are listed in TABLE II.

TABLE II .
SEMANTIC RELATION CONTEXT PATTERN AND ACTIVITY

TABLE
II shows the possible temporal relation of every type semantic relation.The temporal relation should be determined when defining an activity.The different temporal relation is used differently.

TABLE IV .
CONTEXT MODEL OF ACTIVITY IN SINGLE-CRYSTAL X-RAY DIFFRACTION EXPERIMENT