Academic Emotions affected by Robot Eye Color : An Investigation of Manipulability and Individual-Adaptability

We investigate whether academic emotions are affected by the color of a robot’s eyes in lecture behaviors. In conventional human-robot interaction research on robot lecturers, the emphasis has been on robots assisting or replacing human lecturers. We expanded these ideas and examined whether robots could lecture using one’s behaviors that are impossible for humans. Psychological research has shown that color affects emotions. Because emotion is strongly related to learning, and a framework of emotion control is required. Thus, we considered whether emotions related to the learner’s academic work, called “academic emotions,” can be controlled by the color of a robot’s illuminated eye light. In this paper, we found that the robot’s eye light color affects academic emotions and that the effect can be manipulated and adapted to individuals. Furthermore, the manipulability of academic emotions by color was confirmed in a situation mimicking a real lecture. Keywords—Robot lecturer; academic emotions; lecture behavior; human-robot interaction


I. INTRODUCTION
As science and technology have developed, there have been attempts to replace human educators and lecturers with robots.Kamide et al. [1] reported the behaviors which emphasize key points in the screen and keep the attention of audiences are important as nonverbal behaviors when a humanoid robot gives a presentation.In addition, Ishino et al. [2] suggested that robots that can control own nonverbal behavior can emphasize the area that the learner wants to focus on with gaze, gestures, and paralanguage, and this behavior can promote the learner"s interest.However, to date there has been no attempt to use the unique nonverbal behavior of robots in lectures.
Colors affect people psychologically (e.g., [3], [4]); people experience different emotions in response to different colors.During teaching, the expression of a human teacher changes naturally; however, robot lecturers can use unique behaviors, such as changing their eye color.What emotions would be evoked in learners if the teacher"s eyes were to turn red?Investigating what kind of emotions this type of behavior evokes may be expected to lead to a wide range of applications in robot-led education.
The importance of responding appropriately to the learner"s mental state has been highlighted in a study [5].Thus, during learning, it is necessary to consider not only the material but also the emotion presented to the learner.For example, during classroom teaching, whether the teacher is smiling, angry, or sad can have a strong effect on the emotion evoked in the learner.The learner feels stressed when confronted with an angry teacher and relaxed when learning from a smiling teacher.Emotions that are related directly to learning, teaching, and academic achievement are called "academic emotions" by Pekrun et al. [6].Therefore, we considered that it is important to investigate how to improve learners" academic emotions using external factors.
In this paper, to investigate the academic emotions evoked by the color of a robot"s eye lights, the following hypotheses were examined: (A) the learners" academic emotions can be manipulated; (B) adaptive academic emotions can be produced in learners; and (C) academic emotions can be manipulated by colors in a situation like a real lecture.In the future, not only robot instructors will replace lecturers, but it is also possible to add ideal robot students (i.e., who behave to promote classes, such as asking an ideal question) to the classroom.Therefore, the roles of robots were divided into lecturer robots and learner robots, and the effects of color were investigated.The verification of hypotheses (A), (B), and (C) are described in Sections III, IV, and V, respectively.

A. Robot in Education
In recent years, robots have gained considerable attention in educational applications.Deublein et al. [7] suggested that robots" motivational behavior can improve learning outcomes in an educational context.Ishino et al. [2] suggested that the robot can promote the learner's interest by controlling nonverbal behavior in a presentation lecture.Jimenez et al. [8] suggested that expressing the robot's emotions promotes collaborative learning with the learners.In addition, Jimenez et al. [9] examined how robots emotional interact with learners in collaborative learning.Thus, we also know that emotion occurs within human-robot interaction in education.However, the emotions given to learners are not clarified.www.ijacsa.thesai.org

B. Intelligent Tutoring System
In recent years, research on Intelligent Tutoring System (ITS) has been increasing in attempts to support not only learner's knowledge state but also mental state.For example, the Intelligent Mentoring System [10], [11] in ITS is an attempt to support knowledge and mental states by acquiring information with different granularity such as answer history and mouse movement from learners.AutoTutor [12] has shown one of the more practical ways ITS can read learners' emotions.However, these studies have been conducted to read the learner's emotions, and not to control the learner's emotions from the ITS output.

A. Summary
We examined the effect of the color function of Pepper (Softbank Robotics) on the academic emotions of learners" learning activities.Participants were 10 graduate and undergraduate students.

B. Purpose
In this experiment we investigated the following hypotheses.(A-1) The academic emotions of the learner are affected by presenting color information as part of the lecture behavior of the robot and (A-2) the students" academic emotions are manipulated by presenting color information.

C. Stimulus
We prepared four utterance patterns as Pepper"s lecture behavior.There were two utterance patterns each for the roles of lecturer and learner.
 Important: "The point I will explain now is important."(as Lecturer).
 Warning: "Please stop talking, that is not related to class."(as Lecturer).
 Confused: "I had some difficulties, I do not understand."(as Learner).
 Understanding: "I see, right."(as Learner) The color representation in this study changed only the LED color in Pepper"s eyes, and the color of the other parts of the robot and its posture were unchanged.Because only color representation was used, gestures were not performed.Emotions evoked by color are interpreted subjectively; thus, the following correspondence between the colors and emotions was shown to limit interpretation (Fig. 1).
 Red: Anger or strong feeling.
 Green: Joy or mild feeling.
The 12 combinations of the four utterance patterns and three colors were used as stimuli.

D. Questionnaire
In psychology, emotions that are related to learning, teaching, and academic achievement are called academic emotions.These emotions are enjoyment, boredom, anger, hope, anxiety, hopelessness, pride, and relief [6], [13].We prepared seven-point Likert scale questionnaires using these academic emotions as evaluation items.

E. Procedure
In the experiment, all utterance patterns for Pepper were given in order of white, red, and green.
Each utterance pattern was evaluated by a questionnaire with the question "How do you feel when you take a lecture with this robot?" The procedure is as follows (Fig. 2).

F. Results
The results are shown in Fig. 3 to 6.The results were subjected to multiple comparisons using Ryan"s method after confirming a significant difference (p < 0.05) by Friedman"s test.The significance symbols in the figures indicate the significant difference (p < 0.05) calculated by Ryan"s method and the error bars indicate standard errors.Fig. 3 shows the results for the Important utterance pattern.There were significant differences between white and red and between white and green in boredom, pride, hope, and relief.White enhanced boredom and suppressed pride, enjoyment, hope, and relief.Conversely, red and green suppressed boredom and enhanced pride, enjoyment, hope, and relief.There was also a significant difference in enjoyment between white and green suggesting that white suppressed and green enhanced enjoyment.Fig. 4 shows the results of the Warning utterance pattern.There was a significant difference between white and red in hope.White suppressed and red enhanced hope.There were many differences in the values themselves, but no other significant differences were found, probably because the Warning context itself depended on the individual learner.Fig. 5 shows the results for the Confused utterance pattern.There was a significant difference between white and red in hope and relief.White suppressed and red enhanced hope and relief.There was also a significant difference between white and green in enjoyment.White suppressed and green enhanced enjoyment.Fig. 6 shows the results for the Understanding utterance pattern.There were significant differences between white and green and between red and green in pride, hopelessness, enjoyment, and anxiety.Green enhanced pride and enjoyment and suppressed hopelessness and anxiety.Conversely, white and red suppressed pride and enjoyment and enhanced hopelessness and anxiety.There was also a significant difference between white and green in boredom, hope, and relief.White suppressed and green enhanced hope and relief, and white enhanced and green suppressed boredom.There was also a significant difference between red and green in anger.Red enhanced and green suppressed anger.These results show that green had a positive effect on Understanding utterance patterns.

G. Evaluation
These results show that the effects of the combination of utterance and color are different, and that the academic emotions of the learner depended on the color and could be manipulated by changing the color.That is, hypotheses (A-1) and (A-2) were verified.

IV. ANALYSIS BASED ON INDIVIDUALITY OF LEARNERS
In Section III, we found that the learner"s academic emotions were changed and manipulated by changing the color.However, all learner profiles cannot be treated the same, and it is desirable to have a color representation that can respond to the individual characteristics of the learner.Therefore, in this section, we investigated and analyzed the characteristics of the learners using questionnaires on educational psychology.www.ijacsa.thesai.org

A. Questionnaire for Investigating Individuality
Nishimura et al. developed the Japanese short version of the Self-Regulation Questionnaire [14], [15] that assesses intrinsic, identified, introjected, and external regulation in selfdetermination theory.Each type of regulation was measured via five items.The items were rated using a four-point scale (1-strongly disagree to 4-strongly agree).Nishimura et al. [14] reported the validities of this scale by correlation analysis with the original Self-Regulation Questionnaire [16].
The features of each type of regulation are as follows:  Intrinsic: Learning itself is purposeful; for example, learning itself is interesting or desirable and corresponds to conventional internal motivation.
 Identified: A motivation that represents recognizing the value of doing activities and accepting it as one"s own, such as recognizing that learning shapes one"s future.
 Introjected: A negative but partially internalized feature of the activity"s value, such as self-expansion and maintenance of self-worth by comparison with others.
 External: Influenced by reward acquisition and avoidance of punishment and corresponds to conventional external motivation.

B. Method
To examine the individuality of the learner, we asked the participants in Experiment I to answer the Self-Regulation Questionnaire [14] and the values were calculated for each of the four regulations.By correlating the regulation values with the values of eight academic emotions, we obtained individual profiles for each learner that allowed adaptive interactions to be produced for the learners in advance.

C. Results
Table I shows the results of the correlations for each utterance pattern.Each result was tested for noncorrelation, and significant differences are shown ( + p < 0.10, * p < 0.05).A significance of 5% is indicated with a background color (blue, positive; orange, negative).
The results in Table II show that there are places where significant differences occur depending on the combination of utterance pattern, color, academic emotion, and regulation.There were regulations that had significant positive and negative correlations with green (Table Ib).Similar results were observed for pride with white and green (Table Ic).Therefore, regulation and academic emotion are related.

D. Evaluation
Multiple academic emotions may be evoked simultaneously; red was correlated with anger and enjoyment depending on the regulation (Table Ic).Consequently, it was difficult to evoke a single academic emotion alone.Therefore, academic emotion expression was simplified by dividing it into positive emotion and negative emotion, and we examined what kind of interaction caused positive and negative emotions by using an evaluation formula.
We constructed the evaluation formula to evaluate learners" emotions comprehensively.The evaluation of E c,i,s of regulation s, for utterance i, with color c, was 1 for the positive emotions Pride, Enjoyment, Hope, and Relief (p = 1 to 4), and −1 for the negative emotions Boredom, Anger, Hopelessness, and Anxiety (n = 1 to 4).Each academic emotion and each regulation (r c,i,s,n or r c,i,s,p ) were multiplied, and then the total value was calculated as If there is no individual adaptability, the correlation is low and the value of (1) approaches 0. If a certain regulation has a high positive correlation for positive items and a high negative correlation for negative items, (1) takes a positive value, and in the opposite case, (1) takes a negative value.Therefore, when the value of ( 1) is high, it indicates good compatibility with a regulation, and when it is low, it indicates poor compatibility.In other words, if you want the learner to have a positive emotion, it is effective to perform an interaction with the combination of the utterance and the color that results in (1) taking a positive value.Conversely, if you want negative emotions, it is effective to perform an interaction that results in (1) taking negative values.
Table II shows the results of (1).Absolute values of 1 or more are shown in bold, and absolute values of two or more are shown with a background color (blue, positive; orange, negative).
The results indicated that green is effective overall, especially for external regulation, to convey that the lecturer robot is presenting important information.Even if the lecturer robot is warning the learner, green is still effective overall, but red is effective when evoking a negative emotion.Green is also effective for intrinsic regulation when a learner robot shows confusion or understanding, but counterproductive for identified regulation.
Analysis of individuality showed that appropriate interactions depend on the learner's self-regulation, and we examined the possibility of an adaptive interaction for each learner.Thus, hypothesis (B) was proved.www.ijacsa.thesai.org

V. EXPERIMENT II
In Sections III and IV, we used a single utterance and assumed it was similar to a real lecture; however, there was no flow or context.The context of the preceding and following utterances may also affect academic emotions.Therefore, we investigated the influence of continuous utterances on the academic emotions of the learner in the context of a real lecture.

A. Summary
We examined the effect of the color of Pepper"s eyes on the emotion of learners" learning activities.The participants were 18 graduate and undergraduate students (nine participants in the experimental group, and nine participants in the control group).

B. Purpose
We investigated whether academic emotions can be manipulated by colors in a real lecture.

C. Stimulus
As in Experiment I, we only changed the color of Pepper"s eyes, and the eye color and the corresponding emotion were the same.A real lecture on statistical analysis was divided into 12 utterances (Tables III and IV), and we assigned colors when the utterances contained the intents of Important, Warning, Confused, and Understanding.

D. Questionnaire
As in Experiment I, we prepared seven-point Likert scale questionnaires that used the academic emotions as evaluation items.

E. Procedure
In the experiment, a continuous context was created based on a recording of a real lecture, and the interaction between the Pepper robots was created on a video.
An example of the experimental setup is shown in Fig. 7.The participant was given a laptop and wore headphones.A video of the Pepper was shown (Fig. 8).III.
In the video, we gave Pepper the roles of teacher, teaching assistant (TA), and student.We used the utterances that promote Important and Warning for the teacher and the TA roles, and the utterances that represent Confused and Understanding for the student role.Hence, in the video, interactions were performed by Pepper in these three roles.
The participants were instructed to familiarize themselves with the context of the interaction in the video for each utterance in advance, and to assume that they were in the same environment as the three Pepper robots.Then, the emotions felt by the participant when they heard each utterance were recorded with a questionnaire.
The 18 participants were divided into the experimental group (nine participants), who were presented with the colors and the corresponding emotion table, and the control group (nine participants), who were not presented with the colors and the corresponding emotion table.
The procedure was as follows (Fig. 9):  First of all, here is the question and answer section.
One student asked the question "I answered "Please describe xxx" in the Excel text box.Is this right?"This is "basically, let's output the problem in Excel", so save the contents in Excel.You should write in the text or write directly in the cell.However, among the present submissions, someone wrote in the free description column of Moodle.
[(Green) This time, there was a mistake, but everyone should write in Excel from now on; please be careful.] 2 Important Teacher Also, I think that "Statistical analysis" is more difficult than in the previous class because it is more specialized.In my lectures, I would like you not just to remember what I have taught you, but to think "where can I use this?"Because we need to get used to difficult things.
[(Red) In addition, there is one more important point that applies to study in general: when thinking "where is it difficult," "where is it hard to understand," please carefully think about where you stopped understanding.]In fact, the lecture materials are those of my predecessor, so there is no explanation of the data division.Usually, there is nothing perfect about what you get.So, I would like you to develop skills such as supplementing these materials and examining points that you do not understand.
[(Red) Therefore, I think it is very important to do activities that supplement the lecture materials, because it is OK for you to take notes when things that are not in the materials are explained.]6 Important Teacher There was a comment that "the explanation of the composition cumulative ratio and the explanation of the Pareto chart were quite difficult".Pareto charts and cumulative ratios are simple things that you can easily understand if you find easy-to-understand web-pages, but there are many web-pages that are hard to understand.So, I think it would be quite difficult when you use such a web-page.
There was also the question, "Do you want a cumulative ratio for the histogram?"This depends on the situation.In most cases, it may not be required up to that point, and it is often good if a histogram can be created, but in some cases it, a cumulative ratio should be shown.
[(Green) The important thing here is whether you can do it when someone says "do it because I need it".It is more important whether you can do it when told, rather than whether you always do it.]7 Important Teacher Now, I would like to explain the statement I mentioned earlier, "The difference between variance and standard deviation is difficult to understand." [(Red) The explanation here is important, so please listen carefully.]Simply put, both are used for data dispersion.Both the variance and the standard deviation mean that the smaller the value is, the smaller the variance will be.
The difference is that the variance is easier to calculate.Moreover, although the size of the dispersion can be compared, dispersion does not show how large the variation is.Please look at document 1. <About document 1> This is extracted with basic statistics, but dispersion can show that scores in social studies and math are different.However, dispersion does not show how large the variation is.Therefore, using standard deviation is somewhat difficult for mathematical expressions.But how much does the overall score deviate, for example, in the case of an average of about 50 points; is the variation about 5 points or 20 points?It is the standard deviation that is required to determine this.So, basically, standard deviation is more useful.So, in most cases, it is important to keep in mind that the most useful standard deviations are most often used.www.ijacsa.thesai.orgFor the control group, the robots" eyes were always illuminated white in step 2 a).

F. Results
Fig. 10 shows the typical average values for Utterances 2 to 6 in the experiment group, and Fig. 11 shows the corresponding typical average values for the control group.

G. Evaluation
Because the color was always white for the control group, Fig. 11 shows the emotion related to the uttered content itself.There was almost no change in emotion for the continuous utterance in the control group; however, in the experimental group (Fig. 10), there were multiple changes in emotion.For example, in Utterance 3, Boredom in the experimental group is clearly increased, and Anger and Hope are suppressed.Furthermore, in Utterance 5, Pride, Anger, and Hope in the experimental group are clearly increased.These results confirmed that emotion could be manipulated by color even in a situation similar to a real lecture.These results suggest that although teaching using nonverbal behavior in robots has been used in a learning context, teaching using color increases the range of options to provide more diverse and individually tailored teaching.Robots affect the mental state of humans, but the reasons why this is so have not been explicitly characterized.In addition, it suggests that color psychology can be applied to robot-led education.Using the relationship between color and emotion, which has long been cultivated in color psychology, in robotled education is a new method and should help to develop more affect-sensitive robot-led education.

VII. CONCLUSION AND FUTURE WORK
To investigate academic emotions evoked by the color of a robot"s eye lights, the following hypotheses were verified.(A) The learners" academic emotions could be manipulated; (B) adaptive academic emotions could be produced in learners; and (C) academic emotions could be manipulated by colors in a situation similar to a real lecture.The verification of hypotheses (A), (B), and (C) were described in Sections II, III, and IV, respectively.
The results indicated that eye color could manipulate the academic emotion, and that adaptive lecture behavior could be produced based on the individuality of the learner.Furthermore, academic emotions were manipulated by the robot"s eye color in a situation similar to a real lecture.
However, these results are for limited utterance pattern and color combinations and require more detailed investigation.In addition, these results show that they act on the learner's emotions, not the learning effects.Therefore, future works are confirmation of reproducibility in the practical field and confirmation of versatility by combining multiple patterns.

1 ) 2 )
Participants sit in front of Pepper.Participants evaluate Pepper"s lecture behavior.a) Pepper says an [Important] utterance while illuminating its eyes (white).b) Participants evaluate the lecture behavior in context with a questionnaire.c) Repeat a) and b) while changing the utterence patterns in []to Warning, Confused, and Understanding.d) When all of the utterance patterns have been evaluated, change color in () and repeat a) to c).

1 ) 2 )
Participants sit in front of a laptop.Participants evaluate Pepper"s lecture behavior in a video.a) Pepper speaks Utterance [1], while illuminating its eyes when it has an intent.b) Participants evaluate in context with a questionnaire.c) Repeat a) and b) while changing number in [] from 1-12 corresponding to the utterance.www.ijacsa.thesai.org

3
Confused Student [(White) I did not understand where the interval of 10 to 100 was or how to make a data division in a histogram.]4 Confused Student [(Green) The variance and standard deviation could be determined.However, I investigated the problem of describing the difference between variance and standard deviation, but I don"t understand the difference.]5 Important Teacher Thank you, this has been asked a lot.I would like to explain later how to use variance, standard deviation, and their differences.Everyone, you could think about what that meant the words in the histogram and data divisions.

Fig. 10 .
Fig. 10.Results for the experimental group: error bars indicate standard errors

TABLE II .
RESULTS OF INDIVIDUALITY ANALYSIS

TABLE IV .
SCENARIO OF UTTERANCE 8 TO 12 FOR EXPERIMENT II. in front of you is called the learning pyramid.This is important information, so I would like you to remember it, but everyone listens to lectures for about a week at university, right?Researchers found that if you just listen to the lecture, the knowledge retention rate is 5%.So even if I try my best and talk about 100 things, you will only remember five.So, I want you to be conscious of this, and stop just listening to the lecture.I think that you will not get anything out of it with this technique.The knowledge retention rate is 10% for reading and 20% for watching videos.In addition, the rate is about 30% if you see someone operating something, such as a demonstration.All of these are passive learning techniques.How is everyone so far?Probably, there are many people engaged in passive learning.In other words, people who think that they are not linked to their own future, who are not conscious of this, or are interested only in the contents of the lecture, tend to have a low level of understanding.Instead, for example, when you assume you will become a CEO in the future, when you think about it in relation to you, or when you perform activities such as teaching people, 75% to 90% of knowledge is retained.So, basically, knowledge is not retained when you input or listen, but when you output.Knowledge can only be absorbed when you use it, so it is very important to think on your own.Among the techniques, teaching people is very effective, so please teach your friends whenever possible.Excuse me, when I made a graph with an input range, the Lecturer said "Don't forget to check the label," but I do not know what the label is.] 10Warning TA What is the label...I think that when choosing the input range, you chose from the name of the top subject.But isn't the name of the subject a number?Therefore, saying "the name of the top subject is not a numerical value" is a label check.If you add a label, it will analyze it as a simple character, not the data at the top.[(Green)It is easy to make mistakes when testing, so be careful.]Inaddition, the variance uses the VAR function, and the standard deviation uses the STDEV function.There are several types of variance and standard deviation, such as P and S, but you can use any of them in this lecture.