Method for Learning Effciency Improvements Based on Gaze Location Notifications on e-learning Content Screen Display

Method for learning efficiency improvement based on gaze notifications on e-learning content screen display is proposed. Experimental results with e-learning two types of contents (Relatively small motion of e-learning content and e-learning content with moving picture and annotation marks) show that 0.8038 to 0.9615 of R square value are observed between duration time period of proper gaze location and achievement test score.


INTRODUCTION
Computer key-in system by human eyes only (just by sight) is proposed [1], [2].The system allows key-in when student looks at the desired key (for a while or with blink) in the screen keyboard displayed onto computer screen.Also blink detection accuracy had to be improved [3], [4].Meanwhile, influence due to students' head pose, different cornea curvature for each student, illumination conditions, background conditions, reflected image (environmental image) on students' eyes, eyelashes affecting to pupil center detection, un-intentional blink, etc. are eliminated for gaze detection accuracy improvement [5], [6].On the other hands, the system is applied for communication aid, having meal aid, electric wheel chair control, content access aid (e-learning, e-comic, ebook), phoning aid, Internet access aid (including Web search), TV watching aid, radio listening aid, and so on [7]- [17].
The method for key-in accuracy improvement with moving screen keyboard is also proposed [18].Only thing student has to do is looking at one of the following five directions, center, top, bottom, left and right so that key-in accuracy is remarkably improved (100% perfect) and student can use the system in a relax situation.
One of the applications of gaze estimation is attempted in this study.Using gaze estimation method, lecturers can monitor the screen location where students are looking at during they are learning.Sometime students are not looking at the same location where content creator would like students look at.Such students may fail or have a bad score in achievement tests.When students learn with e-learning contents, lecturers can monitor their gaze location so that lecturers may give a caution when students are looking at somewhere else from the location where lecturers would like students look at.Thus learning efficiency may improve somewhat.
The second section describes the proposed system followed by some experimental results.In the experiments, learning with typical e-learning content with gaze estimation is conducted first followed by learning with e-learning contents of moving picture with annotations (lecturer indicates the location where lecturer would like students look at with some marks).Thus effectiveness of the e-learning with gaze estimation is enhanced.Finally, concluding remarks are followed by with some discussions.

A. Gaze Location Estimation Method and System
Students wear a two Near Infrared: NIR cameras (NetCowBoy, DC-NCR1301 ) mounted glass.One camera acquires student eye while the other camera acquires computer screen which displays e-learning content.Outlook of the glass is shown in Figure 1 while the specification of NIR camera is shown in Table 1, respectively.In order to monitor students' psychological situation, Electroencephalography: eeg2 sensor (NueroSky3 ) is also attached to students' forehead as shown in Figure 2. www.ijarai.thesai.org

B. Examples of the Acquired Images
An example of the acquired eye image with the NIR camera is shown in Figure 4 together with the binarized detected cornea and the extracted pupil image.In the NIR eye image shows a clearly different cornea from the sclera.In this case, although influence due to eyelash and eyelid is situated at the end of eye, not so significant influence is situated in the eye center.Also pupil is clearly extracted from the cornea center.NIR camera which shown in Table 1 has the six NIR Light Emission Diode: LEDs 5 which are situated along with the circle shape.The lights from the six LEDs are also detected in the extracted cornea.Firstly, student has to conduct calibration for adjust the distance between the student and the computer display.In the calibration, student has to look at the four corners of the checkerboard which is displayed on the computer screen as shown in Figure 5.
http://www.carelinks.net/books/ch/memory.htm 5 http://www.digitalcowboy.jp/Red rectangles in Figure 6 indicate the programming commands, the detected binarized cornea, and three corners of the checkerboard images.Two images which are acquired with the camera 1 for student's eye and the camera 2 for the image of which student is now looking at.At the bottom right in Figure 7 shows the image which is acquired with the camera 2. With the camera 1 acquired image, the system can estimate the gaze location.At the same time, the system can acquire the image of which the student is looking at.

A. Typical E-learning Contents
An example of typical e-learning contents is shown in Figure 8.The image can be divided into four parts, (1) lecturers' face of moving picture, (2) description of the content, (3) presentation materials, and (4) content (presentation procedure, or order of presentation) of the e-learning content.In addition to these, typical e-learning contents include chat www.ijarai.thesai.organd Bulletin Board System: BBS 6 for Question and Answer (Q/A).Usually, content creators would like student to look at the description and the presentation material back and forth.Students, however, used to look at the different location other than the description and the presentation material.Such those students cannot learn effectively.Figure 7.An example of the image which is acquired with the camera 2 at the bottom right.
The proposed system allows identifications of the location of which student is looking at.Also students can hear lecturers voice of instructions.
Therefore, students can concentrate the presentation materials and the descriptions much more by referring to the difference between the location of which students are looking at and supposed location of which content creator would like students look at. 10 of students have to have achievement test with 12 questions.In this case of the typical e-learning content, (2) of the portion, description of the content is the most appropriate portion of which students would better to look at.Table 2 shows the achievement test score and the time duration for which students are looking at one of the four different portions.
For instance, student No.1 looks at the portion no.2 (descriptions of the content) for 10 unit time followed by the portion no.1 (lecturers' face) for 4 unit time, the portion no.3 (presentation materials) for 3 unit time, and the portion no.4 (The other portion) for 2 unit time.
Figure 9 shows relation between the achievement test results and duration time.There is a good correlation between the score and the duration time for which the students are looking at the portion no.2 (R square value 7 of 0.9615).There are no such high correlations between the score and the duration time for which the students are looking at the portions other than the portion no.2.

B. E-learning Contents withMoving Picture and the Annotations
There are a plenty of e-learning contents featuring moving pictures with annotations.Lecture is provided in accordance www.ijarai.thesai.orgwith presentation materials with annotation of marks (hand written, sometime) and lecturers' face.Figure 10 shows such example of e-learning contents.In this example, lecturer makes handwritten marks at the appropriate time and locations.There are two major portions, the portion #1 (presentation materials) and the portions #2 (Lecturer's face).The portion #1 is divided into the portion no.1 and the portion no.2.The portion no.1 denotes the appropriate location where the locations are marked while the portion no.2 denotes the other locations in the portion #1.The portion no.3 denotes lecturer's face while the portion no.4 denotes the other locations out of the portion #1 and #2.Relation between the score of the achievement test and the portions where the students are looking at is shown in Table 3.It is quite obvious that there is strong correlation between the score and the duration time for the portion no.1.Therefore, it may say that the score is excellent if the students look at the appropriate portions, in particular, marked portions in the presentation materials.The time period of learning with this elearning content is 22 unit time.The highest score is made by the student who looks at the appropriate portions for 16 unit time out of 22 unit time.Correlation between the score and the portion no.1 where the students is looking at is around 0.8038 of R square value as shown in Figure 11.

C. Reading Types of E-learning Content
One of the examples of reading types of e-learning contents is shown in Figure 12.In this example, the location of which e-learning content creator would like students to look at is marked with black circle while the location of which the student looks at is marked with green circle.
The distance between both locations can be calculated.If the accumulated distance exceeds a prior determined threshold, then some caution is made by the proposed system.Thus the student may follow the desirable locations.

D. Monitoring Students' Psycological Status
In order to check students' psychological statue during learning with reading type of e-learning contents, Peak Alpha Frequency: PAF of eeg signal is evaluated.
An example of eeg signal frequency components is illustrated in Figure 13.Also the PAF as function of time is shown in the top of Figure 14 together with the blink occurrence (at the middle) and the distance between the location of which e-learning content creator would like students to look at and the location of which the student looks at (at the bottom).www.ijarai.thesai.orgThe red colored lines are correlations among the PAF, blinking and the distance.Namely, when the students look at far from the location of which content creator would like students look at, the students feel a stress and make a blink mostly.Thus the proposed system makes a caution when PAF is getting large (the distance is getting large as well).IV.CONCLUSIONS Method for learning efficiency improvement based on gaze notifications on e-learning content screen display is proposed.
Experimental results with e-learning two types of contents (Relatively small motion of e-learning content and e-learning content with moving picture and annotation marks) show that 0.8038 to 0.9615 of R square value are observed between duration time period of proper gaze location and achievement test score.

Figure 1 .
Figure 1.Proposed glass with two NIR cameras

Figure 2 .
Figure 2. NeuroSky of EEG sensor System block diagram is shown in Figure 3. Peak Alpha Frequency: PAF of eeg signals 4 represent how relax do students during learning processes with e-learning contents [19].

Figure 3 .
Figure 3. system configuration of the proposed system.

Figure 4 .
Figure 4.An example of the acquired eye image with the NIR camera together with the binarized detected cornea and the extracted pupil image

Figure 5 .
Figure 5.An example of computer screen image which is showing commands for the computer program, student's eyes image together with checkerboard for calibration which allows estimation of distance between student and the computer screen.

Figure 6 .
Figure 6.The programming commands, the detected binarized cornea, and three corners of the checkerboard images.

Figure 9 .
Figure 9. shows relation between the achievement test results and duration time.

Figure 10 .
Figure 10.An example of e-learning contents (lecturer makes handwritten marks at the appropriate time and locations).

Figure 11 .
Figure 11.Correlation between the score and the portion no.1 where the students is looking at is around 0.8038 of R square value

Figure 12
Figure 12 One of the examples of reading types of e-learning contents

Figure 13
Figure 13 An example of eeg signal frequency components

Figure 14 PAF
Figure 14 PAF as function of time (at the top) together with the blink occurrence (at the middle) and the distance between the location of which elearning content creator would like students to look at and the location of which the student looks at (at the bottom).

TABLE III .
RELATION BETWEEN THE SCORE OF THE ACHIEVEMENT TEST AND THE PORTIONS WHERE THE STUDENTS ARE LOOKING AT