Proposal of the Support Tool for After-Class Work based on the Online Threaded Bulletin Board

In this paper, based on the assumption that after-class work in an exercise-based course accompanied by group work is done on an online threaded bulletin board system, the authors propose a support tool for the instructors. Specifically, while focusing on the factors that compose a discussion on the online bulletin board, the users who comment, the topics, and the items (keywords) to be discussed, the authors try to visualize the relationships among these factors as network diagrams. The authors also propose indexes, the comment degree and the activation degree, to evaluate communities formed there. Our experiments in which group work was actually implemented with the application of the proposed tool demonstrated that use of the network diagrams and the evaluation indexes served to distinguish the differences between those groups with properly-proceeding discussions and those without such discussions. The authors confirmed that this can enable the instructors to easily discover those students who do not participate in the discussion and groups with sluggish discussions. Keywords—Learning Support Tool; Online Threaded Bulletin Board; Network Analysis


INTRODUCTION
Many colleges and universities in Japan have recently focused on exercise-based courses accompanied by group work.Such courses have been implemented not only in introductory education typified by information literacy education, but also in specialized subjects including business exercises related to business administration.Introduction of this educational method accompanied by group work is believed to increase the depth of students' understanding.Additionally, it is expected to enhance student's communication skills and cooperativeness.[1] [2] [3].This type of course usually handles one theme over several weeks.Therefore, the students are required to do their group work not only during the class hours, but also after the class without the presence of the instructors.However, the instructors have found it difficult to provide the students with necessary guidance for after class time.As a result, differences in the achievements of each student group might be produced.
Under these circumstances, a wide variety of educational support systems that assist students and the instructors have been proposed, such as e-mail, online bulletin board systems, and SNS tools [4] [5].For example, use of online bulletin boards offers not only easier information sharing within the group, but also enables the instructors to confirm the work progress of each group.Although these tools are utilized, the instructors still need to understand the work in progress of each group so that they can provide proper guidance.This creates even greater difficulty especially when the instructor has many groups of students to teach.
Based on the assumption that after-class work in an exercise-based course accompanied by group work is done on an online threaded bulletin board system (Fig. 1), in this paper, the authors propose a support tool that enables the instructors to easily understand the condition of progress of each group, while verifying the effectiveness of the proposed tool.Specifically, focus on the factors such as comment, the topics, and the items (keywords) to be discussed on the bulletin board; the authors try to visualize the relationships among these factors as network diagrams.Moreover, the authors propose indexes, the comment degree and the activation degree, to evaluate communities formed there.
The composition of this paper is as follows: Section 2 describes the current conditions of after-class work of exercise-based courses accompanied by group work, and the problems of such work that were identified and made clear by the previous studies conducted by the authors.Section 3 proposes the support tool for such work with the purpose of solving the problems described in section 2. Section 4 verifies the effectiveness of the proposed tool.Section 5 summarizes this paper and describes future issues.

A. Group work in exercise-based courses
Group work that is implemented during the exercise-based course includes discussions and debates.The purpose of group work stems from the following three points [6].
 Enhancement of the ability to analyze problems based on diversity of perspectives  Enhancement of the ability to emerge with solutions and ideas  Enhancement of the ability to cooperate while working in a group Issues and themes, which are used in corporate training programs, are actually handled in many exercise-based courses.These issues and themes are based on educational materials, where the points to be discussed are clearly indicated and can be expressed with keywords.In many cases, therefore, the instructors find it easy to use them in their lectures.

B. Group work based on online threaded bulleting boards
As mentioned earlier, after-class work is essential when group work needs to be conducted over several weeks.For this reason, online bulletin boards and SNS tools are commonly used by which users can continue their work asynchronously and remotely.As described above, systems that support such work have also been proposed [4] [5].Actually, however, online threaded bulletin boards and similar systems are often used because they are easy to use and users can easily understand the flow of comments made [7].Where group work is conducted on an online threaded bulletin board, on one theme, the users refer to and make comments on the same topic.This makes it easier to understand the condition of progress of each topic (Fig. 1).

C. Problems in after-class work
By using an online threaded bulletin board, the authors have conducted after-class work in exercise-based subjects for freshmen of private institutes and for those that are juniors and seniors of liberal arts universities in the Tokyo metropolitan area of Japan.The authors also analyzed the communities formed there [8][9] [10].Our analysis clarified that differences could be produced in the achievement level depending on the group, while disparities were also caused in the work level between the members within the group.Furthermore, when there were many groups, it became burdensome and difficult for the instructors to have a good understanding of the condition of each group.
As a result, these findings clarified that the instructors would need to check the checkpoints shown below for providing proper guidance in order for the students to do their work smoothly.
 Participation level of each group member  Level of coverage of items to be discussed  Properness of the proceedings (threads)  Transition of the progress condition

III. PROPOSAL OF A SUPPORT TOOL FOR AFTER-CLASS WORK
Based on the problems related to after-class work clarified in section 2, in this paper, the authors propose a support tool that makes it easier for the instructors to understand the condition of after-class work based on the online threaded bulletin board.
Specifically, the authors propose a function to visualize communities that are formed on the online threaded bulletin board and the indexes used for evaluating the communities formed.

A. Proposal of a function to visualize after-class work
In this paper, in order to achieve support for understanding the work condition on the online bulletin board, the authors focused on the important factors to understand the work condition, users who comment, topics, and items (keywords) to be discussed.Based on this focus, the authors propose a function that visualizes the four relationships shown below between two factors.

 Relationship between the users who comment and the topics
 Relationship between the users who comment and the items to be discussed  Relationship between the topics and the items to be discussed

 Chronological changes in the above-mentioned relationships
The network-analysis [8] was used as the method for visualizing communities.Network analysis that the authors used here is a method that captures relationship patterns within the community as networks and that quantitatively expresses the patterns' structures as undirected graphs [11].This method is used for analyzing communities [12][13][14] [15][16] [17].
In this paper, while expressing the relationships between two factors as one-on-one edges, the authors created undirected graphs based on these edges.In order to create these graphs, the node shape is changed according to each factor so that the node's attribute can be expressed depending on the node size.
In order to create network diagrams, the authors adopted the Kamada-Kawai method [18] based on the spring model, and used Pajek [19] which is software for network analysis.In the Kamada-Kawai method, strongly-related nodes are plotted in a near positional relationship; however, the absolute position of the node is not fixed.This means that a different network diagram is produced each time it is created.The purpose here is to understand the progress condition.Judging that the relative positional relationship between nodes would be important, the authors adopted this method.Fig. 2 shows the overview of the proposed visualization function.www.ijacsa.thesai.org

1) Visualization of the relationship between the users who comment and the topics
Here, a network diagram is created by capturing the relationship between the users who comment and the topics as edges shown in Fig. 3.The size of the topic's node and that of the node of the user who comments are proportioned to the number of comments included within the topic and the number of comments made by the user.The size of the node of the user who comments within this network diagram can make it easier to understand the participation level of each user who comments.The relationship between the topic and the user who comments can probably make it easier to understand the work progress condition.

2) Visualization of the relationship between the users who comment and the items to be discussed
The objective of work focused on in this paper is education as described in section 2.A.Therefore, the items to be discussed are clearly indicated, while keywords can be registered preliminarily.Here, a network diagram is created by using these keywords to connect the items (keywords) to be discussed with the users who comment as a one-on-one edge.As for keyword nodes, the node size expresses the number of reference counts.
The sizes of the nodes within this network diagram can probably make it easier to understand bias in the items to be discussed.

3) Visualization of the relationship between the topics and the items to be discussed
Here, the keywords included in the comments made and the topics referred to are connected with edges.As for the nodes of the topics and keywords, the node size similarly expresses the number of reference counts.
The relationship between the topics and the keywords in this network diagram probably makes it easier to understand the condition regarding the items to be discussed.

4) Visualization of chronolunnyouogical changes
Here, the network diagrams created in sections 3.A.1, 3.A.2, and 3.A.3 are saved in specified time intervals, while these diagrams are presented chronologically at the time of visualization.This function can serve to visualize the network formation process.

B. Proposal of indexes to evaluate communities
Next, indexes to evaluate communities that are formed on the online bulletin board are proposed.Based on previous studies, in this paper, the authors define the characteristics of ideal communities as the following two points.

 Many topics are commonly referred to by all the members
 All the users comment about the items (keywords) to be discussed, with less bias by the user Here, the authors propose indexes which are the comment degree and the activity degree.The following shows the definitions and how to obtain these indexes.
Suppose that the number of the users who comment is , and the number of the topics is .The number of keywords stated by the user ( =1,2,…, ) in the topic ( =1,2,…, ) in a certain group is .The percentage of the keywords referred to by the user in the topic is expressed as (Equation ( 1)).

∑ (1)
This percentage is obtained according to each topic by the user.The total value of all the topics is expressed as the comment degree of the user (Equation ( 2)).Namely, user with a higher comment degree is determined to be making useful comments within the group.

∑
(2) This comment degree is then obtained according to each user in order to obtain the geometric average from all the comment degrees obtained.This geometric average obtained is regarded as the activation degree of discussion, (Equation (3)), on the online bulletin board.

√∏
(3) This activation degree becomes higher where many topics exist and where all the users are evenly discussing the keywords.Therefore, a group with a higher activation degree is determined to be having discussions progressing properly.www.ijacsa.thesai.orgTherefore, providing the instructors not only with network diagrams, but also with the comment degrees and activation degrees can make it easier to understand the condition of each user who comments and the discussion condition.This probably enables them to offer more proper guidance to the students.

IV. EVALUATION
To verify the effectiveness of the proposed function and the evaluation indexes, the authors actually implemented a group work assignment on an online threaded bulletin board.Through this attempt, the authors analyzed the network diagrams, which were obtained from groups with high results and those with low results, and evaluation indexes.

A. Overview of evaluation experiments
The authors used a consensus game which is typically used for corporate training programs, such as the NASA game, as the group work assignment for experiments.The consensus game is an assignment where a certain theme is provided and the group members discuss the theme while trying to derive the conclusion within the specified time period.The purpose of this game is to gain a consensus from all the members of the group.In this evaluation experiment, the authors applied an assignment of ranking much-needed items that remain under a certain crisis situation.Targeting the juniors and the seniors of a private liberal arts university, the authors implemented the above-mentioned assignment on an online bulletin board.Since the purpose of this study is to support after-class work, the authors implemented two experiments consisting of a long-term experiment (a week) and a short-term experiment (about an hour) as a control experiment.A different theme was applied for each assignment.

1) The result of the short-term assignment (Experiment 1)
Tab. 1 shows the results of experiment 1.The score of each group was calculated as the sum of squares of the difference in the correct answer and the answer of each group.Based on this score, groups were ranked from groups with lower scores.2) The result of the long-term assignment (Experiment 2) Tab. 2 shows the results of experiment 2. This table shows the results of implementing this assignment under an asynchronous and remote environment.

C. Evaluation of the function to visualize after-class work 1) Analysis based on the network diagrams of the relationships between the users who comment and the topics
Fig. 4 and Fig. 5 show the network diagrams of the relationships between the users who comment and the topics at the time point when experiment 1 and experiment 2 ended.Here, users who comment were shown circle, and topics were shown square.
In the network diagrams created by the Kamada-Kawai method adopted in this paper, the users who commented on the same topic or the topics consisted of the same users tended to be plotted in a near positional relationship.In the case of groups with high marks (group A in experiment 1 and group G in experiment 2), common topics discussed by all the users increased.In this network diagram, therefore, the nodes of the users who commented or the topic nodes tended to concentrate in the center.Experiment 2, which was conducted for a longer time outside of class hours, confirms that those users taking the lead in the discussion (core users) and free riders remarkably appeared, when compared to experiment 1.
Based on the summary of the results of Tab. 1 and Tab. 2, and Fig. 4 and Fig. 5, the following characteristics can be observed in the network diagrams of those higher groups. There exist many topics that are being discussed by all the members, and the nodes of the users who comment or the topic nodes tend to concentrate in the center  All the members comment evenly, while no free riders exist The above-described analysis shows that by using this network diagram, the instructors can provide more practical guidance by directing the students who make fewer comments to participate in the discussion, or by directing them to divide topics to develop the discussion when the topic nodes become too large.This can enable the instructors to offer support for the students with very little effort in order to promote well-balanced discussions.

2) Analysis based on the network diagrams of the users who comment and the items to be discussed
Next, Fig. 6 and Fig. 7 show the network diagrams of the users who comment and the items to be discussed at the time point when experiment 1 and experiment 2 ended.Here, users who comment were shown circle, and keywords were shown rhombus.
In experiments, keywords were defined as words that suggested the correct item names.As shown by Fig. 6 and Fig. 7, the network diagrams feature that the nodes of the users who made comments that concentrate in the center, and around these nodes keyword nodes are plotted.This is because when all the members participate in the discussion while commenting about all the keywords, the nodes of the users commenting concentrate in the center.The following characteristics can be observed in the network diagrams of those lower groups.
 Some users do not comment about the keywords  The nodes of these users are plotted away from the nodes of other users  Some keywords are referred to less  The sizes of some keyword nodes are biased These tendencies were more remarkably observed in experiment 2 which was conducted outside of class hours.
The above-described analysis shows that by using this network diagram, the instructors can provide more practical guidance by directing the students who less comment about the keywords.This also can make it easier when needed to indicate that some keywords being discussed are biased.3) Analysis based on the network diagrams of the topics and the items to be discussed Next, Fig. 8 and Fig. 9 show the network diagrams of the topics and the items to be discussed at the time point when experiment 1 and experiment 2 ended.Here, topics were shown square, and keywords were shown rhombus.From Fig. 8 and Fig. 9, the following characteristics can be observed in the network diagrams of those higher groups.
 Many topics are connected with all the keywords  The topic nodes concentrate in the center  As for individual keywords, many topics are separately discussed  Topic nodes are plotted outside www.ijacsa.thesai.orgThe above-describe analysis shows that use of this network diagram enables instructors to direct the students to set additional topics for the keywords where biased discussions are going on.

4) Analysis based on the network diagram with the discussion progress condition visualized
Fig. 10 and Fig. 11 show the network diagrams in which the daily progress condition of the highest group G and the lowest group I on the online bulletin board was visualized.The numbers in the upper left in these diagrams indicate the dates.
Comparison of Fig. 10 and Fig. 11 confirms that the discussion proceeds from the initial stage (the 3rd day) in the highest group G.Where the discussion does not proceed during the initial stage, therefore, it is important to encourage the users to advance their discussion.These network diagrams make it easier to find free riders.Therefore, use of these diagrams enables the instructors to guide those users with fewer comments to comment more during the initial discussion stage.
As described above, the authors were able to find that the proposed network diagrams served to distinguish the differences between groups with discussions proceeding properly and those without such discussions.As for those groups with sluggish discussions, therefore, use of this proposed function makes it easier to indicate the problems existing within the group.
Experiment 1 had only a short one hour implementation time.For this reason, experiment 1 had the tendency to have less bias, such as the number of comments that varied by each user, when compared to experiment 2. On the other hand, this problem related to bias could be observed remarkably in experiment 2 which was conducted outside of class hours.Therefore, in after-class work, use of the proposed tool for visualization probably makes it easier to understand the problems related to the discussion progress condition.

D. Analysis based on the evaluation indexes for communities
Tab. 1 and Tab. 2 show that the activation degree by the group is almost in the same order as the score-based evaluation.Use of the proposed indexes is probably determined to be valid in evaluating the discussion progress condition.However, as shown by group E in experiment 1 and by group J in experiment 2, there exist groups with high activation degrees but low scores.It is important to combine the indexes such as the number of topics or comments with network diagrams.
Next, Fig. 12 and Fig. 13 show the comment degree of the users who commented in each group.These diagrams confirm that the nodes of users with high comment degrees suggest the core users, while the nodes with low comment degrees suggest free riders.In addition, a number of users with high comment degrees exist in groups with high marks (groups A and G), while users with low comment degrees exist in groups with low marks (groups J and K).
The above-described analysis confirms that providing the instructors not only with network diagrams, but also with the indexes such as comment degrees and activation degrees can make it easier to understand the level of each user's participation in the discussion and the discussion progress condition of each group.This probably enables them to offer more proper guidance to the students.Visualization based on the proposed support tool indicates the problems within the group without using the contents of comments made on the online bulletin board.In order to understand what kind of conversation is actually conducted within the group, it is necessary to view the contents of each comment made.The purpose of this support tool is to indicate the problems within the group.Therefore, the authors need to conduct more experiments by having the instructors use this support tool in the future.By doing so the authors can clarify to what degree the problems can be identified without viewing the contents of comments, and how much difference is observed when compared with the case of identifying the problems by checking all the comments that are made.

V. CONCLUSION AND FUTURE ISSUES
In this paper, based on the assumption that after-class work in an exercise-based course accompanied by group work is done on an online threaded bulletin board system, the authors proposed a support tool for the instructors.Specifically, while focusing on the factors that compose a discussion on the online bulletin board, the users who comment, the topics, and the items (keywords) to be discussed, the authors tried to visualize the relationships among these factors as network diagrams.The authors also proposed indexes, the comment degree and the activation degree, to evaluate communities formed there.
Our experiments in which group work was actually implemented with the application of the proposed tool demonstrated that use of the network diagrams and the evaluation indexes served to distinguish the differences between those groups with properly-proceeding discussions and those without such discussions.The authors confirmed that this can enable the instructors to easily discover those students who do not participate in the discussion and groups with sluggish discussions.
In the future, in addition to examining more elaborate indexes, the authors are going to verify the further effectiveness of the proposed tool by conducting evaluation experiments with the instructors.

Fig. 1 .
Fig. 1.Example of online threaded bulletin board system

Fig. 2 .
Fig. 2. Overview of the proposed visualization function

 Experiment 1 :
To have the members come to a conclusion in a short period (about an hour)  Experiment 2: To have the members come to a conclusion over a longer period (a week) B. Experimental results

Fig. 4 .
Fig. 4. The network diagrams of the relationships between the users who comment and the topics (Experiment 1)

Fig. 5 .
Fig. 5.The network diagrams of the relationships between the users who comment and the topics (Experiment 2)

Fig. 6 .Fig. 7 .
Fig. 6.The network diagrams of the users who comment and the items to be discussed (Experiment 1)

Fig. 8 .Fig. 9 .
Fig. 8.The network diagrams of the topics and the items to be discussed (Experiment 1)

TABLE I .
THE RESULT OF THE SHORT-TERM ASSIGNMENT