The Implementation of Computer based Test on BYOD and Cloud Computing Environment

Computer-based test promises several benefits such as automatic grading, assessment features, and paper efficiency. However, besides the benefits, the organization should prepare the enough infrastructure, network connectivity, and user education. The problem upsurges when a hundred numbers of users join the computer-based test. This article proposes Bring-Your-Own-Devices (BYOD), and the cloud computing approach to facilitate a hundred numbers of exam participant. Through the experiment method on 393 students, the article determines five central practices that can be used by the organization who want to implement the massive scale computerbased test. Keywords—Computer-based test; cloud computing; BringYour-Own-Devices (BYOD)


I. INTRODUCTION
In a developing country like Indonesia, adopting a computer-based test (CBT) in high school is challenging.The CBT lab preparation, lab policy, student learning challenges, and staffing preparation [1].The preparation is not free either, for example; the infrastructure should be prepared and invested such as the internet, computer, and network.The problem is when the infrastructure cannot be achieved.
One way to achieve the infrastructure is by adopting Bring-Your-Own-Device (BYOD).It can be used to support learning activities such as CBT in high school or higher education [2].In the implementation of the CBT, many of students prefer to use the mobile phone to access the CBT [3].To prepare the student to get accustomed to the CBT system, several types of research do:  The development of mobile learning system to make student accustomed to the written exam CBT [4].
 The automated of course management and evaluation system using mobile web [5].
 The development of an android mobile application for e-exam [6], [7].
 The knowledge level assessment in e-learning system that uses machine learning and user activity analysis [8].
Based on the researches, it is shown that the solution to implement CBT with BYOD is by creating a custom solution to facilitate the CBT.However, in the limited resources such as a high school that doesn't have a programmer or IT administrator.The solution is somewhat tricky.Hardware and software modularity, cost-effectiveness, network, and training are the main factors to implement cloud computing [9].Therefore, the solution that possible is by utilizing cloud computing.In the recent researches, it is shown that cloud computing has the potential to provide simulator facility or ICT enhanced classroom [10], [11].Some researchers also suggest using cloud computing through a teacher training program [12].Although the implementation of cloud computing on education seem promising and useful [13], some questions need to be answered:

 Is the combination between BYOD and Cloud
Computing can be implemented on a real case?
 What are the critical factors that need to be considered to implement BYOD and cloud computing?The questions are referring to the problem to run the CBT with efficient cost and less technical effort.In order to answer the question, this article conducts an experiment to implement BYOD and cloud computing in the CBT.The detail of the experiment is described in Section II.

II. RESEARCH DESIGN
The research design follows qualitative research design on information system [14].It uses case study design by doing participant observation in natural settings to explain the implementation of BYOD on CBT (Table I).The qualitative method follows structured interview that is documented and administered through online questionnaires.
Several considerations in this research are: 1) The research uses student on eight grades rather than seven grades or nine grades.The middle grade is chosen to balance the digital literature knowledge.
2) The CBT software uses Microsoft Forms.It is well integrated on Microsoft Office 365 that already used widely in the school.It is to make sure that the teacher can author their own exam and troubleshoot as a proctor on the exam.
3) The bandwidth that is allocated uses a wireless network provider.The selection of the provider based on the fastest wireless coverage of that era.
4) The school is located on the Jakarta.The author selects the school based on the school agreed to adopt BYOD on CBY.To mimic the majority school in Indonesia, the author www.ijacsa.thesai.orgchooses a school that doesn't have dedicated hardware for the CBT process.The research was done on four steps, namely, infrastructure preparation, exam preparation, exam execution and monitoring, exam evaluation.

III. IMPLEMENTATION PROSES AND RESULT
The result will be described in four steps as mentioned in the research design.

A. Infrastructure Preparation
Infrastructure preparation prepares the infrastructures needed to run the CBT on BYOD system.The infrastructure preparation is done by doing assessments through simulating the exam process in the cloud environment.This step is done by following three main steps.
1) Creating questions on exam software such as Microsoft Forms.On this step, the teacher creates four question configurations as shown in Table II.
2) Measuring the demand estimation for the users.The demand estimation is measured by using network traffics feature in the browser developer toolbar.The demand estimation is shown in Table II.3) Based on Table II, the required bandwidth is 8 Mbps.Assuming that the traffic workload is 25% for the CBT.It will need at least 32 Mbps.
4) The economic router will only support 16 clients.386 students need 18 routers (386/16).However, because of the physical classroom, the research uses 22 routers to facilitate 22 classrooms.

B. Exam Preparation
The exam preparation process is done by creating an exam question on several subjects.There are eight subjects for four days.The subjects are Mathematics, Science, Social, Religion, Indonesian Language, ICT, Arts, and Society.Each subject has 40 questions.The questions are created by the teachers and uploaded to Forms.
After the questions are uploaded, the questions are verified by the exam preparation team.The preparation will evaluate the exam based on several rules such as: The question and answer should be in a text format, except for the figure or the tables.
 The question should be in multiple choice format

 The figure and the table should be readable on mobile devices
The validated question is uploaded and shared as a specific link.The link is converted into QR codes.The link can be accessed at specific time and date.The link is evaluated through a load test.The load test is done by following three steps which are creating a test plan, recording the usage scenario through Visual Studio UI coded test, preparing the cloud test environment to test the UI coded test based on the test plan.The test plan consists of several rules:  There are two types of questions set.The lightweight exam sets and heavyweight exam sets.The lightweight exam sets consist of 100% of the exam is test while the heavyweight exam sets have a picture on each question.The number of the question is 40 questions.
 The length of the test is 30 minutes, 60 minutes, and 90 minutes.This is based on how the student will finish the 40 questions.
 The load test will be done for 400 users with 100 iterations using Microsoft Azure.

C. Exam Execution and Monitoring
Based on the load test, there are several ways to implement the CBT process on the cloud which are:  The exam process is executed by giving each classroom a delay between 5-10 minutes.This is to make sure that the workload of the network still sufficient to download the exams.
 The devices should free from an update such as Windows update, Android update, or apps update.
 The exam should begin earlier to give some time for the student to turn off an unused app that consumes www.ijacsa.thesai.orgbandwidth, turn on the wireless module to get connected and download the exam.It will take five to ten minutes.Therefore, the student should prepare the devices 30 -60 minutes before the exam.
 The student starts the exam by using the QR codes to access the exam portal.
 The student will authenticate through Office 365 account that distributed a week before the exam.The student should test and use the Office 365 especially Microsoft Forms before they use it for the exam.
 The need of Proctor in each classroom and one technical support for the exam solution.
 The indirect assessment of BYOD implementation is used through an exit survey.The student will answer the survey question in Table III.The survey is done through an online mechanism.

D. Exam Evaluation
After the implementation, it is found several challenges and phenomena that can be discussed as follows:  The upstream issues.The student should wait between 1 -5 minutes to submit the answer to the exam.On the tested environment, the downstream is on 30 Mbps while the upstream is on 10 Mbps.
 International bandwidth usage.The Microsoft Forms uses Office 365 that hosted on the Cloud.Therefore, the system uses international bandwidth rather than local bandwidth.
 Difference use experience on the device.The students use a variety of devices started from a smartphone with 4' screen to 15' laptop screen.The variety makes the student should calibrate the zoom factor and answering process.
The exit survey is done when a student finished eight subjects on four days.In the last exam, the student should answer the survey questions.The survey is distributed on 396 students, with 393 valid samples.The average time to complete for the survey is 3.46 minutes.Some finding of the exit survey that related with the cloud computing adoption for BYOD exam solutions are: 1) The students satisfied with the BYOD exam solution with cloud computing.(4.26 of 5.00) 2) The students feel excited about the BYOD exam solution.(4.28 of 5.00) 3) The students have a good exam experience (4.27 of 5.00).
4) 82.2% of students agree that the next exam should use CBT exam.
5) 62.2% of students agree that any exam can be converted into online exam experience.
6) 61.8% of students agree that the capability of digital literacy helps them to finish the exam without a problem.7) 60.5% of students agree that the CBT exam has no problem at all.While other students don't like the concept because slow internet connection (25.3%) unsupported device (3.4%), and user experiences issues on various screen (10.9%).
8) 61.2% of the students feel that the CBT exam has no difficulty at all.While the 19.6% said an easy task, 1.6% said really easy, and 17.3% students said that the exam software is difficult to operate.9) 8.3% of students said that the internet connection is too slow.While 43.2% feel satisfied and 48.6 feel sufficient.
10) 92.5% of students use their own devices while 7.5% uses the device from the school.
11) 34.1% of students access the internet every day.While the 38.8% access on a weekly basis, 13.2% monthly basis, and 14% for bi-weekly basis.This background will help the digital literacy level for the students.
12) 94.6% of students use smartphone, 3.9% uses notebook, and 1.5% uses tablet.This phenomenon shows that the most used BYOD device is a smartphone.
13) 81.9% of students said that the CBT exam is not hard to understand.18.1% students feel that the CBT exam is really hard to uses.
14) 65.2% of students said that the success of BYOD is mostly because of internet connection, 14.47% said the sufficient hardware, and 11.63% is about the socialization of the software.The rest students feel that the combination of the three is the success factor of BYOD exam.
Based on the exit survey, it is shown that the BYOD implementation for the computer-based test with the cloud computing can be adopted by considering several critical factors such as internet connection, the client device/hardware, the exam execution process, student digital literacy, and ready to use software to do CBT like Microsoft Forms.

IV. CONCLUSIONS
This paper discusses the field experiment of the BYOD implementation in the cloud environment for a computerbased test.As a result.The computer-based test can be used by using BYOD and cloud with five main practices that can be a consideration: 1) Capacity planning should be done to understand the usage pattern of the computer-based test.The usage pattern can be simulated through a cloud load test, coded-UI test, and network test.
2) The three key factors that should be prepared before the exam is the readiness of the internet connection, the supported hardware and the socialization of the software to the students.
3) The exam processes should be prepared such as the device validation, the use of QR codes, and the delay time between classrooms to download the exam.
4) The BYOD implementation should consider user experience for the Smartphone, Notebook, and the Tablet.
5) The Cloud-based CBT should consider the bandwidth consumption between international or local bandwidth, upstream and downstream, and a total of wireless access coverage that should be prepared for simultaneous access.