Generating Test Cases using Eclipse Environment – A Case Study of Mobile Application

In Software Development Life Cycle (SDLC), there are four phases involved. They are analysis, design, implement and testing. Testing is done to ensure the functionalities of the system are correct. There are many approaches to software testing. It is usually divided into two approaches: manual testing or automatic testing. However, these days, with the rapidly advanced technology, performing software testing manually has become hugely laborious but still doable. Therefore, experts of the software development field are beginning to go for automatic testing. This paper presents a case study of mobile application and discusses how test cases can be generated automatically from the application using different automatic tools. Three software testing tools have been used to generate test cases automatically. The results from generating test cases automatically from these three tools are then being compared together with the results of generating test cases using manual testing technique. Keywords—Software testing; automation testing; test cases; Eclipse Environment


I. INTRODUCTION
In Software Development Life Cycle (SDLC), software testing is explained as the phase where a program is executed to be evaluated with the intention to find faults [1]. Although the SDLC is considered as an approach of efficient system development, software testing plays an important role as it assists in finding system deficiencies [2]. As such, testing is done to any software components, making it a vital process considering it aids in discovery of how good it works, validating the quality of the software system. To ensure that developed software components are in good quality, it is crucial to do software testing for the verification and validation to be done properly [3]. Considering how costly a software development project can amount to, testing becomes even more important, as prevention of even more highly cost of the software development. Therefore, it is important that the process is began at early stage during development [4] instead of being carried out by the end of the project development.
Software testing can be accomplished in two ways; either manually or automatically [5]. Manual testing is carried out by software testers without the help of any tools; it is a testing method which is most primitive compared to its peers [6]. On the other hand, contrary to manual testing, automatic testing is performed with assistant from automated testing tool whereby test cases will be generated [7]. The performance capability and functionality of all test cases are to be justified. Testing tools are highly required to perform automatic testing. It plays a crucial role during the testing phase of the SDLC [8]. Several known tools include Robotium [9], Appium and Selenium [10].
This research study main aim is to generate test cases automatically from the existing tools and compared the time taken to generate test cases automatically among the tools. The case study is based on an existing Android mobile application called MyNetDiary [11]. The research shall be able to automate the process of generating test cases. There will be three tools used in the research which are JUnit4 [12], TestNG [13], and EPiT [14]. The results of time taken for each tool to generate test cases automatically will be compared together with the time taken to generate test cases using manual testing.

II. TECHNIQUES OF SOFTWARE TESTING
Generally, there are two ways to achieve software testing and those are by manual testing or automatic testing. The idea of manual testing is hugely primitive where the tests are executed in the absence of any tools [7]. Differing with manual testing is the automatic testing by which it is done with the help of automatic testing tools [14]. It is believed that by using automatic tools, the trend of automation testing has managed to have better usability, robustness, and correctness [8].
Software testing levels have been categorized into four levels [15]. These four levels include unit testing, integration testing, system testing, and acceptance testing which is shown in Fig. 1. Unit testing focuses on a software system's smallest element which is also known as modules; they are tested independently. Following after the unit testing is the integration testing where the main concept of it is testing the different integrated modules together. For most software project, the value of system testing being carried out is approximately up to 90 percent [16]. And then the last level of testing is the acceptance testing, performed by targeted end users [17]; it has variants of types which include alpha testing, beta testing, business acceptance testing, and the user acceptance testing. www.ijacsa.thesai.org    2 shows the visualisation of three common software testing approaches [17]. Over the time period of rapid software development expansion, the common software testing techniques known to most are the black-box testing, white-box testing, and the grey-box testing methods. The grey-box method is a combination of the black-box and whitebox testing methods [18].

A. Black-box Testing
Going by many other names such as behavioral testing and functional testing, black-box testing is usually driven without test models or even precise formal documented specifications [18]. The idea of black-box testing is the software testers do not know which of the system's component is being tested. As shown in Fig. 2, the idea of black-box testing is where the users or testers are without knowledge of the system's internals. The testing method concept is accepting inputs and producing expected outputs; black-box testing method borders on the foundation aspects of the system [19].

B. White-box Testing
There are many other nicked names to white-box testing method. Some of them include clear-box testing and glass-box testing. Just as the visual on Fig. 2 suggests, it is a testing method whereby the internals of the system are fully known [20]. As its nickname (clear-box testing), the back end of the system (or its components) is known to testers making it highly efficient in bugs-detection [21]. However, in largescale software systems, this method is seldom used.

C. Grey-box Testing
Being the combination of black-box testing and white-box testing is the grey-box testing technique [22]. Fig. 2 shows the internals of the system is relevant to the testing being carried out known by the testers. The concept of grey box is commonly known of testers having bits of internal working but going against its specifications [17]. The method typically applies reverse engineering but is not categorized as biased and intrusive, therefore the testers are not inclined to gain access on the internal source code.

III. RELATED WORK
Li et al. [23] present DroidBot, an automatic software testing tool which is compatible to most Android mobile apps. DroidBot is said as something that is lightweight and test on UI-guided input generators. It does not require any instrumentation. DroidBot also makes use of malware analysis as it uses a model-based generator that has information about app under test (AUT) from device at runtime, enabling it to trigger sensitive behaviours.
Alotaibi, & Qureshi [10] discuss a new framework to be used for automation testing on mobile application which will be using the Appium framework. According to them, in order to ensure high performance application within a short-given time, the automation of software testing is highly necessary. They specifically discuss Appium as it is considered as a power tool that helps in delivering features. What Appium does, to be precise, is the direct automation on mobile devices. It supposedly works for almost all of hybrid, native, applications of mobile-web for iOS and even Android. [24] introduce Sapienz which is an Android testing approach that has significantly performed better than even the widely-used tool known as Android Monkey. According to them, Sapienz is better than Monkey is due to the fact that Monkey does automation testing in a deliberate unintelligent way of randomness. Sapienz, on the other hand, is a new automated testing that combines traditional automated testing with the quirks of expanding it to Android testing.

Mao, Harman, & Jia
Dolan-Gravitt et al. [25] focus on PANDA's four principal criterion; the system's ability to record/replay, the system's plugin architecture, the system's capability in single analysis execution process on multiple architectures, and lastly the ability of Android systems emulation. PANDA is versatile and has simplicity, allowing support of new myriad of architectures and devices with no extra labour. The replay method itself is able to overcome the complexity of operating systems as it is able to record boot for myriads of operating systems. The system is more widely received considering its full repeatability features, a big convenience for dynamic analysis. Hence, considering PANDA is not focused solely on record and replay, it is adequately different than QEMU 2.1.0's numbers just as shown on the table below. However, PANDA takes almost the same amount of time as QEMU 2.1.0.
Hussain, Razak, & Mkpojiogu [26] discuss the perceived usability sentiments regarding the automated testing tools that exist for mobile testing. They discuss that many mobile application developers are using automated testing tools these days and that include MonkeyTalk, Robotium, and more. They state how it is no longer foreign that automated testing tools are gaining trend as it greatly reduces the time taken to www.ijacsa.thesai.org conduct the process of testing, excluding errors, and even omitting possible errors due to human factor. They argue how it has become highly important for automated testing tools to be of good usability as automated testing tools should not only support either native or hybrid, but they shall be able to do both. And that includes for Android and iOS.
Rosziati Ibrahim et al. [14] discuss the automatic testing tool called EPiT for generating test cases automatically. EPiT is a plug-in tool that can be installed in Eclipse environment. EPiT has a parser that reads the source codes line by line and then extracts all the attributes and functions from the classes and finally generates the test cases of all functions automatically.
Salihu et al. [27] propose a model to generate test cases from mobile application based on GUI. AMOGA framework is used for the generation of test cases with two important algorithms embedded within the framework. They are greedy algorithm and crawler algorithm.
IV. UML SPECIFICATION UML diagrams are considered as the de-facto standard tool being used for the documentation of object-oriented modelling [28]. Two diagrams have been used for this project. They are use-case diagram and class diagram. Fig. 3 shows the use-case diagram of the research study. Based on Fig. 3, the actor is a user who can execute the tool in order to read the source code files of the case study. After doing so, it will be able to extract the classes and interface information, as well as checking the functions dependency. At last, it will generate the test cases.

B. Class Diagram
The class diagram portraits the classes that are going to be implemented during the development cycle. Fig. 4 shows the class diagram of this research study.  Based on Fig. 4, it shows the specific of which methods belong to either the user, tool or the Eclipse IDE itself. The diagram does not exactly illustrate the directional work flow of the testing but it shows the classes that are being used for the implementation.

V. RESEARCH METHODOLOGY
The research study follows a specific process that consists of four stages to be carried out in order. For this research study, it will include a total of four major stages, the first stage being the requirement analysis. Next, it is followed by the design, implementation, and testing stages in an orderly manner. These four phases are illustrated in Fig. 5. Fig. 5, the requirements analysis stage is critical to this research study. As stated by Shukla, Pandey, & Shree [29], many other phases depend on requirements engineering and that includes the design, coding and testing. In this research study, this phase includes identifying the necessary tools and requirements needed. After identification, the requirements needed have been noted. The case study of the research is based on an Android mobile application which is MyNetDiary [11]. From MyNetDiary, the scope is further narrowed down to its 3 modules. The platform used for the research development is Eclipse IDE with the implementation of the Java programming language. Several other software and plugins are required for this research. As the codes of MyNetDiary mobile application cannot be fully obtained, it is determined that the software testing technique used is greybox testing.

Based on
As the analysis phase, design phase is also included in SDLC. On a generic sense, during the design phase, the technical details of a software project are discussed, and this usually comprises of several aspects such as the technologies to be used, constraints, design approach, and so forth [30].
For the implementation phase, Fig. 6 shows the steps for implementing the tool.
Based on Fig. 6, the implementation process begins with first reading the source code file of the case study. Once the source codes are obtained, the automated testing tools which are running on Eclipse IDE will identify the classes and functions to be extracted. After that, the automated testing tools will begin generating the test cases automatically and the time taken for each of the tools and techniques will be observed, and recorded. For each software testing methods, both manual testing and automation testing; the tests will be www.ijacsa.thesai.org run a total of 5 times for each Module 1, Module 2, and Module 3. This was done in order to get the optimal and most accurate data for the research. Lastly, the evaluation of time taken between the manual and automated testing will be made.

A. Manual Testing Flowchart
There are three basic activities to be done during the manual testing process as shown in Fig. 7(a). The case study file will first be run and executed, and then software tester will start inserting inputs. The time taken for the process to generate test cases will be recorded.

B. Automatic Testing Flowchart
Similar to the previous process of manual testing. Automatic testing also follows several steps on generating test cases as shown in Fig. 7(b). The flowchart consists of four activities. The step begins with source code files of the case study being read. Its classes and interface information will be extracted, and the functions dependency will also be checked. Lastly, the test cases will be automatically generated by the selected tools.

VI. RESULTS AND DISCUSSION
Based on MyNetDiary [11], three modules have been used in order to generate the test cases. Table I shows the details of these three modules.
The data recorded from all the tests run during the research have been tabulated as each module is run at least five times for each respective automatic testing tools. The formula used to calculate the average time taken of tests run is: where ∑ indicates the summation of the time taken to run for each module. Table II shows the calculation of data on the results of time taken to manually generate the test cases for all three modules.     Table III shows the calculation of data on the results of time taken for Junit4 [12] to generate the test cases automatically for all the three modules. Fig. 9 is the graphical diagram from Table III. It depicts the value of the average time taken to generate test cases automatically for Module 1, Module 2, and Module 3. It took 1.363s, 1.093s, and 0.598s to generate the test cases for Module 1, Module2, and Module 3 respectively. Table IV shows the calculation of data on the results of time taken for TestNG [13] to generate the test cases automatically for all three modules.     Table V shows the calculation of data on the results of time taken for EPiT [14] to generate the test cases for all the three modules.

E. Comparative Analysis
Table VI and Fig. 12 are the tabulated data and graphical diagram representation of all testing methods. The time taken to generate the test cases using manual testing takes a significantly longer time than the time taken for the automatic testing tools to generate the test cases. This is clearly shown in Table VI. From Table VI and Fig. 13, the time taken to generate test case using manual testing takes a significantly longer time than the time taken for the automatic testing tools to generate test cases. Among the three automation tools used, JUnit4 took the significantly greatest time which total reached more www.ijacsa.thesai.org than 3s. Meanwhile TestNG only took 0.05s to generate all test cases for all modules. Meanwhile, EPiT took the shortest time at only 0.007s.
Regarding the differences in time taken to generate test cases of the modules, this can be justified on the code lines of the case study. While the case study has simple time complexity of O(1), the total number of lines for each modules significantly differs with Module 1 having the most number of lines written, followed by Module 2, and Module 3. This causes for the time taken to generate test cases to differ from each of the respective modules. Beside from that, we can conclude that automatic testing is definitely better than manual testing. However, it needs to be noted that manual testing cannot be simply abandoned as it is still necessary for several tasks in any software development projects.
From Table VI, it is noted that manual testing has the biggest time difference compared to the others, which is just as expected. This is because manual testing demands a lot of resources which is one of them is the time resource [31]. Among the three automated testing tools, it is noted that the differences of time taken to generate test cases between JUnit4 and TestNG, as well as EPiT; JUnit4 takes the longest time. In one paper, Kumbhar, Gavekar, & Kulkarni [32] stated that JUnit is quite a lacking tool in generating test result compared to other testing tools. Meanwhile, it is no surprise that TestNG took shorter time than JUnit4 in generating the test cases, as according to Jacob and Karthikevan [33]. EPiT [14] is the latest software testing tool that has the shortest time to generate test cases automatically for the three modules. EPiT uses the algorithm in [34] in order to reduce the redundancy of test cases generated.

VII. CONCLUSION
This paper has discussed and compared the three automatic tools namely Junit4, TestNG and EPiT for generating test cases automatically. All three tools are plugged into Eclipse IDE. The time taken to generate the test cases has been compared among the tools. After the tests are run, it has been observed that JUnit4 took the longest time to generate all test cases, the time taken being almost up to 3s. Meanwhile TestNG only took 0.05s to generate all test cases for all modules. On the other hand, EPiT took the shortest time at only 0.007s. Therefore, EPiT gives the shortest time in order to generate test cases automatically. Beside from that, we can conclude that automatic testing is definitely better than manual testing. However, it needs to be noted that manual testing cannot be simply abandoned as it is still necessary for several tasks in any software development projects.