Scalability and Performance of Selected Websites of Universities : An Analytical Study of Punjab ( India )

Today, education has emerged as a major area of commercial activities. The access to various University websites through Internet has opened up new opportunities for the beneficiaries. The creation of these websites fully serves the purpose of educational institutions in advancing and achieving their goals. The varied information made available on these websites and the minimum transaction response time to address the queries of end-users can go a long way to influence their decision in selecting a particular course and an institution. The issue assumes greater significance especially in a developing country like India where a website development and deployment activity is primarily facing the shortage of formalized website design techniques and testing procedures. The performance of most of the University websites is reasonably well, but when accessed by only a few concurrent users. Thus, the aim of this study is to analyze and compare the scalability and performance of selected University websites of Punjab (India) by means of load testing. Simulation of realistic users’ behavior is achieved through LoadRunner, a software tool for performance testing. Of all the University websites under study, on the basis of their scalability and performance, it has been found that the websites of Deemed and Central universities are the most and least efficient respectively. The findings of this study can be of great significance for the higher educational institutions in improving the performance quality of their websites resulting into their better ranking and satisfaction of the stakeholders. The paper also outlines the scope for further research in the area under study. Keywords—Hits per second; scalability; performance; throughput; transaction response time; university websites


INTRODUCTION AND RELATED WORK
The advent of World Wide Web has greatly influenced every business in one way or the other.Frankly speaking, it has completely transformed our lives by providing access to a vast knowledge and information on every subject.Millions of Web applications serve billions of Web pages daily through software systems.Web applications have become interactive, dynamic and asynchronous through a number of revolutions [1], [2].Web applications are an integral part of any website; and subsequently, existing websites have evolved from static information pages to dynamic and service-oriented applications.These are highly used for a broad range of activities on a daily basis in the health care, education, consumer business, banking and manufacturing sectors [3].Academic institutions make their websites for a wide range of purposes which mainly include distribution of information to the public, delivering online learning facilities to students, promotion of their educational and research programmes and the like.Thus, through this medium, the universities are communicating with and disseminating information to various stakeholders.Students / prospective students, employees / prospective employees, parents, ranking organizations, and the media were identified as the regular users of academic websites [4].If it is assumed that universities are the brands for education marketing, then websites emerge as a crucial part of this marketing process.
The website of an institution is a gateway to its information and services offered.As such, it should meet all the requirements of its stakeholders.A poor website of an organization can spoil its brand image, loose the potential customers, and weaken its organizational position.Thus, it is of utmost importance to explore those factors which highly influence the users" attitude towards a website which helps the organizations to chalk out a successful e-strategy for the purpose.
Many research studies have established the relationship between Web quality factors and user acceptance [5]- [12].This is due to the reason that Web quality factors can be controlled by an organization; and these can influence users" beliefs and their behavioral intention.A poorly performing website offering a product may fail in its objective.An effective website keeps a balance between end users" expectations and experience with the services being offered.Only those organizations can successfully achieve their goals which are able to lift end users" experience to a level that exceeds expectations.As the end users" expectations always keep on increasing, it is essential for organizations to improve their quality constantly.It raises the question: what should be improved to keep the end users satisfied?Lin [13] emphasized not only on the quality of information, but also the quality of system.System quality is technology-based and enables the users to get faster responses with more convenience and privacy [14].The time to download a Web page is an important factor for most of the Internet users.The study undertaken by Hoffman and Novak [15] has established a positive correlation between website loading time and user satisfaction.Therefore, fast loading becomes essential for online transactions to be finalized.When loading time is below the expectations of users, they will either prefer to redirect the search engine to another site or give up their search [16]- [19].Thus, it can be assumed that due to technology advances end users expect sites to be even quicker.The poor performance of a website may downgrade it in search engine results rankings.www.ijacsa.thesai.orgAlthough a good number of websites exist in the academic domain today, yet only a small percentage of these websites satisfy their end users" requirements especially in developing countries like India.It has been observed that during the admission days, many of the University websites are not able to perform well on account of links opening slowly, some of the important links not opening at all, and lost payment transactions.However, such problems can be attributed to the limited use of formalized website design techniques, rapid advancement in Web technologies, limited experience and knowledge of individual designers and developers, short development and evaluation life cycle, lack of formalized website testing procedures, and less resource allocation for website design and development project [20].Hite and Railsback [21], in their study, confirm that University websites have developed almost as rapidly as corporate websites.But a proper engineering approach for building a web system is not followed; and the engineering process itself is still to be engineered.
The whole scenario gives rise to the need for a thorough analysis of websites both during development and after deployment in order to ensure their conformance to high standards of quality especially in terms of performance.Website testing is considered from two distinct perspectives.The first perspective focuses on identifying the failures in functionality of a website, while the other perspective verifies the conformance of the site behavior with the specified nonfunctional requirements.The functionality of a website means what a system is supposed to do, while its non-functionality requirements means how a system is ought to be [22], [23].The non-functional requirements of a system are often described as its "quality attributes".The non-functional requirements that a website is usually required to satisfy, either explicitly or implicitly, are shown in Figure 1.Most of the methods and approaches followed to test the functional requirements of "traditional" software can also be used for testing the websites.However, traditional testing theories and methods cannot be used as such to verify each non-functional requirement because of the peculiarities and complexities of websites.The Web application system is typically composed of a database (or the back-end) and Web pages (the front-end) with which users interact over a network through a browser.Garousi et al. [24] considered the website as a distributed system, with a client-server or multi-tier architecture.The other main characteristics are as follows:  A wide number of users can access it concurrently.
 The use of different hardware, operating systems, Web browsers and network connections generates heterogeneous execution environments.
 A large variety of software components establish the heterogeneous nature of a system which it generally includes.These components can be constructed of different technologies / programming languages, for instance, Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), JavaScript on the clientside and Hypertext Preprocessor (PHP), Ruby, Java on the server-side.
 The dynamic nature of a system makes the software components generate at run time as per user inputs and server status.
The characteristics as mentioned above pose a number of technical and non-technical challenges before the testers to effectively test the sites; and additional efforts are required in web testing [25].Thus, specific testing activities need to be explored to test the non-functional requirements of a website.
Performance testing is a subset of performance engineering which strives to build performance into the implementation, design and architecture of a system.Liu [26] considers performance testing as a measure to find how fast an application can perform certain tasks, whereas scalability is used to measure performance trends over a period of time with increasing amounts of load.Performance testing consists of simulating multiple virtual users that send requests to the tested server concurrently for evaluating the application performance under a particular load.It is defined as the technical investigation carried out to identify the hurdles in a system, supports a performance tuning effort, determines compliance with performance goals & requirements, and/or collects other performance-related data enabling the stakeholders to make decisions related to the overall quality of the application/system being tested [27].The performance of any system depends upon many parameters like response time, high throughput from the system, etc. [28].In 2006, Google revealed that by reducing the size of web page "Google Maps" from 100KB to 80KB, their traffic shot up by 10% in the first week and then 25% in the following three weeks.The results produced by Amazon in 2007 were also the same.It was revealed that for every 100ms increase in load time of Amazon.comtheir sales decreased by 1% [29].Thus, the performance of a website needs to be monitored regularly as it is an integral part of a Web design workflow and quality assurance programme.
Although website performance testing is of great significance, yet there has not been any significant study which examined the performance and scalability of University websites of Punjab (India).The higher education sector in the state of Punjab is highly vibrant, fast growing and highly competitive.This can be confirmed from the fact that 10 new universities have started their venture during the last five years.

Compatibility
Performance Security Usability www.ijacsa.thesai.orgTherefore, it is highly important for the universities not only to improve their academics and administrative procedures, but also the websites which is a common interface with their end users.Thus, the present study is the outcome of this research gap.

II. RESEARCH OBJECTIVES
The objectives of the study are as hereunder:  To evaluate the scalability behavior of University websites by measuring how the average throughput and hits per second will increase with an increase in user load.
 To examine the performance of University websites by measuring average response time and the amount of data processed (throughput) for the same user load.

III. RESEARCH METHODLOGY
The study focuses on certain selected websites of universities located in Punjab (India).Presently, there are twenty-five universities in Punjab which include central university (01), deemed universities (2), private universities (13), and state universities (09) (Appendix A).Therefore, stratified random sampling technique was applied to identify the University websites for the purpose of analysis.The university websites selected for this study are listed in Table 1.University websites are analyzed through LoadRunner software, a tool for performance testing.It largely suits this research because the measurement of website performance parameters is, generally, beyond the scope of other techniques such as heuristic evaluation, user evaluation, etc.Also, the evaluations exercised by human users are, usually, based on qualitative criteria which can be prone to error.For performance testing, throughput and response time should be absolutely measured in a concrete and verifiable manner for a set number of users.The framework of evaluation procedure has been shown in Figure 2. Planning the test was the first stage of experimental framework.After selecting the universities, the next step was to identify the web pages or links maximum explored by the end users.For this, a survey was conducted among the students who had recently taken admission in first year of the University programme.Their parents were also included in the survey.An enquiry was made from them to know which links were explored by them the most on the University websites at the time of seeking admission to various courses; and what sort of information they sought or expected under these links.The nine links identified for the purpose were "About Us", "Admissions", "Fee Structure", "Ph.D. / Research & Consultancy Cell", "Training & Placement Cell", "Hostel", "Downloads", "Contact Us" and "Scholarships / Fellowships / Financial aid".On the basis of these inputs, an attempt was made to find which pages/links of the University websites are suitable in the creation of script for performance testing.Uniform Resource Locator (URL) addresses for the identified pages/links of all the universities are listed in Table 2.
Further, scripts were created in Virtual User Generator (VuGen), a component of HP LoadRunner software.HP LoadRunner has 3 components, namely, Virtual User Generator (VuGen) which is used for script creation; Controller is used for executing the performance tests; and Analysis is used for analyzing the performance test results.The protocol used for script creation was Web (HTTP/HTML).The critical flows (Table 2) were first recorded for each university in a separate script file.The scripts were then optimized through parameterization, content checking, transaction naming and custom coding.The user inputs were handled through parameterization.The expected and actual response of the server request was matched through content checking.The transaction was considered passed only when there was a matching between the expected and actual response, otherwise, it was considered failed.Every user request was encapsulated within a transaction name.LoadRunner identified each Analyzing Test Results

Planning the Test Creating Vuser Scripts
Creating the Scenario Running the Scenario www.ijacsa.thesai.orgtransaction through a name and reported its response time.Custom codes were added to determine the size of each visited web page.Scripting being a one-time activity, the same scripts were further reused for various test executions.Once the script was created for each university, scenarios were created in Controller.A scenario is a file that defines the scripts to execute, the number of virtual users executing those scripts, and the machine that will host (load generators) the virtual users.Virtual users are not real users.Each virtual user works according to its script taken up for execution.For each University, scenario was created for 15, 30 and 45 users load test.Each scenario had a ramp-up phase (starting Vusers) during which the users were added gradually to the application till its predetermined limit was reached.After ramp-up, the script was executed for a fixed time span.During this period, users kept on doing their activities continuously.Later, during www.ijacsa.thesai.org the ramp-down phase, users started leaving the application gradually.The properties set for these actions in a particular scenario are depicted as below in Table 3.All the scenarios were executed after their creation; and the test results were taken up for analysis under the third component of HP LoadRunner software, namely, Analysis

A. Scalability comparison of selected University websites
Scalability is a performance testing parameter that helps us to identify if an application (website) is capable of scaling with increase in number of concurrent users.Various attributes of an application can be used to gauge its scalability like response time, transactions per second, etc.However, the best possible ways to determine the scalability of any application are throughput and hits per second.Throughput is a performance testing metric which helps us to determine the amount of data (in bytes) an application is able to process.Typically, it is expressed as bytes/sec.Hits per second is a performance testing metric which helps us to determine the number of hits made on the Web server by users during each second of the load test [30].The scaling of various websites with respect to throughput parameter with varying user loads is expressed through Table 4 and Figures 3 to 6. Table 4, column III determines the average throughput received; while column IV highlights the total data received from the web server during the entire test duration.Average throughput is calculated as total throughput divided by total test duration (in sec).Column V determines by what percentage the user load has increased as compared to previous runs.When user load is raised from 15 to 30, it shows 100% increase; and the increase is 50% when raised from 30 to 45. Column VI displays the percentage increase in average www.ijacsa.thesai.orgthroughput from the previous run.NA (not applicable) indicates that the current test has been taken as a baseline test.
It can be observed from the throughput graphs given below that for all the universities across various load tests, throughput has increased with the ramping up of users in ramp-up phase.The ramp-up phase period is constituted of initial 3 minutes 30 seconds, 7 minutes 15 seconds, and 11 minutes for the 15, 30 and 45 users" load test respectively.It was noticed during the ramp-down phase that the throughput decreased as the users came out of the application gradually; and finally, reached to zero.The ramp-down phase start time was 1 hour 3 minutes 30 seconds, 1 hour 7 minutes 15 seconds, and 1 hour 11 minutes for the 15, 30 and 45 users" load test respectively.
Figure 3 clearly depicts that with the increase in user load from 15 to 30 (100% rise) in the case of Central University of Punjab the throughput also increased by 79.344% (Table 4), though not proportionally.Later on, the user load was further increased from 30 to 45 users, but the throughput managed to increase by 4.197% only (Table 4).For this test, the throughput was neither stable, nor it showed any increase with the increase in user load (during a span of 12 to 35 minutes).Also, the throughput managed to become stable, but it does not scale up proportionally (during a span of 36 to 72 minutes).This indicates that the site under investigation faces the problem of scalability.
The curves shown in Fig. 4 represent the throughput achieved by Thapar University (a deemed University) for all the three tests.It shows that once the users" load stabilized, the throughput also got stabilized with ignorable fluctuations during a span of 15 to 65 minutes.With an increase in user load, the throughput also increased proportionally.With increase in user load from 15 to 30 (i.e., 100% rise) and 30 to 45 (i.e., 50% rise), the throughput increased by 93.005% and 48.511% respectively (Table 4), which evidently indicates that the website is scalable.Fig. 5 highlights the throughput achieved by Chandigarh University (a private University) for all the three tests.It shows that once the users" load got stabilized, the throughput also got stabilized with least fluctuation for 15 and 30 users test during a span of 15 to 65 minutes which was not observed for 45 users test.In the case of this University, when user load was increased from 15 to 30 (i.e., 100% rise), the throughput also increased proportionately, i.e., 91.406%.However, when the test was conducted for 45 users the throughput was highly unstable; and it showed an increase of only 42.572% (Table 4).It indicates that the site under investigation faces the problem of scalability.
The throughput achieved by Guru Angad Dev Veterinary and Animal Sciences University (a state government University) under all the three tests is highlighted in Figure 6.The results showed that there was 87.339% increase in the throughput after raising the user load from 15 to 30 (Table 4).Thus, the proportional increase in throughput was considerably less than that of user load.Also, when the user load was increased from 30 to 45, the throughput with the increase of 43.186% was stable, but not proportional.Thus, the results indicate the University site under study needs to focus on certain scalability issues.
Hits per second determine the number of hits made on the web server by the users during the load test.Table 5 carries the data showing hits per second for all the University websites under study.The columns III and IV represent the number of hits per second and total hits received respectively by the web server during the entire test duration.It was observed that 100% increase in user load resulted into a maximum increase of 92.693% hits per second in the case of Thapar University (a deemed University), while it was the least, i.e., 79.016% in Central University of Punjab.However, when the user load was increased by 50%, the hits per second showed a maximum increase of 48.893% in the case of Thapar University, while it was as low as 1.367% in Central University of Punjab.Similar results were found after a comparison was made between the data provided in column VI of Tables 4 and 5.   On the basis of results pertaining to throughput (Table 4) and hits per second (Table 5), it can be said that of all the four universities under study, the websites of Thapar University and Central University of Punjab are the most and least scalable respectively.

B. Performance comparison of selected university websites
The performance of University websites under study has been evaluated on the basis of time taken (response time) and data processed (throughput) for the same user load.
Transaction response time is the basic performance testing metric which helps us to determine the time taken by an application to process the user request.By definition, it is the time duration between users sending the request and receiving its complete response from the server, expressed in seconds.Tables 6 to 9 contain the data showing the transaction response time for all the universities across various user loads.Here, column I demonstrates all the links of the website that a virtual user hits in its iterations while running the script.Column II exhibits the user load for which the test was executed.Columns III to V display the minimum, average and maximum response time taken by that transaction for the respective user load.Column VI determines the number of times an expected response was received for that transaction.Similarly, column VII explains the number of times an expected response failed in its transaction.Most of the transactions failed due to step download timeout error.The error of step download timeout occurs when the user does not receive the whole response in a stipulated time as mentioned in load runner scripts.In this study, tests have a fixed value of 300 seconds.Column VIII demonstrates the page size of the respective link.
The data shown in Table 6 clearly reflects that the web page "Fee Structure" has taken the highest "average transaction response time" for all test runs, i.e., 10.754, 30.856 and 71.281 seconds for 15, 30 and 45 users load test respectively.The page size is of 4.337 MB which makes it the heaviest of this website.In the "Fee structure" page, a PDF file having all the fee details of various courses offered by the university has been downloaded from the server.The web page taking the least "average transaction response time" is not consistent across the different load tests, i.e., 1.423 secs by "Contact Us" page (15 users test), 4.176 secs by "Downloads" page (30 users test) and 10.247 secs by "Training & Placement Cell" page (45 users test).The page having the smallest size of 0.254 MB is "Downloads" for this university.It has been observed that the failure rate for the transactions is as high as 11.7% of the passed transactions for 45 users test run.
A glance at Table 7 provides that the web page "Hostel" has taken the highest "average transaction response time" for all the test runs, i.e., 12.132, 13.609 and 15.856 seconds for 15, 30 and 45 users load test respectively.The page size is of 6.988 MB which makes it the heaviest of this website.Hence, the average response time taken for a transaction is also the highest.However, the web page "Fee Structure" has shown the least "average transaction response time".The average transaction response time recorded after a load test of 15, 30 and 45 users is 1.618, 1.806 and 2.576 seconds respectively.Further, this page is of the smallest size, i.e., 0.326 MB.
As per the results shown in Table 8, in the case of Chandigarh University, the web page "Training & Placement Cell" is the heaviest of this website with a size of 1.241 MB.Thus, the "average transaction response time" is also the highest, i.e., 10.688, 11.075 and 27.731 seconds for all the test runs conducted under a load test of 15, 30 and 45 users respectively.However, the web page "Download" is the smallest with a size of 0.459 MB.Thus, the "average transaction response time" is also the minimum, i.e., 1.991, 2.041 and 7.884 seconds for a similar load test of 15, 30 and 45 users respectively.www.ijacsa.thesai.orgIt is clear from Table 9 that in the case of Guru Angad Dev Veterinary and Animal Sciences University, the web page "Fee Structure" is the heaviest of this website with a size of 1.461 MB.Thus, the "average transaction response time" is also the highest for all the test runs, i.e., 6.440, 7.440 and 17.403 seconds for 15, 30 and 45 users load test respectively.Of all the web pages, the best "average transaction response time" for the web page "Training & Placement Cell" is 3.203, 3.716 and 8.039 seconds for 15, 30 and 45 users load test respectively.Further, the web page "Hostel" is of the smallest size, i.e., 0.492 MB. www.ijacsa.thesai.orgWhile analyzing the results of all the University websites under study, it has been found that more the size of web page, higher would be the response time and vice versa.The composition of each web page, i.e., the type of files it integrates has also been analyzed.However, the composition of only those web pages having a size of over one MB in the case of all the University websites is shown in Table 10.http://www.cup.ac.in/campus_life.php),images for the web page "Hostel" constitute 97.026% of the overall page size which lead to increase the transaction response time.Further, the website of Deemed University has mainly used PNG images which are heavier in size as compared to other types of images such as JPG and GIF used by other universities for their websites.However, in the case of Central University website, no PNG image has been used.Table 11 demonstrates the data explaining average response time with respect to all user load tests undertaken for all the four selected University websites.The average response time has been calculated by taking mean of "average transaction response times".The results, also presented graphically in Figure 7, explain the performance of all the University websites more clearly and effectively.Finally, the overall average response time results calculated for each university across all load tests are presented at the bottom of this table .A good website is always capable of having a stable response time and maximum throughput under different user load tests.Here, Central University"s website has the highest overall average response time of 13.324 seconds across all user load tests (Table 11).As is evident from Fig. 7, there is a great variation in the average response time of selected University websites under different user load tests.This variation is the highest in the case of Central University"s website.Thus, it can be said that the website of this University is the least performing among the four University websites.Further, although the overall average response time for the State University website is 6.098 seconds which is less than that of Deemed University (6.755 seconds), yet the throughput is comparatively more than double in the case of Deemed University under all the load tests (see Table 11).Furthermore, Deemed University"s website has shown the most stable average response time under different user loads.Thus, it can be said that Deemed University"s website is the best performing site of all the four selected University websites.

V. CONCLUSIONS
The main conclusions drawn from this research work are as follows:  From the scalability point of view, the website of Deemed University under study has been found to be the best as an increase in user load has resulted into an increase in the throughput and hits per second both in tandem and proportionally.Whereas the Central University"s website is the least scalable among the sites studied in this research as there has not been a proportional increase in throughput and hits per second against the user load.
 As far as the performance of all the University websites under study is concerned, in terms of overall average response time along with the amount of data processed and stability of average response time relative to varying user load tests, the websites of Deemed and Central universities have been found to be the most and least performing sites respectively.
All the considered Universities have their own underlying hardware, server configurations and technology architecture for their websites.Therefore their performance testing results are bound to be different despite the fact that all the other test parameters such as test users, test scenarios, internet speed etc. are the same for all of these Universities.However each of the universities has a common goal of serving to all the stakeholders" viz. the existing & aspirant students, the faculty members and providing them a good user experience.Hence, this study is an attempt to highlight which university website is designed for providing better performance and scalability www.ijacsa.thesai.org

D. Scope for further research
 The present study is confined to the websites of selected universities located in the state of Punjab (India) only.However, to corroborate and extend the outcomes of this study, an extensive research is required to be carried out with a larger sample of universities covering diverse regions of the country and world as well.
 The dynamic nature of websites suggests that a longitudinal approach can be followed to examine the changes in performance level of university websites.

Fig. 1 .
Fig. 1.Non-functional requirements of a website

TABLE . I
. UNIVERSITY WEBSITES CONSIDERED FOR THE PURPOSE OF ANALYSIS http://www.gadvasu.in/

TABLE .
IV. THROUGHPUT STATISTICS REPRESENTING THE SCALABILITY BEHAVIOR OF UNIVERSITY WEBSITES

TABLE .
VI. TRANSACTION RESPONSE TIME STATISTICS OF CENTRAL UNIVERSITY OF PUNJAB FOR ALL LOAD TESTS

TABLE .
VII. TRANSACTION RESPONSE TIME STATISTICS OF THAPAR UNIVERSITY FOR ALL LOAD TESTS

TABLE .
VIII.TRANSACTION RESPONSE TIME STATISTICS OF CHANDIGARH UNIVERSITY FOR ALL LOAD TESTS

TABLE .
IX. TRANSACTION RESPONSE TIME STATISTICS OF GURU ANGAD DEV VETERINARY AND ANIMAL SCIENCES UNIVERSITY FOR ALL LOAD TESTS

TABLE . X
. CONTENT BREAKDOWN OF WEB PAGES (IN KB) OF SIZE AT LEAST ONE MB FOR ALL THE UNIVERSITY WEBSITES * Validation scripts, fonts, URLs, directory overheads, etc

Table 10
reveals that University sites under study have used various types of images like PNG, JPEG and GIF.Images constitute 97.420% of page size for the web page "Hostel" of