Performance Evaluation of Cloud Computing Resources

Cloud computing is an emerging information technology which is rapidly growing. However, measuring the performance of cloud based applications in real environments is a challenging task for research as well as business community. In this work, we focused on Infrastructure as a Service (IaaS) facility of cloud computing. We made a performance evaluation of two renowned public and private cloud platforms. Several performance metrics such as integer, floating Point, GFLOPS, read, random Read, write, random write, bandwidth, jitter and throughput were used to analyze the performance of cloud resources. The motive of this analysis is to help cloud providers to adjust their data center parameters under different working conditions as well as cloud customers to monitor their hired resources. We analyzed and compared the performance of OpenStack and Windows Azure platforms by considering resources like CPU, memory, disk and network in a real cloud setup. In order to evaluate each feature, we used related benchmarks, for example, Geekbench & LINPACK for CPU performance, RAMspeed & STREAM for memory performance, IOzone for disk performance and Iperf for network performance. Our experimental results showed that the performance of both clouds is almost same; however, OpenStack seems to be better option as compared to Windows Azur keeping in view its cost as well as network performance. Keywords—Cloud computing; OpenStack; windows azure


I. INTRODUCTION
Cloud Computing is an evolutionary technology which delivers computing software as well as hardware resources as a service over the internet.Cloud computers are interconnected and virtualized in a distributed and parallel system.The access to the infrastructure and computing resources such as compute, memory, storage, network, software, application and platform is permissible for any user for building applications.
The cloud services are mainly classified as Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS).IaaS provides infrastructural hardware, the most frequently used cloud, as a service to the users.The cloud infrastructure allows service providers to offer the infrastructural facility to the customers so that they can access the IaaS resources and enjoy the better service provision for smooth working of their applications without buying other resources.The clients just pay for the resources and services used on the basis of adapted Service Level Agreement (SLA) with a service provider [1].The cloud clients pay for these resources just like as they pay for utility services such as water, gas, and electricity as per need and use [2].Amzaon, Rackspace, Google, Eucalyptus, XEN, OpenNebula, Nimbus, Microsoft and OpenStack are major cloud computing resource/service providers.Monitoring of cloud resources is important both for cloud service providers and for cloud customers.Cloud providers monitor the efficiency and current status of assigned resources in order to handle future requests from customers.Monitoring helps cloud customers to investigate the resources assigned to them, and ensures that they get the demanded resources they are paying for.Further, it allows them to know when to demand for additional resources, when to surrender any underutilized resource, and what amount of numerous physical resources are suitable for different type of applications.
OpenStack is open source technology to provide elastic cloud operating systems.Windows Azure is public, private and hybrid cloud platform.
Moreover, diverse nature of different types of applications comes with different demands leading toward the need for different features in a platform naturally.Other aspects such as service models and prices are also taken into consideration.Performance evaluation of cloud services and resources is an important issue for cloud customers as well as for cloud providers.Cloud computing performance can be in terms of response time, throughput, reliability, security and availability.Monitoring of cloud services over internet based applications in a real cloud setup is much needed but difficult task.Research as well as business community is paying much attention to improve performance of IaaS resources.This work mainly focused on performance analysis of two well-known IaaS clouds platforms, i.e., OpenStack [3] and Windows Azure [4] and compared their performance by considering various cloud resources such as CPU, memory, disk and network.OpenStack is a popular and fast growing open source cloud computing for private, public and hybrid clouds while Windows Azure is one of the mostly used private clouds.Our results for comparison analysis are based on configuration made in a real time environment.

A. Objectives/Contribution
The major objective of this work is to evaluate and compare performance of various IaaS resources of two famous cloud platforms i.e., OpenStack & Windows Azure platform in a real cloud environment.The considered IaaS resources include CPU, memory, disk and network.www.ijacsa.thesai.org

B. Research Approach
The research process for this work comprised of the following steps:  Survey and literature review about performance evaluation of IaaS resources of various cloud platforms. The comparative results were presented in the form of graphs by using MATLAB.

C. Organization of the Paper
The rest of the paper is organized as follows: Section II provides detailed background of the problem under study and related works in this regard.In Section III, we provide the experimental set up.Section IV presents benchmarking tools used for performance evaluation and comparison.In Section V, results obtained through experiments and related observations are provided.Finally, Section VI concludes the work and hints towards further research issues and challenges.

II. BACKGROUND AND RELATED WORK ABOUT PERFORMNACE OF CLOUDS
Performance of cloud resources is very crucial for cclients as well as for service providers.Consequently, such performance evaluation is essential from the point of view of cloud service providers and clients.

A. Background
Cloud service providers are interested in evaluating the performance of different infrastructure based cloud resources such as compute, storage, network and virtual machines.A single individual component is unable to provide complete performance report of a cloud.Infrastructure Response Time (IRT) is a new approach to get more accurate performance of virtualized cloud environment.IRT is explained as the time it takes for an application to put a request (I/O) over virtual environment and get back its response.This request can be a normal data transfer between two Virtual Machines or complex one like transaction of database and storage into a storage array.
The most common idea for achieving best performance is by increasing the resources.However, customers are forced to bear higher costs when they opt for purchasing their own resources which is not a better solution.Therefore, cloud customers use needed resources from cloud service providers and pay as per use without investing heavy amount in infrastructure setup.Therefore, cloud consumer is interested in performance of applications hosted on the cloud platform which greatly depends upon performance of IaaS resources.Application Response Time (ART) is an important metric in application performance management which is calculated as time taken by the application to respond to other users' requests.Thus, cloud provider is more interested to have a complete view of health of whole cloud for better service provisioning [5].
In a cloud environment, the tasks can be classified as computation intensive and communication intensive tasks.RAM and CPU cores are important cloud resources consumed by computation-intensive tasks.In a cloud application, a communication-intensive task normally produces large number of network transactions between cloud user devices and cloud www.ijacsa.thesai.orgsystems.Therefore, network monitoring is critical to analyze the estimation of network performance monitoring (NPM) which is analyzed through different network monitoring techniques.Active monitoring [6,7], method of using SNMP agent, and passive monitoring [8] are normally three strategies for network monitoring.
Computation-intensive tasks, such tasks can be divided into CPU-intensive tasks and memory-intensive tasks [9].For energy efficiency of cloud systems, Lefebvre and Orgerie [10] analyzed multicore platform with focus on CPU cores only.This work provided an assessment of the energy consumption during relocation of VMs only with computation-intensive cloud applications.Liu et al. [11] provided a new cloud infrastructure which can dynamically associate Virtual Machines (VMs) based on CPU utilization of servers to detect idle physical servers.Idle physical servers can be turned off to protect energy.However, these energy protecting strategies do not take into account the work load in cloud systems and hence are very coarse-grained.
In communication intensive tasks, quantity of user's requests and related data size can have huge impact on the system performance [12].Many web services are installed inside servers (weather physical or virtual).If webhosting server serves the peaks, then it is most likely over-provisioning when the demand is high.In cloud computing, the performance analysis of network resource is necessary for best use of resources as in traditional computing.Cloud infrastructure consists of different component like server virtual machine, network interface and users are required to select component according to need.A well-known communication intensive application for cloud computing is Community Atmospheric Model (CAM) [13], a massive parallel application used in worldwide weather prediction.

B. Related Work about Performance
The objective of performance evaluation of cloud computing is to investigate and compare the performance attributes of the system [14].Amazon Elastic Cloud (EC2), Microsoft Windows Azure, GoogleApps Engine, Sales-force and Drop-box are well known commercial cloud service providers.Commercial success of any cloud computing platform depends upon its ability to deliver guaranteed Quality of Services (QoS) [15].Performance can be evaluated through measurements, simulation and/or modeling [16].
Zhang et al. [17] performed an evaluation of four commercial cloud platforms and drew a conclusion that prevailing platforms provide different types of services which are offered at various levels of abstraction.Therefore, endusers should select more than one cloud platform keeping in view their requirements to satisfy business needs.Buyya et al. [18] provided an evaluation of some cloud platforms considering market oriented cloud computing.The study focused on the need for advent of techniques for business cloud management on the basis of risk management and customer's requirements.
Hofer et al. [19] provided classification by considering features of various cloud systems.This work considered various characteristics which include service model, license type , cost model, supported languages and operating systems, virtualization mechanism and development tools.
Rimal et al. [20] made a classification on the basis of features and used it for the purpose of comparison of system offered by providers.The attributes of providers considered for comparison were architecture, interoperability and security, virtualization technique; services provided mechanisms for load balancing, and support for software and programming languages.
Ostermann et al. [21] analyzed the performance of EC2 utilizing micro-benchmarks like LMbench, Bonnie, CacheBench, HPC Challenge (HPCC) and kernels.They concluded that achieved performance of virtualized resources from public clouds is lower when compared with the theoretical performance limits, particularly for compute and network intensive applications.They compared the observed virtualized performance of a private cloud with the nonvirtualized performance.They used metrics such as CPU, I/O, and memory hierarchy on the Single Instance benchmarks to evaluate performance.Some studies performed qquantitative comparisons among different providers.For example, authors of [22] proposed a framework namely Cloud Comp which provides performance comparison of various providers [22].
Zheng et al. [23] carried out comparison of four commercial cloud providers, namely Amazon EC2, Google AppEngine, Windows Azure and Rackspace cloud servers on the basis of a few components of computing, network, database and storage.Different issues affecting startup time of cloud VMs across Amazon EC2, Google AppsEngine, Windows Azure and Rackspace are studied in [24].
Li et al. [25] carried out a performance and cost comparison between four major public clouds.The clouds are compared on the common functionality set, which incorporates elastic computing, intra-cloud network, persistent storage and Widerange network.
Table I shows different studies reported on the topic.It classifies these studies as qualitative, taxonomy based and qualitative comparisons.
The latest research of cost effective cloud computing mostly analyzes the cloud service provider cost for offering cloud solution, for example, the authors in [26,27] explored cloud energy consumption and cost efficiency and discussed about different issues and challenges.According to [3], with on-demand resource provisioning and utility based costing; cloud service provider can really expand resource utilization and reduce their operational cost.www.ijacsa.thesai.org

C. Issues with Existing Approaches
Despite the fact that research community is focusing a lot on performance of cloud resources, however, in-depth knowledge about performance in a real set up is still lacking in single document.Most of the works do not provide true picture for performance analysis as those works are carried out using different types of simulators.Therefore, there is a need of having results and findings in a real set up instead of the same in a simulation environment.

D. Our Approach
While taking into consideration the related work about cloud performance analysis, we observed that (i) there is no comparative study about performance evaluation of OpenStack and Windows Azure, and (ii) no in depth performance analysis related to selected cloud resources is found for analysis purpose.
To the best of our knowledge, this work is first of its in kind which provides detailed performance analysis of various critical issues including the price and benefits with applications in a cloud.We developed a real time environment to analyze the cloud performance like compute, network, and storage and disk workload.Here is some description which shows our work is unique.
First, we explore the performance of cloud computing and setup experiment by using OpenStack and Windows Azure.www.ijacsa.thesai.orgThen, we verify our results by repeating experiments multiple times.The results of experiments can be helpful for cloud service providers in managing their services effectively in order to meet consumer's requirement.Moreover, our findings can provide useful insights to cloud consumers to manage idle resources more efficiently for smooth running of applications.The next section describes the experimental set up in detail.

III. EXPERIMENTAL SETUP
The experimental settings consist of following specification, we use two servers ProLiant DL380 G7 for cloud set up separately (refer to Table II for specifications of servers) and minimum requirement for client machine is Pentium 4 Machine with 512 MB RAM, network card,80GB Hard drive.
We installed OpenStack under Ubuntu on standalone machine and Windows Azure on a separate machine.Subsequent sub-sections describe the considered cloud platforms i.e.OpenStack and WinAzur.

A. OpenStack
In our experiments, we used the Juno version of OpenStack which is an open source software.Juno is the tenth release for structure of public, private, and hybrid clouds.The OpenStack contains lot of features to support application development.Many organizations like Rackspace, NASA, Citrix, Dell, Cisco, Canonical and many more participants of worldwide software community support OpenStack.
The OpenStack cloud allows service providers to propose computing resources by catering huge networks of VMs.To make an effective image provisioning, OpenStack stores images on the computes nodes, eliminating the needs of shifting the VM image on the network every time it is requested.
The hardware used for OpenStack implementation scenario is a HP ProLiant DL380 G7 series.This server has intel Xeon CPU E5620 dual processors and had a specific role that require huge processing capability.Server has the following specifications: Latest version of stable release of OpenStack from OpenStack repositories was downloaded and only one machine with above specification was used for our experimental.To assist flexible services, we used different OpenStack components for installation of cloud.The cloud controller and nova compute are installed on same server.Some additional software like MySQL data base server, RabbitMQ messaging queuing, Apache webserver and KVM are also installed on this machine.The Service Management API is an application of the Windows Azure Service Management that provides a reliable customer API that talk to the WAP fabric underneath.If the services in the WAP are not deployed similar as in Windows Azure, the access to the Service Management API is consistent.The portal in the WAP looks same as the portal in Windows Azure.However, you are able to adjust the portal as you like as a service provider or if you don't like the portal at all, you can www.ijacsa.thesai.orgbuild customized portal and use the required Management Service APIs.The provider portal allows the WAP administrator to design the WAP infrastructure to define plans that is assigned to an end user [34].
Windows instances are accessed through remote desktop to clients via public or private IP address, the password is encrypted and portal can be accessed through secure browser by using https.The server with same specifications as for OpenStak was used.
In windows azure virtual machines can be create, delete or re-create on demand and these virtual machines can be access just like physical server.To create virtual machine virtual hard disks (.vhdx files) are used.This supports image and disk categories of virtual hard disk.To create a virtual machine from image the following procedure can be used:  Use Azure portal to make a virtual machine from image.
 Build and store a. vhdx file that comprises an image to WinAzure portal which is used to make a virtual machine.
Windows Azure gives particular arrangements of central processing unit (CPU) cores and memory for IaaS virtual machines.To create a virtual machine, select a particular size available from list however the size can be altered after deployment.The extreme size of a working disk can be 127 GB.When an OS disk is generated in Windows Azure three replicas of the disk are made for high availability.
Virtual machine storage: In Windows Azure a virtual machine is generated from an image or a disk.These virtual machines run one or various data disks installed operating system.All images are generated from virtual hard disk (. vhdx) which are stored in the form of blobs in in Windows Azure storage account.

Virtual machine network:
The virtual machine systems are committed to virtual machine LAN activity.A virtual machine network can be two or more 1 GbE networks have been made through NIC Teaming.

IV. BENCHMARKS AND IMPLEMENTATION
The best approach to study the performance of specific system is to run the actual workload on the hardware platform and analyze its results.However, in certain scenarios, this method is not feasible.In such situations, analysts prefer to use typical benchmark results.Benchmarking is the basic technique for determining the performance of a real machine.Benchmarking mentions to running a set of typical programs on different workstations, networks and evaluating the results.Benchmark outcomes are used to analyze the performance of a particular system on agreed workload.Normally, comparative study of products depends upon benchmarks.They are used similar to monitoring and analysis software.System vendors, developers, and customers run benchmarks to identify performance problems of new systems.
In this research work, our concern was time and rate efficient evaluation of a particular cloud platform, as organizations and individuals usually do not want to spend too much time and money on such concerns.We have chosen those benchmarks and platforms that strengthen this viewpoint.
Our focus was on actual hardware components for such as CPU, memory, disk, and network etc. Cloud providers are required to provide clear information such metrics.

A. Cloud Plateform Selection
OpenStack cloud was our first choice due to the fact that it is mostly used in both industry and academia and it offers wide variety of service.The second choice was Microsoft windows Azure cloud, a new competitor in the IaaS platform, Microsoft traditional grip of the enterprise marketplace makes them a rational selection for various industries.In this research all VMs use the same version of Windows Server 2012 R2 64-bit operating system to reduce uncertainty and all instances on the particular platforms for the Windows instances.Furthermore, the chosen instance sizes and types are provided in Table III.

B. Benchmarks Selection for Performance Evaluation
In Cloud computing different applications have different hardware requirements.Our focus was to select those benchmarks that can evaluate cloud system resources like CPU, memory performance, storage and network.These parameters are appropriate to measure system's performance and the set of chosen benchmarks are applicable for an extensive mainstream of applications.We selected freely available and commonly used benchmarks which offer transparency, availability, and efficiency.Authors of [35] discussed a distributed testing model.

CPU performance: CPU performance is determined by two renowned parameters i.e. MIPS (Million Instructions Per Second) and FLOPS (Floating Point Operations Per Second).
MIPS unit is tough to compare between CPU architectures, and workloads.CPU performance is discussed in [36].
Integer and Floating Point are computing intensive calculations.Integer data contains complete numbers, text and other similar items.But Floating Point Unit (FPU) procedures are further complex than integer.Examples of applications building a full usage of FPU are worksheets, graphical theory applications, games, and subsequently.Therefore, to achieve best performance processors required to perform the process of integer and FPU as fast as likely.
FLOPS are normally used to analyze the performance of a processor.A FLOPS simply determines floating point calculations and not integer operations.Therefore, FLOPS can www.ijacsa.thesai.orgexactly calculate a processor floating point unit (FPU).In order to exactly measure the processing capabilities of a CPU, different types of tests needs to be run.For our analysis w.r.t.CPU performance, we used FLOPS, Integer and floating point.
For FLOPS, we selected LINPACK benchmark, a tool for performing numerical linear algebra.In this specific test we used LINPACK version developed by the Intel Corporation, which is open-source but with a closed source front-end called IntelBurnTest which is commonly used in overclocking circles.
To determine the performance of integer and floating point unit, Geekbench benchmark tool was chosen which is designed to work on different platform and it is used on Linux, Windows and Solaris.

Memory Performance:
The metrics used for determining system performance in terms of memory are pretty simpler than CPU metrics as the memory system is debatably a simpler component than the CPU.RAMspeed and STREAM are renowned and most commonly used tools for benchmarking cloud performance w.r.t.memory.

Disk performance:
The metrics used for calculating disk bandwidth are less complex and we are more anxious how rapidly the disk can read and write blocks of data to and from the hard drive.IOzone is a filesystem benchmark tool that is useful for performing an extensive file system analysis of a computer system.It allows number of options to be set as well as it contains a throughput choice in which you are required to mention limited parameters.We used IOzone method for quick evaluation of system's performance.

Network performance:
The performance of network is decreased as a result of large VM working on the same machines.A large amount of data is communicated when a large number of machines use the network at the same time.A cloud platform needs fast network equipment and links in order to provide best performance.Bandwidth, latency, packet loss and jitter are interesting parameters concerning the network of a cloud provider.In this research we tried to evaluate the internal bandwidth of cloud providers by setting up instances within the same area and running network experiments between them.In order to determine the performance, we used Ipref tool which works as clinet-server model.Iperf, software for network analysis, can generate TCP and UDP data streams and calculate throughput of the network.It permits the user to set different parameters for analysis the network but more significantly one can assess bandwidth, jitter and throughput easily by modifying just a few of the typical configuration useful for quick evaluation.

V. RESULTS AND DISCUSSION
This section provides performance results of two clouds by using benchmarks described earlier.During experiments, it was ensured that no customer applications were in operation concurrently with the benchmarking applications on the experimental VMs.The only running processes were those which were essential for operating system to work itself at boot time.The subsequent sub-sections provide results with respect to performance of CPU, memory, disk and network.

A. CPU Performnace
To measure the performances of the CPU, we used Geekbench and LINPACK tools.We used one Windows instance initially at a time to perform the performance tests on the cloud.The launched instances were running at 100% of CPU usage.The workload of Geekbench is divided into Integer Performance and Floating point performance.These workload measure the performance by performing intensive tasks related to processor that use heavily integer and floating-point instructions.High score lead to better overall performance.Floating point measures are important, for example in video games.
The benchmark conducted using the program IntelBurnTest with a matrix size of the Standard 512MB on m1.tiny and MicroVM instances while rest of the instances with standard 1024 MB, this test run 10 times on every machine being the default configuration.The complete outcomes of the test are shown in Tables IV, V and Fig. 1 to 3. The data using LINPACK benchmark shows Windows Azure gradually outperforms OpenStack (Fig. 1  The results shown in Fig. 1 are taken by using LINPACK tool while that shown in Fig. 2 and 3 are obtained by using Geekbench tool.Windows Azure performance in integer is some better than OpenStack.In integer all instances of Windows Azure outperform to all instance of OpenStack in single core as well as multicore integer.This data shows two opinions of interest.The primary is that OpenStack gradually outperforms Windows Azure while Azure performance in floating Point is a bit better than OpenStack.In floating point, CPU performance for single core instance of Windows Azure is better, while in multicore OpenStack tiny and large instances perform better than that of windows Azure micro and high instances.Moreover, windows Azure in mini and medium instances outperforms in small and medium instances of OpenStack.Why this is so, is difficult to say without more information about the particular datacenters.Moreover, both platforms scale in a way constant with the Cloud provider's assurance in terms of CPU capacity.OpenStack for instance state that their tiny, small, medium and large instances each provide 1, 2 and 4 cores.In other way the CPU performance is almost twice every time the instance type is increased by a step.Microsoft states similar kind of facts about their platform.In summary, though both perform almost at the same level, but OpenStack, being feely available, seems to be a better option.

B. Memory Performance
We used RAMspeed and STREAM benchmark for memory performance.The performance of memory measured RAMspeed by using integer and floating point numbers.The purpose for using Integer and floating point is that it has to provide a lower limit on the memory performance as integer procedures are usually faster.Moreover, most of the mainstream applications use both integer and floating point numbers in which the floating point memory procedures be the bottleneck as far as memory goes.
The memory performance benchmark based on a bandwidth assessment as this is differentiates among System memories categories.Stream is memory benchmark that efforts to make best use of memory bandwidth, it is severely load the memory without using appropriate pressure on the CPU.
The command to measure memory performance using Integer with RAMspeed is >ramspeed-win32 -b 3 -l 50 while the command to measure memory performance using Floating Point RAMspeed is >ramspeed-win32 -b 6 -l 50.The parameter -b is for bandwidth, 3 for Integer memory and 6 for Float memory, -l for length of benchmark we use 50 test and take average of them.The outcomes obtained by running RAMspeed and STREAM for both clouds are provided in Tables VI, VII and Fig. 4 and 5.
In RAMspeed integer performance in windows Azure's micro and medium instances performance is better than OpenStack's tiny and medium instances but OpenStack small and high instance performance is better than windows Azure mini and high instance (Fig. 4).
In STREAM benchmark windows Azure's all instance performance is better than OpenStack all instances (Fig. 5).Windows Azure memory performance is much better than in OpenStack in terms of raw performance.Moreover, OpenStack shows a much less diverse performance than Windows Azure.In any case which metric worth most, consistent, or high performance based on application and condition.If experiments are run for a certain application and the outcomes show high difference but also that the real performance did not below the minimum definite point for the application to function correctly, then difference in the situation is not a problem.Difference only develops a problem if it differs with goes below that definite point.You need to choose in the event that lowest value that has most prominent impact on your application and select from subsequently.Finally, these values confirm that there is a lack of performance segregation in the cloud today.

C. Disk Performnace
Disk performance test is conducted on machines installed Windows operating system, the NTFS files system is used for benchmarking on the local drives.The theme of these tests is to calculate the time it proceeds and access data on the disk.We are interested in the entire sequence from main memory to disk and the other way from disk to main memory.In simple words we required the throughput speed.The software needs to know how many threads to use for the throughput test which was provided to it by initial testing the number of threads the CPU supports.The outcome obtained by using IOzone in throughput approach with some custom configuration indicating that we are concerned in sequential and random read and write speeds.
The 0=write/rewrite, 1= read/re-read, 2= random-read/write where -t flag indicate how many threads to use for the throughput test.We use 2 processes for throughput test conducted with each process writes a 512 Kbyte file in 4 Kbyte records and we take maximum throughput of per process in this benchmark.
The both provider's information similar variance and the typical performance on both platforms are very similar as given in Tables VIII, IX and Fig. 6 and 7.
Read, Random read, Write and Random write of OpenStack Tiny, medium and large instance is better than Windows Azure Macro, medium and high instance while Read, Random read, Write and Random write performance of Windows Azure mini instance is better than OpenStack small instance.This data provides several facts received from IOzone test.The benchmark provides extremely high speeds information.We notice few causes about disk performance.In virtual machine we have not control over several caches.It might be fine that caching inside the disks or other buffers change these figures.If this happening, then there is a needs to run the throughput experiment once again using a file size that is greater than the amount of free memory.That will effect on the time and these tests may take more than reasonable time.
The primary hardware consumes RAM drive as a buffer to the local hard drive which can make common sense if the actual hard drive is not located jointly with the other hardware holding the application.This is the reality that memory is very costly than hard disk.But alternatively if the primary hardware reports free memory space it uses as disk cache between other things.The random operations are approximately similarly fast as the sequential operations.
The change among the same tasks on different instance hosted by the same cloud provider is vast.We can simply describe it by conflict of resources among tenants unspecified that disk operations are particularly liable to suffer from.Finally, we observe that the systems design appears to be relatively similar between OpenStack and Windows Azure.

D. Network Performnace
In network performance test, we take single instances on each platform.UDP was used in order to gather information on packet loss, TCP was used to measure bandwidth and jitter.For network performance comparison of both clouds, we consider three network parameters, i.e. network bandwidth (it refers to the volume of information that can be transferred over a network in a certain amount of time, typically stated in bps), jitter (difference in the latency on a packet flow among two systems, when some packets take more time as compared to others to transfer), and throughput (the average rate of successful message delivery over a communication channel).The outcomes are shown in Tables X, XI and Fig. 8 to 10.The important fact is that OpenStack network points out greater bandwidth statistics.Microsoft Azure internal network underperforms than OpenStack, though not much low.We run Iperf Server on main machines while run Iperf client on virtual Machine in LAN.We observed that network performance of OpenStack is slightly better than WinAzure.Authors of [37] discussed network simulation resources.
To measure network bandwidth (Fig. 8) and jitter (Fig. 9) start Iperf on server in server mode with these parameters >iperf -s -u -P 0 -i 1 -p 5001 -f m and on client side run this command >iperf -c <Server ip address> -u -b 100m Here, -b selection is used to identify the bandwidth to use.Normally Iperf UDP usage 1Mbps we suggest usage complete offered bandwidth to get an idea.To measure network throughput (Fig. 10) Iperf on server in server mode with these parameters >iperf -s -P 0 -i 1 -p 5001 -f m on client side run this command >iperf -c <Server ip address> -P 20 Where P indicates maximum Parallel TCP connections (20 in this case) to measure throughput of LAN.
In output screen we get 20 different ports on the client is linked to the default 5001 port with the server.Every link has different transfer rate and finally we receive sum of throughput in Mbits/s.

E. Overall Summary of Results
Table XII presents overall picture of performance comparison of two considered cloud platforms w.r.t.CPU, memory, disk and network tested on related benchmarking applications.
Microsoft's Azure platform performs better in terms of CPU-integer Speed on single as well as multi core as compared to OpenStack, whereas in CPU-Floating point regrading single core, though Windows Azure is fractionally better as compared to OpenStack under macro, medium and high instances but its performance is comparatively less under small instance.Considering Floating Point speed with multicore, OpenStack performs a bit better.Overall, Windows Azure in term of CPU performance is marginally better than that of OpenStack when tested through Geekbench benchmark, whereas when checked with LINPACK benchmark, OpenStack's performance is better under tiny and small instances.In Table XII, WA is used for Windows Azur, OS for OpenSatck, SC for single core, MC is used for multicore, B for bandwidth and T is used for throughput.
In term of memory performance, Windows Azure performs better under all instances when tested with Stream benchmark.With RAMspeed benchmark, the integer and floating Point memory performance of windows Azure is more under tiny/macro and medium instances while under of small/mini and large/high instances, OpenStack turns out to be better.
With respect to disk Performance in terms of read, random read, write and random write operations, OpenStack is better than Windows Azure under all instances except small one.
OpenStack outperforms Windows Azure in network wise performance in terms of Bandwidth, jitter and throughput.
In summary, the performance of both clouds is almost same but OpenStack is somehow better option keeping in view its cost as well as network performance.www.ijacsa.thesai.org

VI. CONCLUSION
This work explored the performance of two real clouds, namely OpenStack and windows Azure by setting up real time configuration by research team.Performance is one of the major concerns for customers; therefore, we concentrated on observing the performance of major cloud resources like CPU, memory, disk and network using suitable benchmarks for each.These techniques are used to find the issues where the cloud's performance depends on installed software.This research is implemented in a cloud platform where environment load is usually challenging to identify and uncontrollable.The Performance is decreased due to virtual machines hosted on a same physical server running resource-intensive tasks.The performance is decreased in CPU and memory slightly due to intensive task assignment but the difference is more in disk and network intensive task assignment.
We evaluated cloud computing resources from customer's perspectives as well.We tested performance of OpenStack and Microsoft Windows Azure with the help of popular benchmarking tools which are open source and freely available over internet.In some instances, OpenStack performance is more than Windows Azure and vice versa.Overall the performance of Windows Azure and OpenStack cloud is almost same but at some point windows Azure performance is slightly better than OpenStack cloudbut the network performance of OpenStack is much better than Windows Azure.However, in our opinion, OpenStack, being freely available, requiring less infrastructure deployment, less power consumption and incurring less licensing cost is better option.
As a future work, elastic cloud resource management in real cloud environment is good choice for performance analysis of OpenStack and Windows Azure.Authors of [38] provided some hints in this respect.
Virtual machine and network isolation is also very hot issue in the performance of cloud, as a number of machines are using the same hard drive and network which may impact on performance of a cloud.When many users transfer data these resources are over utilized as a result of performance cloud.
The cloud evaluation is examined on single organization, more research and analysis of varying sizes organizations and in different industries is required to observe its overall application.Further, more research is required on how to find the most suitable cloud solution for a certain organization and system.To support this conclusion more research for the long term effects on organizations of implementation for cloud computing is required.
Results were obtained for parameters selected for each resource by using related benchmarking tools.Average results were calculated by repeating the process a number of times and repeated a number of times o drive rational conclusion.
 Benchmarking the cloud based on benchmarking of CPU performance, Memory performance, Disk performance and network performance for evaluation for determining the performance in real machines. Different benchmarking tools are available to test performance of cloud resources.Among the available benchmarking tools, we used Geekbench, LINPACK, RAMspeed, STREAM, IOzone and Iperf benchmark tools depending upon the nature of cloud resource being evaluated.o Disk Performance: Read, Random Read, Write, Random write o Network Performance: Bandwidth, Jitter and Throughput 

TABLE II
Ubuntu 14.04, Microsoft Virtual Machine Manager 2012 R2, Service Provider Foundation, System Center 2012 R2 Orchestrator, Windows Assessment and Deployment Kit 8.1, Microsoft SQL server 2012 R2, Hyper-V, Windows Azure pack, and necessary tools for monitoring, Windows server 2012 Standard/Data Center Edition B. WinAzur WinAzur is a collection of integrated services like computing, storage, database, mobile and networking.Microsoft virtualization platform Hyper-V helps optimize hardware resources by combining multiple client operation systems on a single server.This describes the method of setting up a Private Cloud using Microsoft technologies such as Hyper-V & System center.The System Center delivers the fabric management and monitoring that is required for the services.Once the installation and configuration are complete, it is possible to use Microsoft System Center Virtual Machine Manager 2012R2 for a private cloud to be built and managed.The Microsoft System Center allows cloud administration and management to deploy, monitor and report about it.The basic understanding about the roles and services in Windows 2012 R2, information of how to install SQL Server 2012 R2, The System Center Orchestrator 2012 R2 allows you to install the Service Provider Foundation and a practical knowledge System Center Virtual Machine Manager working.It is more necessary for a production environment, that careful installation of SQL server 2012R2 is required to ensure the proper working of System Center.The service and workload layer discloses the knowledge of the Windows Azure Pack to Windows Azure.The WAP is built in a way that allows you to offer more services into the Windows Azure Pack.

TABLE III
to 3).OpenStack tiny & small instance outperforms Windows azure but Windows Azure medium & high instance outperforms OpenStack.The performance of windows Azure is a bit better than OpenStack in large instance.

TABLE IV .
OPENSTACK CPU PERFORMANCE BY INSTANCE TYPE

TABLE VI .
OPENSTACK MEMORY PERFORMANCE BY INSTANCE TYPE

TABLE VIII .
OPENSTACK DISK PERFORMANCE BY INSTANCE TYPE

TABLE IX .
WINAZURE DISK PERFORMANCE BY INSTANCE TYPE

TABLE X .
OPENSTACK NETWORK PERFORMANCE BY INSTANCE TYPE

TABLE XII .
OVERALL SUMMARY OF RESULTS