A Features-based Comparative Study of the State-ofthe-Art Cloud Computing Simulators and Future Directions

Cloud computing has emerged during the last decade and turned out to be an essential component for today’s business. Therefore, many solutions are being proposed to optimize and secure the cloud computing environment. To test and validate the proposed solutions before deploying in real cloud infrastructure, a cloud computing simulator is the key requirement. There are several cloud computing simulators that have been used by research community for this purpose. In this paper, we have discussed modern cloud simulators and presented comprehensive comparison based on their features. Keywords—Cloud computing; simulation; cloud simulator; cloud performance analysis; simulator features


INTRODUCTION
Cloud introduced a new way of distributed computing by providing users with on-demand access to resources with minimal efforts and management overheads [1]- [3] .Further, it offers the cost-effective solution for utilizing computing resources and, at the same time, a good attractive business for the enterprises who own computing resources for rent.Cloud computing provides many features including adaptability, accessibility, cost reduction, flexibility, reliability, and scalability [4].According to National Institute of Standards and Technology (NIST), a cloud ensures the five important features that are "on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service" [5], [6].Besides this, the goal is to provide on-demand computing services to cloud consumers with the guarantee of reliability, availability and scalability.Cloud provides three basic service models termed as "SaaS (Software as a Service), PaaS (Platform as a Service) and IaaS (Infrastructure as a Service)" [7], [8] and is deployed in four ways known as "Public, Private, Community and Hybrid Clouds".Fig. 1 summarizes the cloud services and deployment models along with some application domains that may consume the cloud resources [9], [10].
The simulation of newly developed applications, algorithms, and architectural components is vital before its deployment in a real cloud environment.This is essential for analyzing the behavior of new algorithms and for further improvements.Therefore, cloud simulators play an important role and facilitate researchers for rapid evaluation of the efficiency, reliability, and functioning of proposed algorithms on the big heterogeneous cloud infrastructure [11], [12].Many cloud simulators are available out of which some are commercial and some are open source.These simulators emphasizes on the simulation of particular cloud computing component.For example, some simulators target the simulation of large-scale data centers, some of them simulates the cloud applications and analyze their behavior and some focuses on the workload distribution and fault tolerance analysis.Author in [13]- [15] presented the study and comparison of the cloud simulators but they do not impart the simulation emphasis of the presented simulators and the in-depth analysis of their features.This paper aims to review the state-of-the-art cloud simulators in order to explore their features, limitations and research opportunities.The rest of the paper is structured as follows.Section 2 followed by an introduction, discusses the issues related to performance analysis in cloud computing.Section 3 gives the overview of modern cloud simulators.Section 4 provides the discussion and the detailed analysis and feature based comparison of these simulators.Section 5 concludes the paper and future research directions.www.ijacsa.thesai.org

II. PERFORMANCE ANALYSIS AND ISSUES IN CLOUD COMPUTING
The performance analysis of a cloud computing system refers to evaluate it from different perspectives that include the assessment of the SLAs, resources distribution and its complex path generation for service delivery, and secure cloud storage.For evaluating the performance of a cloud computing system, one of the key performance requirements is to assure that it is a SLA-driven system performance [16].To determine the cloud performance, the major challenges in large-scale cloud computing, are massive scalability, dynamic configuration, and complexity of component interactions.The cloud systems are managed dynamically by SLAs that are settled an agreement between cloud consumer and service provider for service usage and policies.The performance metric for SLA is response time that deals with a response time of requested services to be delivered.
The cloud users connect with cloud and its service components through the internet.The cloud service components may be located at multiple hosts and the hosts may be in federated clouds.The delivery of services to users generates different types of executions and delivery paths that become complex and challenging to determine cloud behavior for performance analysts.An example for such challenge is to identify the service component that causes the main problem when system performance is not satisfying the expectations.
Cloud storage and evaluating its performance is very critical in terms of security, reliability, and availability because it contains the valuable business data.A 10-point execution assessment structure for existing distributed storage framework proposed in [17] is useful for the evaluation and simulation of distributed storage.

III. STATE-OF-THE-ART CLOUD SIMULATORS
Cloud computing evolved in last few years and facilitates with a new way of delivering on-demand computing services.The evolution roots of cloud computing are in Grid Computing and Cluster Computing.Academic and industrial researchers are participating in this field as it is yet evolving.For deploying cloud infrastructure, cloud services, policies related to service delivery and users for example SLA (Service Level Agreement), energy efficient computing, risk management and related policies, load balancing, communication between components and federated clouds, QoS etc. requires cloud simulators for researchers to test, evaluate and improve their ideas and findings before actual implementation in real clouds.There are few cloud simulators available that offer different features to researcher community for simulation of some aspects of clouds.The list below contains the names of modern cloud simulators that are discussed in proceeding sections.
CloudSim, with the latest release of version 3.03 at May 2, 2013, is a simulation and modeling framework for cloud services and infrastructure, developed by the "University of Melbourne's Cloud Computing and Distributed Systems (CLOUDS) Laboratory".Its development started with the motivation of providing a simplified, comprehensive and extensible structure to the researchers, developers and cloud analyst for modeling, simulation, experimentations and performance analysis [18].The main advantages of CloudSim are time effectiveness, flexibility, and applicability.It facilitates the researchers to seamlessly perform experiments and investigate results focusing towards the certain system design issues regardless of low-level infrastructural implementation details of cloud computing.CloudSim eases to model and simulate large-scale data centers, host server's virtualization with customizable policies for resource provisioning, topologies for data centers, applications that use MPI (Message Passing Interface) and federated clouds.It further offers run-time inclusion of simulation components with stop and resumes feature.Defining user-defined policies enables provisioning of resources and to analyze it [19].
CloudSim has the layered architecture and existing simulation libraries of GridSim and SimJava are exploited for development.As shown in Fig. 2, the first layer is User Code that allows cloud application developers to configure applications, design scenarios for cloud availability and testing, and implement policies for resource provisioning.It contains basic user-level entities such as the number of machines including specifications, required applications, virtual machines, the number of users and policies etc.The most important layer, CloudSim layer is the simulation layer that contains "user interface structures, virtual machine services, cloud services, cloud resources and network implementation details".CloudSim has the following main components:  Data center cops the core cloud services.It comprises a set of host entities that are allocated to virtual machines using allocation policy.VMs perform the "low level" processing.Minimum one data center need to be created for simulation.
 Host referred to a computing server that has the processing power, memory, storage and authority for assigning processor cores to virtual machines from a pool of VMs managed by the host.www.ijacsa.thesai.org The provisioning of virtual machine is the assignment of different VMs to different hosts.These are assigned so that host can schedule processing cores to virtual machines.This assignment of VM depends on the application, and the default policy is "First-Come-First-Serve".
 Datacenter broker plays a role between users and service providers to identify the best provider for user subject to conditions of QoS imposed by the user.
 Cloudlet component of CloudSim signifies the application service whose complexity is modeled in terms of the computational requirements.
 CloudCoordinator is responsible for communicating with other CloudCoordinator, services, and brokers.Furthermore, it observes the internal state of a data center periodically.

A. CloudAnalyst
CloudAnalyst is a visual tool for analyzing the cloud computing environment and applications.It is developed based on CloudSim [19].The motivation behind CloudAnalyst was the unavailability of tools that can help to estimate the requirements related to the workload on computing servers and user for geographically distributed cloud applications [20].This requirement and performance analysis is important because cloud contains a distributed infrastructure and applications may run in different geographical locations.This distribution of application effects the performance.CloudAnalyst enables to analyze the performance of extensive cloud applications based on various deployment setups by simulating them.
The CloudAnalyst is built with the extensions of CloudSim Toolkit as shown in Fig. 3.It extends the GUI package to ease with separation of programming and simulation exercises.The existing CloudSim libraries are used to model the simulation and analysis of applications behavior.CloudAnalyst has the following main components:  Internet: This is used to exhibit the Internet and traffic routing.
 InternetCharacteristics: This is used to define the Internet characteristics that are used for simulation including latency, bandwidth, and region etc.
 VmLoadBalancer: Used to implement the load balancing policies for data centers.
 CloudAppServiceBroker: This defines the cloud service broker who is responsible for managing traffic and service delivery between user and service provider.

B. GreenCloud
The absence of a meticulous simulator accessible in the market was the inspiration to develop GreenCloud that enables researchers to watch, communicate and measure cloud executions.GreenCloud is a modern open source cloud computing simulator which has been expounded with regards to the GreenIT and ECO-Cloud projects [21].
GreenCloud is NS2 (Network Simulator) simulator that focuses the packet-level simulation of cloud data centers for energy-aware computing [22].It enables the simulation of workload distribution along with acquiring the information related to energy utilized by data center components including servers, switches, and links.Furthermore, it simulates the packet level communication patterns.Fig. 4 elaborates the architecture of GreenCloud.
The architecture of GreenCloud is deployed on the basis of three-tier network architecture.Data Center layer is added on top of core network layer which provides an interface for cloud users.The components of Data Center layer are Data Center Characteristics and Task Scheduler.Data Center Characteristics is used to define the data center and Task Scheduler is responsible for schedule and workload distribution.At the "core network, aggregation network and access network layers", TaskCom Agent are responsible for routing and communication between computing servers, switches and data center components.These layers also contain the L3 and L2 energy model to capture the energy consumed by layer 3, layer 3 and rack switches.For each computing server, there is a scheduler for scheduling computing tasks, an energy model to capture energy consumed by server and server characteristics component to define the server capabilities.

C. iCanCloud
iCanCloud is an adaptable and scalable cloud computing simulation tool intended for modeling and mimicking the large cloud environments both existent and fictional cloud architectures.It was developed by the "Computer Architecture, Communications and Systems (ARCOS) Research Group at Universidad Carlos III de Madrid, Spain" [23].It provides the valuable information to users about expenses by forecasting the trade-offs between price and performance of specific applications running on certain hardware.iCanCloud can be utilized by a variety of clients, from basic active users to developers of large distributed applications.Fig. 4. GreenCloud architecture [22].www.ijacsa.thesai.orgiCanCloud has a layered architecture [24] as elaborated in Fig. 5.The lowest layer contains the hardware models such as "CPU, memory, storage and network system".It also contains the basic systems API that is responsible for providing and interfaces between hardware models and applications.The second layer is VMs repository layer on top of hardware models.It is an essential element for creating cloud system and contains the list of previously created virtual machines and Amazon EC2 VM instances.The application repository layer, on top of VMs repository layer, contains the pre-defined cloud applications that can be customized by users.The cloud hypervisor layer contains three components: 1) job management manages the incoming jobs and VMs instances where the job is performed, 2) brokering policies to define policies related to brokers and 3) cost policies to define the cost of utilized resources.The topmost layer is the cloud system layer that defines the architecture of cloud and deployment of virtual machines.Fig. 5. Architecture of iCanCloud [24].

D. MDCSim
MDCSim is an adaptable and scalable simulation toolkit for comprehensive evaluation of multi-tier data centers [25].It mainly focuses the simulation of three-tier clustered datacenters.MDCSim is comprehensive because all characteristics of a multi-tier data center are implemented in detail.It is flexible as different characteristics of the data center can be manipulated.For instance, users have provision to change the number of tiers, customize the algorithms for scheduling, and alter the communication mechanisms and interconnect.
CSIM [26] is the underlying platform for MDCSim and the simulation is constituted into three layers called "communication layer, kernel layer, and user-level layer".Fig. 6 gives an overview of MDCSim platform.

E. DCSim
DCSim is a datacenter simulator for assessing management of dynamic virtualized resource [27].It is an event-driven simulation tool that aims to simulate IaaS offering of a data center to various clients.It focuses on modeling transactional, continuous workloads (such as a web server), but can be extended to model other workloads as well.The foundation of the DCSim development has many motivational reasons including the unavailability of customizable and extensible simulation tool for, modeling multi-tenant data centers, simulating the interactions and dependencies between many VMs, resource management, VM migration and modeling of host power states.Furthermore, it enables the computation and recording of SLA violation, active host, host hours, active host utilization, the number of migrations performed, power consumption, and simulation and algorithm running time.Fig. 7 gives an overview of the architecture of DCSim.The key class is termed as DataCenter that contains hosts, virtual machines and various components for management and defining policies.A host is a collection of VMs who is responsible for hosting the task and a data center consists of many interconnected host machines.The responsibilities of the host include VM allocation, resource management, CPU scheduling and power consumption modeling.www.ijacsa.thesai.org

F. NetworkCloudSim
NetworkCloudSim is an extension of CloudSim to model the parallel applications in real networked data centers [11].Particularly, it addresses the problems and solutions with respect to modeling the internal network and applications of a data center.Furthermore, it supports applications with communicating elements or tasks such as MPI and workflows.In NetworkCloudSim, there are three main actors called Switch, NetworkDatacenter and NetworkDatacenterBroker as shown in Fig. 8.There are two main components of NetworkCloudSim, one of which contains the Switch, NetworkPacket and HostPaket classes for modeling a network topology within the data center.The second component contains the NetworkCloudlet and AppCloudlet classes for application modeling and simulation between different tasks.

G. EMUSIM
EMUSIM is targeted to support the public cloud providers for simulation and analyzing the behavior of cloud applications.It is an incorporated emulation and simulation tool for modeling, assessment, and confirmation of the execution of cloud computing applications [28], [29].Fig. 9 gives the overview of EMUSIM.It uses Automated Emulation Framework (AEF) [30] for automatically extracting information from the application performance.This extracted information is then used to model the simulation and CloudSim are used for the simulation.It also uses the QAppDeployer [31] a QoS-aware application deployerfor load generation and automated application deployment to virtual machines.Emulation empowers the execution of the authentic application in a limited environment that models the concrete creation framework, while, simulation permits evaluation of how a system/application acts light of various conditions.

H. SPECI
SPECI is a simulator for resilient cloud infrastructures that aims to explore the facets of scaling and performance properties of cloud-scale data centers [32].SPECI is built on two packages, where first specifies the layout and topology of a data center, and second comprises of the elements execution of various experiments and evaluating the results.The data center layout package is a collection of classes for each type of component that is part of the data center, for instance, nodes and network links.This is used to design the layout and topology of the data center for observation and experiments of interest.The experiment component of the SPECI uses SimKit [33] for testing and recording results.The latest public release is named as SEPCI-2 [34] in which the data center layout is structured in a hierarchical fashion.

I. D-Cloud
D-Cloud facilitates with a parallel software testing environment for reliable distributed systems that uses cloud computing technology and VMs with the facility of fault instillation [35].D-Cloud supports the fault tolerance analysis related to the failures of hardware that happen in the computing www.ijacsa.thesai.orgmachine.For this, the virtual machine layer of D-Cloud offers the facility of fault injection.Furthermore, it enables to manage computing resources flexibly and automatically, for instance, simulation test can be performed quickly by simultaneous use of resources if available.Moreover, it automates the process of system setup including fault instillation based on test scenario provided by the tester.Additionally, it automates testing phenomena by utilizing the descriptions for system configuration, and test-cases to perform tests on cloud computing systems.The design of D-Cloud exploits QEMU [36] for virtualization and Eucalyptus [37], [38] for mimicking cloud environment to manage the computing resources.D-Cloud has three components namely, QEMU nodes, Controller node, and Frontend as depicted in Fig. 10.

J. eXo Cloud-IDE
eXo IDE [39] is a web application that facilitates with an extensive test environment to develop various scripts, content, and services.The advantage of using eXo is that, it does not need installations, it runs in a browser and facilitate for retrieving and operating files online.It is capable of working with remote file system with the help of virtual file system (VFS).Additionally, it provides code editor enriched with popular programming languages such as HTML, XML, Java, PHP, Ruby and JSP for designing and testing applications.Furthermore, it includes many tools for client-side and serverside application development.Also, it facilitates with the deployment of services using "sandbox" technique.

IV. DISCUSSION
There are several simulators available for simulating cloud environment that have been discussed above.These simulators have many common and distinct features along with providing the environment for the different type of simulations.For instance, CloudSim focuses the simulation of large-scale data centers, host server's virtualization with customizable policies for resource provisioning, topologies for data centers, applications that use MPI, and federated clouds.The CloudAnalyst emphasizes on the cloud applications behavior.Further, the GreenCloud cover the simulation of the data center with respect to energy consumption.The iCanCloud aims to forecast the trade-off between price and performance.The MDCSim focuses the simulation of multitier data center applications while DCSim covers the workload and VM management.The NetworkCloudSim emphasis on the internal network of data centers.The EMUSIM targets the evaluation of cloud applications behavior.SPECI explores the requirements for scalable data centers.The D-Cloud focuses the analysis of fault-tolerance in hardware and eXo Cloud-IDE provides the environment for designing and testing the cloud applications.Table 1 gives the summary of these cloud simulators with respect to simulation emphasis.The simulation results generated with these simulators for the focused domain, applications and algorithms are helpful for Simulation is extremely important in order to validate cloud applications, algorithms, protocols, and infrastructure.During last decade, many cloud simulators have been developed by the academic and industrial researchers to simulate cloud environments for testing and evaluation.These simulators are built on different architectures and emphasis on simulating different components of cloud including applications and their behavior, data centers, virtualization, workload balancing, and internetworking of data center components.Many of these simulators facilitate with multiple features to test the applications in one or more specified areas based on some evaluation matric.Some of the available cloud simulators support cloud federation and semi multi-cloud simulations but up to the best of our knowledge, there is no simulator available that supports the full simulation of multiple clouds with different ownership, administration, and policies at the same time.The full multicloud simulation refers to simulating the multiple clouds in a single environment with complete privileges.Such privileges include the inter-connection among multiple clouds with administrative control and with running different administrative, access and security policies at multiple cloud.Further, it includes the simulation of multiple clients of multiple clouds that accesses the resources in real time.
The full multi-cloud simulation is important if we need to design and validate the cross-cloud communication models and intercommunication protocols among cloud networks.Further, it is of utmost important to test the scenarios of resource sharing and load balancing among connected clouds.In situations, when a cloud may be overloaded with requests of resources and the other cloud with under-utilized resources is available in cloud network, they may facilitate to share the cloud resources for business purposes.To design and validate such a cloud network, full multi-cloud simulation is required that may facilitate with multiple clouds running with different access and security policies.

Fig. 3 .
Fig. 3. CloudAnalyst architecture [20]. GUI Package: The front-end is the graphical user interface to control screen transitions and related functionalities. Simulation: This important component enables the development and execution of simulation by retaining the simulation parameters.UserBase: It is used to model the users and users' traffic.

TABLE I .
SUMMARY OF CLOUD SIMULATORS CloudAnalystAnalysis of the behavior of extensive cloud applications with a geographical distribution that affects the application performance.GreenCloudWorkload distribution with gathering information of energy utilized by datacenter components.Packet level communication pattern in cloud data centers for energy-aware computing.iCanCloud Forecasting the trade-offs between price and performance of applications running on certain hardware.