Comparative Analysis of Spark and Ignite for Big Spatial Data Processing

Recently, spatial data became one of the most interesting fields related to big data studies, in which the spatial data have been generated and consumed from different resources. However, the increasing numbers of location-based services and applications such as Google Maps, vehicle navigation, recommendation systems are the main foundation of the idea of spatial data. On the other hand, several researchers started to discover and compared spatial frameworks to understand the requirements for spatial database processing, manipulating, and analysis systems. Apache Spark, Apache Ignite, and Hadoop are the most widely known frameworks for large data processing. However, Apache Spark, Apache Ignite have integrated different spatial data operations and analysis queries, but each system has its advantages and disadvantages when dealing with spatial data. Dealing with a new framework or system that needs to integrate new functionality sometimes becomes a risky decision if we did not examine it well The main aim of this research is to conduct a comprehensive evaluation of big spatial data computing on two well-known data management systems Apache Ignite and Apache Spark. The comparative has been done on four different domains, experimental environment setup, supported features, supported functions and queries, and performance and execution time. The results show that GeoSpark has recorded more flexibility to use than SpatialIgnite. We thoroughly investigated and discovered that multiple factors affect the performance of both frameworks, such as CPU, Main memory, data set size the complexity of data type, and programming environment. spark is more advanced and equipped with several functionalities that made it well suitable with spatial data queries and indexing. such as kNN queries; in which these functionalities are not supported in SpatialIgnite.

@article{Abuayeid2021,
  title     = {Comparative Analysis of Spark and Ignite for Big Spatial Data Processing},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {12},
  number    = {9},
  year      = {2021},
  publisher = {The Science and Information Organization},
  author    = {Samah Abuayeid and Louai Alarabi},
  doi       = {10.14569/IJACSA.2021.0120980},
  url       = {https://doi.org/10.14569/IJACSA.2021.0120980}
}