Impact of Data Compression on the Performance of Column-oriented Data Stores

Compression of data in traditional relational database management systems significantly improves the system performance by decreasing the size of the data that results in less data transfer time within the communication environment and higher efficiency in I/O operations. The column-oriented database management systems should perform even better since each attribute is stored in a separate column, so that its sequential values are stored and accessed sequentially on the disk. That further increases the compression efficiency as the entire column is compressed/decompressed at once. The aim of this research is to determine if data compression could improve the performance of HBase, running on a small-sized Hadoop cluster, consisted of one name node and nine data nodes. Test scenario includes performing Insert and Select queries on multiple records with and without data compression. Four data compression algorithms are tested since they are natively supported by HBase - SNAPPY, LZO, LZ4 and GZ. Results show that data compression in HBase highly improves system performance in terms of storage saving. It shrinks data 5 to 10 times (depending on the algorithm) without any noticeable additional CPU load. That allows smaller but significantly faster SSD disks to be used as cluster’s primary data storage. Furthermore, the substantial decrease in the network traffic is an additional benefit with major impact on big data processing.

@article{Mladenova2021,
  title     = {Impact of Data Compression on the Performance of Column-oriented Data Stores},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {12},
  number    = {7},
  year      = {2021},
  publisher = {The Science and Information Organization},
  author    = {Tsvetelina Mladenova and Yordan Kalmukov and Milko Marinov and Irena Valova},
  doi       = {10.14569/IJACSA.2021.0120747},
  url       = {https://doi.org/10.14569/IJACSA.2021.0120747}
}