Copyright Statement: This is an open access article licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, even commercially as long as the original work is properly cited.
Digital Object Identifier (DOI) : 10.14569/IJACSA.2012.030925
Article Published in International Journal of Advanced Computer Science and Applications(IJACSA), Volume 3 Issue 9, 2012.
Abstract: This paper describes mutual exclusion principle for multithreaded web crawlers. The existing web crawlers use data structures to hold frontier set in local address space. This space could be used to run more crawler threads for faster operation. All crawler threads fetch the URL to crawl from the centralized frontier. The mutual exclusion principle is used to provide access to frontier for each crawler thread in synchronized manner to avoid deadlock. The approach to utilize the waiting time on mutual exclusion lock in efficient manner has been discussed in detail.
Kartik Kumar Perisetla, “Mutual Exclusion Principle for Multithreaded Web Crawlers” International Journal of Advanced Computer Science and Applications(IJACSA), 3(9), 2012. http://dx.doi.org/10.14569/IJACSA.2012.030925