Abstract
Graph processing model is being adopted extensively in various domains such as online gaming, social media, scientific computing and Internet of Things (IoT). Many frameworks have been developed in recent years to facilitate analytics and computing of large-scale graphs. Dynamic scalability is always a major concern and It becomes even more important when there is a correlation between scalability and monetary cost. The pay-as-you-go model that is used by public cloud providers enables users to pay only for the number of resources they utilize. Nevertheless, processing large-scale graphs in such environments has been less studied. In this paper, we have developed algorithms to take advantage of resource heterogeneity in cloud environments. Using these algorithms, the system can automatically adjust the number and types of virtual machines according to the computation requirements for convergent graph applications to improve the performance and reduce the dollar cost of the entire operation. Also, a smart profiling mechanism along with a novel dynamic repartitioning approach helps to distribute graph partitions expeditiously. It is shown that this method outperforms popular frameworks such as Giraph and decreases more than 50% of the dollar cost compared to Giraph.
Funding Information
  • ARC Future Fellowship
  • ARC Discovery

This publication has 31 references indexed in Scilit: