Total views : 579

Two Objectives Big Data task Scheduling using Swarm Intelligence in Cloud Computing

Affiliations

  • Faculty of Engineering, International Islamic University Malaysia, Jalan Gombak - 53100, Kuala Lumpur, Malaysia

Abstract


Cloud computing is the latest and the most used type of distributed computing systems and also it covers most of their features. It has been widely used for its enormous benefits and its ability to cope with large scale data such as workflows and big data applications. On the other hand, scheduling algorithms; starting from traditional to Hyper-heuristic; are widely used in computing systems such as cloud computing to monitor the use of resources. However, these scheduling algorithms vary in term of their performance and most of these traditional and simple scheduling algorithms may not be efficient for large scale data. Although many scheduling algorithms have been implemented for cloud computing, it has been realized that most of the applications nowadays require different objectives that simple scheduling algorithms fail to achieve. Either one of the objective is violated or the results are far from the optimal solution. In this direction, this paper first gives review of some previous scheduling algorithms used in cloud. Then, it proposes a type of swarm intelligence called Particle Swarm Optimization (PSO) algorithm to diminish cost though meeting deadlines. The proposed method is evaluated using CloudSim and big data applications are used as sample of applications. From the results, it can be seen that PSO works better for big data applications and the cost is reduced to more than half when compared with ordinary scheduling algorithms such as First-Come-First-Serve (FCFS).

Keywords

Cloud Computing, Hadoop and Big Data, Scheduling, Swarm Optimization.

Full Text:

 |  (PDF views: 565)

References


  • Fox A, Griffith R, Joseph A, Katz R, Konwinski A, Lee G, Stoica I. Above the clouds: A Berkeley view of cloud computing. Dept Electrical Eng. and Comput Sciences, University of California, Berkeley, Rep. UCB/EECS. 2009; 28(13).
  • Zhang S, Zhang S, Chen X, Huo X. Cloud computing research and development trend. 2010 Second International Conference on In Future Networks, ICFN'10, Ieee. 2010 Jan; 93–7.
  • Rodriguez MA, Buyya R. Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Transactions on Cloud Computing. 2014; 2(2):222–35.
  • Tsai CW, Huang WC, Chiang MH, Chiang MC, Yang CS. A hyper-heuristic scheduling algorithm for cloud. IEEE Transactions on Cloud Computing. 2014; 2(2):236–50.
  • Wu Z, Liu X, Ni Z, Yuan D, Yang Y. A market-oriented hierarchical scheduling strategy in cloud workflow systems. The J Supercomput. 2013; 63(1):256–93.
  • Kwok YK, Ahmad I. Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Computing Surveys (CSUR). 1999; 31(4):406–71.
  • Arya LK, Verma A. Workflow scheduling algorithms in cloud environment-A survey. In Engineering and Computational Sciences (RAECS), 2014 Recent Advances, IEEE. 2014 Mar; 1–4.
  • Mohammadi A, Selim G. Akl. Technical Report No. 2005-499 Scheduling Algorithms for Real-Time Systems. 2005 Jul.
  • Thekkilakattil A, Baruah S, Dobrin R, Punnekkat S. The global limited preemptive earliest deadline first feasibility of sporadic real-time tasks. 2014 26th Euromicro Conference In Real-Time Systems (ECRTS), IEEE. 2014 Jul; 301–10.
  • Puchinger J, Raidl GR. Combining metaheuristics and exact algorithms in combinatorial optimization: A survey and classification, Springer Berlin Heidelberg. 2005; 41–53.
  • Astaraky D, Patrick J. A simulation based approximate dynamic programming approach to multi-class, multi-resource surgical scheduling. European Journal of Operational Research. 2015; 245(1):309–19.
  • Geranmayeh S. Optimizing surgical scheduling through integer programming and robust optimization. 2015.
  • Stavrinides GL, Karatza HD. A Cost-Effective and QoS-Aware Approach to Scheduling Real-Time Workflow Applications in PaaS and SaaS Clouds. 2015 3rd International Conference on Future Internet of Things and Cloud (FiCloud), IEEE. 2015.
  • Alrokayan M, Vahid Dastjerdi A, Buyya R. SLA-aware Provisioning and Scheduling of Cloud Resources for Big Data Analytics. 2014 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM), IEEE. 2014 Oct; 1–8.
  • Abrishami S, Naghibzadeh M, Epema DH. Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds. Future Generation Computer Systems. 2013; 29(1):158–69.
  • Kantorovich LV. A new method of solving of some classes of extremal problems. In Dokl. Akad. Nauk SSSR. 1940; 28:211–4.
  • Zhou J, Love PE, Wang X, Teo KL, Irani Z. A review of methods and algorithms for optimizing construction scheduling. Journal of the Operational Research Society. 2013; 64(8):1091–105.
  • Zhao Y, Calheiros RN, Gange G, Ramamohanarao K, Buyya R. SLA-Based Resource Scheduling for Big Data Analytics as a Service in Cloud Computing Environments. 2015 44th International Conference on Parallel Processing (ICPP), IEEE. 2015 Sep; 510–9.
  • Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. Equation of state calculations by fast computing machines. The journal of chemical physics. 1953; 21(6):1087–92.
  • Kune R, Konugurthi PK, Agarwal A, Chillarige RR, Buyya R. Genetic Algorithm based Data-aware Group Scheduling for Big Data Clouds. 2014 IEEE/ACM International Symposium on Big Data Computing (BDC), IEEE. 2014 Dec; 96–104.
  • Zhang W, Tan S, Lu Q, Liu X, Gong W. A genetic-algorithm-based approach for task migration in pervasive clouds. International Journal of Distributed Sensor Networks. 2015; 14.
  • Zhong H, Tao K, Zhan, X. An approach to optimized resource scheduling algorithm for open-source cloud systems. 2010 Fifth Annual in China Grid Conference (ChinaGrid), IEEE. 2010 Jul; 124–9.
  • Kennedy J. Particle swarm optimization. In Encyclopedia of machine learning . Springer US. 2011; 760–6.
  • Wang X, Cao B, Hou C, Xiong L, Fan J. Scheduling Budget Constrained Cloud Workflows with Particle Swarm Optimization. 2015 IEEE Conference on Collaboration and Internet Computing (CIC), IEEE. 2015 Oct. p. 219–26.
  • Chen H, Guo W. Real-Time Task Scheduling Algorithm for Cloud Computing Based on Particle Swarm Optimization. Cloud Computing and Big Data. Springer International Publishing. 2015; 141–52.
  • Radha K, Rao BT. Slot Utilization and Performance Improvement in Hadoop Cluster. In Information Systems Design and Intelligent Applications. Springer India. 2016; 49–62.
  • Xu L, Qian F, Li Y, Li Q, Yang YW, Xu J. Resource allocation based on quantum particle swarm optimization and RBF neural network for overlay cognitive OFDM System. Neurocomputing. 2016; 173:1250–6.
  • Zaharia M, Borthakur D, Sen Sarma J, Elmeleegy K, Shenker S, Stoica I. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In Proceedings of the 5th European conference on Computer systems. ACM. 2010 Apr; 265–78.
  • Marz N, Warren J. Big Data: Principles and best practices of scalable realtime data systems. Manning Publications Co. 2015.
  • Cheng S, Zhang Q, Qin Q. Big data analytics with swarm intelligence. Industrial Management and Data Systems. 2016; 116(4).

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.