微信服务号

微信订阅号

2025-4-24- 15
Home > Archive>Volume 31, Issue 6, 2020 >1860-1874. DOI:10.13328/j.cnki.jos.005710
PDF HTML XML Export Cite reminder
Heuristic Based Resource Provisioning Approach for Big Data Analytics in Cloud Environment
Author:
Affiliation:

Clc Number:

TP316

Fund Project:

National Key Research and Development Program of China (2017YFB1400804); Beijing Natural Science Foundation (4182070); Ant Financial Research Fund (XZ502017000730); Youth Innovation Promotion Association of Chinese Academy of Sciences Fund (2018144)

  • Article
    • | |
  • Metrics
    • |
  • Reference
    • |
  • Related
    • |
  • Cited by
    • | |
  • Comments
    Abstract:

    It is a big challenge to pick up the best cloud configuration for recurring big data analytics jobs running in clouds. Prior efforts may get in a sub-optimal configuration due to a broad spectrum of cloud configurations with a few test runs, such as CherryPick. RP-CH, presented in this paper, is a resource provisioning system that leverages heuristic rules based on classification information to identify the optimal cloud configuration for big data analytics jobs, while the insight is classifying a job by comparing its resource preference and usage information with other jobs. Then, heuristic rules are used to distinguish bad samples from good ones in Bayesian optimization algorithm. The experiments on HiBench and SparkBench in Aliyun ECS show that the performance of job has been improved by 58% in average comparing with CherryPick, meanwhile the resource cost has been reduced by 44% in average.

    Reference
    Related
    Cited by
Get Citation

吴悦文,吴恒,任杰,张文博,魏峻,王焘,钟华.面向大数据分析作业的启发式云资源供给方法.软件学报,2020,31(6):1860-1874

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:June 06,2018
  • Revised:September 30,2018
  • Online: June 04,2020
  • Published: June 06,2020
Links:Institute of Software, CASChina Computer FederationCASNSFC
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063