[1]秦军[],童毅[],戴新华[],等. 基于MapReduce数据密集型负载调度策略研究[J].计算机技术与发展,2015,25(04):48-52.
　QIN Jun[],TONG Yi[],DAI Xin-hua[],et al. Research on Scheduling Strategy of Data Intensive Workloads Based on MapReduce[J].,2015,25(04):48-52.
点击复制

 基于MapReduce数据密集型负载调度策略研究()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
25
期数:
2015年04期
页码:
48-52
栏目:
智能、算法、系统工程
出版日期:
2015-04-10

文章信息/Info

Title:
 Research on Scheduling Strategy of Data Intensive Workloads Based on MapReduce
文章编号:
1673-629X（2015)04-0001-04
作者:
 秦军[1]; 童毅[2]; 戴新华[2]; 林巧民[1]
1.南京邮电大学 教育科学与技术学院；2.南京邮电大学 计算机学院,
Author(s):
 QIN Jun[1] ; TONG Yi[2];  DAI Xin-hua[2] ; LIN Qiao-min[1]
关键词:
 ; 云计算; 数据本地性; MapReduce; Hadoop
Keywords:
 ; cloud computing; data locality; MapReduce; Hadoop
分类号:
TP301.6
文献标志码:
A
摘要:
 针对云计算环境中大规模数据集的处理,MapReduce集群已成为一个强大的处理平台。文中提出了一种基于虚拟化平台动态资源重配置的资源评价和动态资源重新配置调度算法。该算法动态地评估作业在截止时间内完成所需要的Map和Reduce计算资源数量,并在不违反用户设定的时间目标的情况下,通过动态地增加或减少独立虚拟机的方式来调整CPU资源,以实现提高数据本地性,同时提高系统在运行作业时的资源利用率。仿真实验结果表明,该算法可以使集群上的MapReduce作业的吞吐率有明显的提高。
Abstract:
 Aiming at processing of large-scale data set in cloud computing environment,MapReduce has become a powerful processing platform. In this paper,propose a resource evaluation and dynamic resource reconfiguration and scheduling algorithm based on virtualiza-tion platform dynamic resource reconfiguration. It can dynamically evaluate the required number of Map/Reduce slots for every job to meet completion time guarantee and adjust the CPU resources while not violating completion time goals of the users by dynamically in-creasing or decreasing individual VMs to maximize data locality and also to maximize the use of resources within the system among the active jobs. Simulation results show that the algorithm can improve the throughput of MapReduce jobs on the cluster significantly.

相似文献/References:

[1]王茜,朱志祥,史晨昱,等.应用于数据库安全保护的加解密引擎系统[J].计算机技术与发展,2014,24(01):143.
　WANG Qian[],ZHU Zhi-xiang[],SHI Chen-yu[],et al.Encryption and Decryption Engine System Applying to Database Security and Detection[J].,2014,24(04):143.
[2]陈丹伟 黄秀丽 任勋益.云计算及安全分析[J].计算机技术与发展,2010,(02):99.
　CHEN Dan-wei,HUANG Xiu-li,REN Xun-yi.Analysis of Cloud Computing and Cloud Security[J].,2010,(04):99.
[3]孙放 陈云芳 林杭锋.适用于富客户端的云计算模型[J].计算机技术与发展,2010,(08):96.
　SUN Fang,CHEN Yun-fang,LIN Hang-feng.Cloud Computing Model Applicable to Rich Client Applications[J].,2010,(04):96.
[4]郭苑 张顺颐 孙雁飞.物联网关键技术及有待解决的问题研究[J].计算机技术与发展,2010,(11):180.
　GUO Yuan,ZHANG Shun-yi,SUN Yan-fei.Research of Key Technologies and Unresolved Questions of Internet of Things[J].,2010,(04):180.
[5]李玲娟 张敏.云计算环境下关联规则挖掘算法的研究[J].计算机技术与发展,2011,(02):43.
　LI Ling-juan,ZHANG Min.Research on Algorithms of Mining Association Rule under Cloud Computing Environment[J].,2011,(04):43.
[6]王德政 申山宏 周宁宁.云计算环境下的数据存储[J].计算机技术与发展,2011,(04):81.
　WANG De-zheng,SHEN Shan-hong,ZHOU Ning-ning.Data Storage in Cloud Computing Environment[J].,2011,(04):81.
[7]宋丽华 姜家轩 张建成 田长录 马文征.黄河三角洲云计算平台关键技术的研究[J].计算机技术与发展,2011,(06):40.
　SONG Li-hua,JIANG Jia-xuan,ZHANG Jian-cheng,et al.Research of Key Technologies of Cloud Computing of Yellow River Delta[J].,2011,(04):40.
[8]田宏伟 解福 倪俊敏.云计算环境下基于粒子群算法的资源分配策略[J].计算机技术与发展,2011,(12):22.
　TIAN Hong-wei,XIE Fu,NI Jun-min.Resource Allocation Algorithm Based on Particle Swarm Algorithm in Cloud Computing Environment[J].,2011,(04):22.
[9]张慧 邢培振.云计算环境下信息安全分析[J].计算机技术与发展,2011,(12):164.
　ZHANG Hui,XING Pei-zhen.Information Security Analysis in Cloud Computing Environment[J].,2011,(04):164.
[10]张建成[] 宋丽华[] 鹿全礼[] 郭锐[] 刘永泉[].云计算方案分析研究[J].计算机技术与发展,2012,(01):165.
　ZHANG Jian-cheng,SONG Li-hua,LU Quan-li,et al.Study and Analysis of Cloud Computing Procedure[J].,2012,(04):165.
[11]王雷,陈彦先,袁哲,等. 面向预拌混凝土行业的云计算[J].计算机技术与发展,2014,24(08):14.
　WANG Lei,CHEN Yan-xian,YUAN Zhe JI Xu. Research on Cloud Computing for Ready-mixed Concrete Industry[J].,2014,24(04):14.
[12]殷小龙,李君,万明祥. 云环境下基于改进NSGA II的虚拟机调度算法[J].计算机技术与发展,2014,24(08):71.
　YIN Xiao-long,LI Jun,WAN Ming-xiang. Virtual Machines Scheduling Algorithm Based on Improved NSGA II in Cloud Environment[J].,2014,24(04):71.
[13]张也弛,周文钦,石润华. 一种面向云的大数据完整性检测协议[J].计算机技术与发展,2014,24(09):68.
　ZHANG Ye-chi,ZHOU Wen-qin,SHI Run-hua. A Big Data Integrity Checking Protocol for Cloud[J].,2014,24(04):68.
[14]徐源吾[][],王珣[][]. 基于Hadoop的智能家居信息处理平台[J].计算机技术与发展,2014,24(09):183.
　XU Yuan-wu[] [],WANG Xun[][]. nformation Processing Platform of Smart Home Based on Hadoop[J].,2014,24(04):183.
[15]谢福伟,梁昌勇,马银超. 基于云计算的景区数据仓库应用研究[J].计算机技术与发展,2014,24(09):198.
　XIE Fu-wei,LIANG Chang-yong,MA Yin-chao. Research on Data Warehouse Application of Tourist Areas Data Based on Cloud Computing[J].,2014,24(04):198.
[16]孙滔,王杉,邢军. 文献共享系统和数据共享系统的云计算平台建设[J].计算机技术与发展,2014,24(09):206.
　SUN Tao,WANG Shan,XING Jun. Construction of Cloud Computing Platform of Sci-tech Literature Sharing System and Data Sharing System[J].,2014,24(04):206.
[17]周文琼[],王乐球[],郑述招[]. 云环境下的数据库扩展策略的设计[J].计算机技术与发展,2014,24(09):213.
　ZHOU Wen-qiong[],WANG Le-qiu[],ZHENG Shu-zhao[]. Design of Database Expansion Strategy under Cloud Computing[J].,2014,24(04):213.
[18]申侃,梁昌勇,赵树平. 基于云的MIS开放式体系结构[J].计算机技术与发展,2014,24(10):21.
　SHEN Kan,LIANG Chang-yong,ZHAO Shu-ping. Open Architecture of MIS Based on Cloud[J].,2014,24(04):21.
[19]王霞俊. 云环境下一种基于能耗感知的虚拟机部署算法[J].计算机技术与发展,2014,24(10):88.
　WANG Xia-jun. A Virtual Machine Allocation Algorithm Based on Power-aware in Cloud Computing[J].,2014,24(04):88.
[20]孟蒙,茅苏. 基于云计算的可反馈负载均衡策略的研究[J].计算机技术与发展,2014,24(10):135.
　MENG Meng,MAO Su. Study on Feedback Load Balancing Strategy Based on Cloud Computing[J].,2014,24(04):135.

更新日期/Last Update: 2015-06-04