[1]郑羽,胡积宝.Hadoop集群中给定候选任务集的最大利润问题[J].计算机技术与发展,2018,28(12):194-199.[doi:10.3969/j.issn.1673-629X.2018.12.041]
　ZHENG Yu,HU Jibao.Maximum Profit Problem of a Given Candidate Set in Hadoop Cluster[J].,2018,28(12):194-199.[doi:10.3969/j.issn.1673-629X.2018.12.041]
点击复制

Hadoop集群中给定候选任务集的最大利润问题()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
28
期数:
2018年12期
页码:
194-199
栏目:
应用开发研究
出版日期:
2018-12-10

文章信息/Info

Title:
Maximum Profit Problem of a Given Candidate Set in Hadoop Cluster
文章编号:
1673－629X(2018)12－0194－06
作者:
郑羽1; 胡积宝2
1.安庆师范大学现代教育技术中心;2.安庆师范大学物理与电气工程学院
Author(s):
ZHENG Yu1; HU Ji－bao2
1．Modern Education and Technology Center，Anqing Normal University，Anqing 246011，China; 2．School of Physics and Electronic Engineering，Anqing Normal University，Anqing 246133，China
关键词:
MapReduce;  任务集;  调度算法;  利润;  大数据
Keywords:
MapReduce; task set; scheduling algorithm; profit; big data
分类号:
P301.6
DOI:
10.3969/j.issn.1673-629X.2018.12.041
摘要:
随着计算机网络和传感器网络的迅速发展,数据呈指数级增长,特别是在因特网上.为了有效地处理大规模数据,需要具有良好的可伸缩性、灵活性和容错性的并行分布式集群.目前,许多企业基于自己的Hadoop集群提供云服务.因为单个Hadoop集群的资源是有限的,Hadoop集群必须将有限的资源分配给一些特殊的任务以获得最大的利益.文中研究给定候选任务集的最大利润问题.用有效的序列描述候选任务集,并提出了一种基于序列的调度策略.为了提高查找有效序列的效率,设计了一些修剪策略,并给出了相应的调度算法.最后,在某些任务运行超时的情况下,提出了超时处理算法.实验结果表明,该算法的总收益非常接近理想的最大值,在不同的实验环境下明显优于相关的调度算法.
Abstract:
With the rapid development of computer networks and sensor networks，data are exponentially increased，especially on the Internet． In order to deal with large－scale data efficiently，a parallel and distributed cluster with better scalability，flexibility and fault tolerance is needed． Nowadays，many enterprises provide cloud services based on their own Hadoop clusters． Because the resources of a Hadoop cluster are limited，the Hadoop cluster must select some specific tasks to allocate limited resources in order to get the maximal profit． In this paper，we study the maximal profit problem for a given candidate task set． We describe the candidate task set with a valid sequence and propose a sequence－based scheduling strategy． In order to improve the efficiency of finding a valid sequence，we design some pruning strategies and give the corresponding scheduling algorithm． Finally，we propose a timeout handling algorithm when some task runs timeout． Experiment shows that the total profit of the proposed algorithm is very close to the ideal maxima and is obviously bigger than related scheduling algorithms under different experimental settings．

相似文献/References:

[1]李玲娟 张敏.云计算环境下关联规则挖掘算法的研究[J].计算机技术与发展,2011,(02):43.
　LI Ling-juan,ZHANG Min.Research on Algorithms of Mining Association Rule under Cloud Computing Environment[J].,2011,(12):43.
[2]李远方 邓世昆 闻玉彪 韩月阳.Hadoop-MapReduce下的PageRank矩阵分块算法[J].计算机技术与发展,2011,(08):6.
　LI Yuan-fang,DENG Shi-kun,WEN Yu-biao,et al.PageRank Matrix Partitioned Algorithm Using Hadoop-MapReduce[J].,2011,(12):6.
[3]李远方 贾时银 邓世昆 韩月阳.基于树结构的MapReduce模型[J].计算机技术与发展,2011,(08):149.
　LI Yuan-fang,JIA Shi-yin,DENG Shi-kun,et al.MapReduce Model Based on Tree Structure[J].,2011,(12):149.
[4]王梅,朱信忠,赵建民,等.基于 Hadoop 的海量图像检索系统[J].计算机技术与发展,2013,(01):204.
　WANG Mei,ZHU Xin-zhong,ZHAO Jian-min,et al.Massive Images Retrieval System Based on Hadoop[J].,2013,(12):204.
[5]贺瑶,王文庆,薛飞.基于云计算的海量数据挖掘研究[J].计算机技术与发展,2013,(02):69.
[6]舒琰,向阳,张骐,等.基于PageRank的微博排名MapReduce算法研究[J].计算机技术与发展,2013,(02):73.
　SHU Yan,XIANG Yang,ZHANG Qi,et al.Research on MapReduce Algorithm of Micro Blog Ranking Based on PageRank[J].,2013,(12):73.
[7]朱贤军,李敬兆.无加密模式下对云数据的隐私保密[J].计算机技术与发展,2013,(06):216.
　ZHU Xian-jun,LI Jing-zhao.Cloud Data Privacy under None Encryption[J].,2013,(12):216.
[8]周婷,张君瑛,罗成.基于Hadoop的K-means聚类算法的实现[J].计算机技术与发展,2013,(07):18.
　ZHOU Ting[],ZHANG Jun-ying[],LUO Cheng[].Realization of K-means Clustering Algorithm Based on Hadoop[J].,2013,(12):18.
[9]孙媛,黄刚. 基于Hadoop平台的C4.5算法的分析与研究[J].计算机技术与发展,2014,24(11):83.
　SUN Yuan,HUANG Gang. Analysis and Study of C4 . 5 Algorithm Based on Hadoop Platform[J].,2014,24(12):83.
[10]王添,姜麟,米允龙. 海量数据下不完备信息系统的知识约简算法[J].计算机技术与发展,2015,25(01):137.
　WANG Tian,JIANG Lin,MI Yun-long. Knowledge Reduction Algorithms of Incomplete Information System in Massive Datasets[J].,2015,25(12):137.

更新日期/Last Update: 2018-12-10