[1]李振举,李学军,刘涛,等. MapReduce 性能预测模型构建[J].计算机技术与发展,2016,26(01):70-73.
　LI Zhen-ju,LI Xue-jun,LIU Tao,et al. Performance Prediction Model Construction of MapReduce[J].,2016,26(01):70-73.
点击复制

 MapReduce 性能预测模型构建()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
26
期数:
2016年01期
页码:
70-73
栏目:
智能、算法、系统工程
出版日期:
2016-01-10

文章信息/Info

Title:
 Performance Prediction Model Construction of MapReduce
文章编号:
1673-629X（2016)01-0170-04
作者:
 李振举; 李学军; 刘涛; 杨晟
 装备学院 信息装备系
Author(s):
 LI Zhen-ju; LI Xue-jun; LIU Tao; YANG Sheng
关键词:
 ; MapReduce; 云计算模型; 性能预测; 多元线性回归模型
Keywords:
 ; MapReduce; cloud computing model; performance prediction; multiple linear regression model
分类号:
TP393
文献标志码:
A
摘要:
 MapReduce 是目前大数据处理中应用最广泛的云计算模型，预测其性能有利于提高云计算的效率。然而 MapRe-duce 运行需要依赖大量的配置参数，这些参数会对 MapReduce 性能产生较大的影响。传统的 MapReduce 模型的配置参数的预测方法都是基于管理员经验的定性分析，无法准确预测 MapReduce 模型运行时间。为更好地对 MapReduce 性能进行预测，利用数学分析中的多元线性回归方法，在分析现有的影响 MapReduce 性能的配置参数的基础上，构建了 MapReduce性能和其配置参数之间的多元线性回归模型。为了验证该方法的正确性，以两个最重要的配置参数 Map 和 Reduce 数量为例进行了算例验证。实验结果表明，多元线性回归模型可以用来预测 MapReduce 性能。
Abstract:
 apReduce is the most popular cloud computing model in big data processing. Predicting the performance of MapReduce could be used to increase the cloud computing efficiency. However,MapReduce runs based on a huge number of configuration parameters which would affect the performance. Traditional predicting of configuration is based on the experience of administrator,and this approach is of low accuracy. In order to give a better prediction of MapReduce performance,a multiple linear regression model based on the configura-tion parameters was proposed. With the aim to verify the model,an experiment was carried out taking the Map number and Reduce num-ber as an example. The experiments results indicate that the proposed model can be used in predicting the MapReduce performance.

相似文献/References:

[1]李玲娟 张敏.云计算环境下关联规则挖掘算法的研究[J].计算机技术与发展,2011,(02):43.
　LI Ling-juan,ZHANG Min.Research on Algorithms of Mining Association Rule under Cloud Computing Environment[J].,2011,(01):43.
[2]李远方 邓世昆 闻玉彪 韩月阳.Hadoop-MapReduce下的PageRank矩阵分块算法[J].计算机技术与发展,2011,(08):6.
　LI Yuan-fang,DENG Shi-kun,WEN Yu-biao,et al.PageRank Matrix Partitioned Algorithm Using Hadoop-MapReduce[J].,2011,(01):6.
[3]李远方 贾时银 邓世昆 韩月阳.基于树结构的MapReduce模型[J].计算机技术与发展,2011,(08):149.
　LI Yuan-fang,JIA Shi-yin,DENG Shi-kun,et al.MapReduce Model Based on Tree Structure[J].,2011,(01):149.
[4]王梅,朱信忠,赵建民,等.基于 Hadoop 的海量图像检索系统[J].计算机技术与发展,2013,(01):204.
　WANG Mei,ZHU Xin-zhong,ZHAO Jian-min,et al.Massive Images Retrieval System Based on Hadoop[J].,2013,(01):204.
[5]贺瑶,王文庆,薛飞.基于云计算的海量数据挖掘研究[J].计算机技术与发展,2013,(02):69.
[6]舒琰,向阳,张骐,等.基于PageRank的微博排名MapReduce算法研究[J].计算机技术与发展,2013,(02):73.
　SHU Yan,XIANG Yang,ZHANG Qi,et al.Research on MapReduce Algorithm of Micro Blog Ranking Based on PageRank[J].,2013,(01):73.
[7]朱贤军,李敬兆.无加密模式下对云数据的隐私保密[J].计算机技术与发展,2013,(06):216.
　ZHU Xian-jun,LI Jing-zhao.Cloud Data Privacy under None Encryption[J].,2013,(01):216.
[8]周婷,张君瑛,罗成.基于Hadoop的K-means聚类算法的实现[J].计算机技术与发展,2013,(07):18.
　ZHOU Ting[],ZHANG Jun-ying[],LUO Cheng[].Realization of K-means Clustering Algorithm Based on Hadoop[J].,2013,(01):18.
[9]张志宏,吴庆波,邵立松,等.基于飞腾平台TOE协议栈的设计与实现[J].计算机技术与发展,2014,24(07):1.
　ZHANG Zhi-hong,WU Qing-bo,SHAO Li-song,et al. Design and Implementation of TCP/IP Offload Engine Protocol Stack Based on FT Platform[J].,2014,24(01):1.
[10]梁文快,李毅. 改进的基因表达算法对航班优化排序问题研究[J].计算机技术与发展,2014,24(07):5.
　LIANG Wen-kuai,LI Yi. Research on Optimization of Flight Scheduling Problem Based on Improved Gene Expression Algorithm[J].,2014,24(01):5.
[11]孙媛,黄刚. 基于Hadoop平台的C4.5算法的分析与研究[J].计算机技术与发展,2014,24(11):83.
　SUN Yuan,HUANG Gang. Analysis and Study of C4 . 5 Algorithm Based on Hadoop Platform[J].,2014,24(01):83.
[12]王添,姜麟,米允龙. 海量数据下不完备信息系统的知识约简算法[J].计算机技术与发展,2015,25(01):137.
　WANG Tian,JIANG Lin,MI Yun-long. Knowledge Reduction Algorithms of Incomplete Information System in Massive Datasets[J].,2015,25(01):137.
[13]秦军[],童毅[],戴新华[],等. 基于MapReduce数据密集型负载调度策略研究[J].计算机技术与发展,2015,25(04):48.
　QIN Jun[],TONG Yi[],DAI Xin-hua[],et al. Research on Scheduling Strategy of Data Intensive Workloads Based on MapReduce[J].,2015,25(01):48.
[14]陈静,郑彦. 基于二叉树的并行频繁项集挖掘算法[J].计算机技术与发展,2015,25(10):80.
　CHEN Jing,ZHENG Yan. Parallel Algorithm of Frequent Itemset Mining Based on Binary-tree[J].,2015,25(01):80.
[15]李程,曹菡,师军. 基于MapReduce的混合推荐算法及应用[J].计算机技术与发展,2016,26(04):74.
　LI Cheng,CAO Han,SHI Jun. Hybrid Recommendation Algorithm Based on MapReduce and Its Application[J].,2016,26(01):74.
[16]郭先超,林宗缪,姚文勇. 互联网+质量检测平台设计[J].计算机技术与发展,2016,26(05):120.
　GUO Xian-chao,LIN Zong-miao,YAO Wen-yong. Design of Platform for Internet+ Quality Inspection[J].,2016,26(01):120.
[17]范素娟[],田军锋[]. 基于Hadoop的云计算平台研究与实现[J].计算机技术与发展,2016,26(07):127.
　FAN Su-juan[],TIAN Jun-feng[]. Research and Implementation of Cloud Computing Platform Based on Hadoop[J].,2016,26(01):127.
[18]蒋菱[],王旭东[],于建成[],等. 基于分布式计算的海量用电数据分析技术研究[J].计算机技术与发展,2016,26(12):176.
　JIANG Ling[],WANG Xu-dong[],YU Jian-cheng[],et al. Research on Power Usage Behavior Analysis Based on Distributed Computing[J].,2016,26(01):176.
[19]王伟,杨庚,张成果. CryptDB密文数据库系统并行方案研究[J].计算机技术与发展,2017,27(02):90.
　WANG Wei,YANG Geng,ZHANG Cheng-guo. Investigation on Parallel Scheme of CryptDB Encrypted Database System[J].,2017,27(01):90.
[20]杨洁,黄刚. 基于云计算的SPRINT算法研究[J].计算机技术与发展,2017,27(03):108.
　YANG Jie,HUANG Gang. Research on SPRINT Algorithm Based on Cloud Computing[J].,2017,27(01):108.

更新日期/Last Update: 2016-04-12