[1]范素娟[],田军锋[]. 基于Hadoop的云计算平台研究与实现[J].计算机技术与发展,2016,26(07):127-130.
　FAN Su-juan[],TIAN Jun-feng[]. Research and Implementation of Cloud Computing Platform Based on Hadoop[J].,2016,26(07):127-130.
点击复制

 基于Hadoop的云计算平台研究与实现()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
26
期数:
2016年07期
页码:
127-130
栏目:
应用开发研究
出版日期:
2016-07-10

文章信息/Info

Title:
 Research and Implementation of Cloud Computing Platform Based on Hadoop
文章编号:
1673-629X（2016)07-0127-04
作者:
 范素娟[1]; 田军锋[2]
1. 河南大学 信息化管理办公室；2.河南大学 计算机与信息工程学院
Author(s):
 FAN Su-juan[1]; TIAN Jun-feng[2]
关键词:
 ; Hadoop; HDFS; MapReduce; 云计算
Keywords:
 ; Hadoop; HDFS; MapReduce; cloud computing
分类号:
TP39
文献标志码:
A
摘要:
 随着网络技术的发展，网络数据量正以指数级增长且规模日渐庞大。面对正在增长的海量数据，传统的数据处理方法存在效率低下等诸多缺点。人们需要一种新的技术思想来解决这些问题。因此，云计算的思想被提出。云计算是一种新兴的计算模型，是分布式计算技术的一种。而Hadoop作为一个开源的分布式平台是当前最为流行的云计算平台实现之一，被用于高效地处理海量数据。为了提高对海量数据处理的效率，文中首先简要分析了云计算的概念和Hadoop主要组件的工作流程，然后详细介绍了基于Hadoop的云计算平台配置方法和实现过程，并对云平台的搭建过程中遇到的典型问题进行了总结阐述。最后通过实验证明，该平台可以有效地完成分布式数据处理任务。
Abstract:
 With the development of network technology,the number of online information is increasing in exponential and becoming lar-ger and larger. With the growing amount of data,the traditional methods for processing massive data have many shortcomings like low ef-ficiency. A novel technology is needed to solve these problems,so the cloud computing has been brought. It is an emerging computational model,as a kind of distributed computing technology. Hadoop is one of the most popular cloud computing platforms as a kind of open sources distributed platform,which is always applied on the area that needs to handle massive data efficiently. In order to improve the effi-ciency of processing massive data,it briefly analyzes the concept of cloud computing and the work flow of the main components of Ha-doop in this paper,then introduction of the implementation method of the cloud computing platform based on Hadoop in detail,discussion of the typical problems encountered in the process of building cloud computing platform. Finally,the experiments show that the platform can effectively complete the processing tasks of distributed data.

相似文献/References:

[1]李远方 邓世昆 闻玉彪 韩月阳.Hadoop-MapReduce下的PageRank矩阵分块算法[J].计算机技术与发展,2011,(08):6.
　LI Yuan-fang,DENG Shi-kun,WEN Yu-biao,et al.PageRank Matrix Partitioned Algorithm Using Hadoop-MapReduce[J].,2011,(07):6.
[2]李远方 贾时银 邓世昆 韩月阳.基于树结构的MapReduce模型[J].计算机技术与发展,2011,(08):149.
　LI Yuan-fang,JIA Shi-yin,DENG Shi-kun,et al.MapReduce Model Based on Tree Structure[J].,2011,(07):149.
[3]王梅,朱信忠,赵建民,等.基于 Hadoop 的海量图像检索系统[J].计算机技术与发展,2013,(01):204.
　WANG Mei,ZHU Xin-zhong,ZHAO Jian-min,et al.Massive Images Retrieval System Based on Hadoop[J].,2013,(07):204.
[4]王晓军,孙惠.基于MapReduce的多路连接优化方法研究[J].计算机技术与发展,2013,(06):59.
　WANG Xiao-jun,SUN Hui.Research of Optimizing Multiway Joins Based on MapReduce[J].,2013,(07):59.
[5]朱贤军,李敬兆.无加密模式下对云数据的隐私保密[J].计算机技术与发展,2013,(06):216.
　ZHU Xian-jun,LI Jing-zhao.Cloud Data Privacy under None Encryption[J].,2013,(07):216.
[6]周婷,张君瑛,罗成.基于Hadoop的K-means聚类算法的实现[J].计算机技术与发展,2013,(07):18.
　ZHOU Ting[],ZHANG Jun-ying[],LUO Cheng[].Realization of K-means Clustering Algorithm Based on Hadoop[J].,2013,(07):18.
[7]吕婉琪,钟诚,唐印浒,等.Hadoop分布式架构下大数据集的并行挖掘[J].计算机技术与发展,2014,24(01):22.
　L Wan-qi,ZHONG Cheng,TANG Yin-hu,et al.Parallel Mining of Large Dataset in Hadoop Distributed Computing Framework[J].,2014,24(07):22.
[8]张志宏,吴庆波,邵立松,等.基于飞腾平台TOE协议栈的设计与实现[J].计算机技术与发展,2014,24(07):1.
　ZHANG Zhi-hong,WU Qing-bo,SHAO Li-song,et al. Design and Implementation of TCP/IP Offload Engine Protocol Stack Based on FT Platform[J].,2014,24(07):1.
[9]梁文快,李毅. 改进的基因表达算法对航班优化排序问题研究[J].计算机技术与发展,2014,24(07):5.
　LIANG Wen-kuai,LI Yi. Research on Optimization of Flight Scheduling Problem Based on Improved Gene Expression Algorithm[J].,2014,24(07):5.
[10]黄静,王枫,谢志新,等. EAST文档管理系统的设计与实现[J].计算机技术与发展,2014,24(07):13.
　HUANG Jing,WANG Feng,XIE Zhi-xin,et al. Design and Implementation of EAST Document Management System[J].,2014,24(07):13.
[11]王晓军,邹亮亮. Hadoop迭代优化技术的研究[J].计算机技术与发展,2014,24(09):98.
　WANG Xiao-jun,ZOU Liang-liang. Research on Optimizing Iterative Technology of Hadoop[J].,2014,24(07):98.
[12]徐源吾[][],王珣[][]. 基于Hadoop的智能家居信息处理平台[J].计算机技术与发展,2014,24(09):183.
　XU Yuan-wu[] [],WANG Xun[][]. nformation Processing Platform of Smart Home Based on Hadoop[J].,2014,24(07):183.
[13]孙媛,黄刚. 基于Hadoop平台的C4.5算法的分析与研究[J].计算机技术与发展,2014,24(11):83.
　SUN Yuan,HUANG Gang. Analysis and Study of C4 . 5 Algorithm Based on Hadoop Platform[J].,2014,24(07):83.
[14]王全民,苗雨,何明,等. 基于矩阵分解的协同过滤算法的并行化研究[J].计算机技术与发展,2015,25(02):55.
　ANG Quan-min,MIAO Yu,HE Ming,et al. Parallelized Research on Collaborative Filtering Algorithm Based on Matrix Factorization[J].,2015,25(07):55.
[15]方木云,刘洪彬,谢恩文. Hadoop下基于边聚类的重叠社区发现算法研究[J].计算机技术与发展,2015,25(03):58.
　FANG Mu-yun,LIU Hong-bin,XIE En-wen. Research on Overlapping Communities Detecting Algorithm Using Hadoop Based on Edge Clustering[J].,2015,25(07):58.
[16]秦军[],童毅[],戴新华[],等. 基于MapReduce数据密集型负载调度策略研究[J].计算机技术与发展,2015,25(04):48.
　QIN Jun[],TONG Yi[],DAI Xin-hua[],et al. Research on Scheduling Strategy of Data Intensive Workloads Based on MapReduce[J].,2015,25(07):48.
[17]徐新瑞,孟彩霞,周雯,等. 一种基于Spark时效化协同过滤推荐算法[J].计算机技术与发展,2015,25(06):48.
　XU Xin-rui,MENG Cai-xia,ZHOU Wen,et al. A Real-time Collaborative Filtering Recommendation Algorithm Based on Spark[J].,2015,25(07):48.
[18]李晨,杨子江,朱世伟,等. 基于Hadoop的网络舆情监控平台设计与实现[J].计算机技术与发展,2016,26(02):144.
　LI Chen,YANG Zi-jiang,ZHU Shi-wei,et al. Design and Implementation of Network Consensus Monitoring System Based on Hadoop[J].,2016,26(07):144.
[19]马腾腾[],朱庆华[],曹菡[],等. 基于Hadoop的旅游景点推荐的算法实现与应用[J].计算机技术与发展,2016,26(03):47.
　MA Teng-teng[],ZHU Qing-hua[],CAO Han[],et al. Implementation and Application of Algorithm of Tourist Attractions Recommendation Based on Hadoop[J].,2016,26(07):47.
[20]李正杰,黄刚. 基于Hadoop平台的SVM KNN分类算法的研究[J].计算机技术与发展,2016,26(03):75.
　LI Zheng-jie,HUANG Gang. Research on SVM KNN Classification Algorithm Based on Hadoop Platform[J].,2016,26(07):75.

更新日期/Last Update: 2016-09-28