[1]王永洲[],茅苏[].HDFS中的一种数据放置策略[J].计算机技术与发展,2013,(05):90-92.
　WANG Yong-zhou,MAO Su.A Blocks Placement Strategy in HDFS[J].,2013,(05):90-92.
点击复制

HDFS中的一种数据放置策略()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2013年05期
页码:
90-92
栏目:
智能、算法、系统工程
出版日期:
1900-01-01

文章信息/Info

Title:
A Blocks Placement Strategy in HDFS
文章编号:
1673-629X（2013）05-0090-03
作者:
王永洲[1; 2; 3]; 茅苏[1; 2; 3]
[1]南京邮电大学 计算机学院;[2]江苏省无线传感网高技术研究重点实验室;[3]宽带无线通信与传感网技术教育部重点实验室
Author(s):
WANG Yong-zhou; MAO Su
关键词:
HDFS; 云存储; 数据放置; 吞吐量
Keywords:
HDFS; cloud storage; data placement; throughput
文献标志码:
A
摘要:
将HDFS应用于云存储备份系统已引起学术界和企业界的广泛关注,但是HDFS假设集群中的节点是同构的,且在数据块的放置时采用了随机的数据放置策略,而在实际的云存储系统中节点的同构性并不理想并且随系统的运行节点的存储性能动态变化,随机选取节点的数据放置策略也可能会导致集群中数据放置的不均衡性.为此,文中综合考察了影响节点存储性能的CPU、内存、系统结构、磁盘的读写速度等因素,提出了一种对节点存储性能进行度量的方法.在数据存储时综合考虑节点的存储性能和节点与客户端的网络距离两个因素进行数据节点的选择.实验结果表明,与系统默认策略相比提出的策略不仅可以改进数据放置的负载均衡而且可以提高异构集群的数据传输速率
Abstract:
The application of HDFS in cloud storage has caused wide attention in academic circles and corporate community,but HDFS assumes that the nodes in a cluster are homogeneous and the data node selection for block placement in cluster is random. The current strategy ignores the heterogeneity of cluster and could lead to unbalanced load between data nodes. In this paper,put forward a method for measuring the storage performance of the data node which takes CPU,internal storage,system structure and disk speed for account. The new strategy places data according to the storage performance and the network distance between client and data node. The experimantal results indicate that the strategy can not only behave better in load balancing but also improve the data transmission speed

相似文献/References:

[1]王德政 申山宏 周宁宁.云计算环境下的数据存储[J].计算机技术与发展,2011,(04):81.
　WANG De-zheng,SHEN Shan-hong,ZHOU Ning-ning.Data Storage in Cloud Computing Environment[J].,2011,(05):81.
[2]秦军,邓谦,张建平.HDFS的多安全级数据销毁机制设计[J].计算机技术与发展,2013,(03):129.
　QIN Jun,DENG Qian,ZHANG Jian-ping.Design of Multi-grade Safety Data Destruction Mechanism of HDFS[J].,2013,(05):129.
[3]卞艺杰,马玲玲.云环境下移动视频监控系统安全性研究[J].计算机技术与发展,2013,(09):119.
　BIAN Yi-jie,MA Ling-ling.Research on Security of Mobile Video Surveillance System under Cloud Environment[J].,2013,(05):119.
[4]李东阳,刘鹏,田浪军.基于SSD的云存储主服务器元数据管理研究[J].计算机技术与发展,2013,(10):68.
　LI Dong-yang,LIU Peng,TIAN Lang-jun.Research on Master Server Metadata Management in Cloud Storage System Based on SSD[J].,2013,(05):68.
[5]付雄[],徐松[],周代明[]. 云计算环境下基于信任的访问控制模型研究[J].计算机技术与发展,2015,25(09):139.
　FU Xiong[],XU Song[],ZHOU Dai-ming[]. Research on Trust-based Access Control Model in Cloud Computing Environment[J].,2015,25(05):139.
[6]王凤领. 基于Hadoop高校教育资源云存储平台构建研究[J].计算机技术与发展,2016,26(03):176.
　WANG Feng-ling. Study on Construction of Cloud Storage Platform for College Education Resources Based on Hadoop[J].,2016,26(05):176.
[7]庞立君,廖春伟,黄波,等. 基于GID的车联网数据安全方案[J].计算机技术与发展,2016,26(04):101.
　PANG Li-jun,LIAO Chun-wei,HUANG Bo,et al. Data Security Scheme of IOV Based on GID[J].,2016,26(05):101.
[8]王法,谭郁松,伍复慧,等. 基于云存储视频处理框架的研究与实现[J].计算机技术与发展,2016,26(05):1.
　WANG Fa,TAN Yu-song,WU Fu-hui,et al. Research and Implementation of Video Processing Framework Based on Cloud Storage[J].,2016,26(05):1.
[9]范素娟[],田军锋[]. 基于Hadoop的云计算平台研究与实现[J].计算机技术与发展,2016,26(07):127.
　FAN Su-juan[],TIAN Jun-feng[]. Research and Implementation of Cloud Computing Platform Based on Hadoop[J].,2016,26(05):127.
[10]周晓梅[],朱金付[],段洪秀[]. LeanCloud在图片管理系统中的应用研究[J].计算机技术与发展,2016,26(07):138.
　ZHOU Xiao-mei[],ZHU Jin-fu[],DUAN Hong-xiu[]. Application Study of LeanClound in Picture Management System[J].,2016,26(05):138.

更新日期/Last Update: 1900-01-01