[1]张祥俊,伍卫国.基于FastDFS的数字媒体系统设计与实现技术研究[J].计算机技术与发展,2019,29(05):6-11.[doi:10. 3969 / j. issn. 1673-629X. 2019. 05. 002]
　ZHANG Xiang-jun,WU Wei-guo.Research on Design and Implementation of Digital Media System Based on FastDFS[J].,2019,29(05):6-11.[doi:10. 3969 / j. issn. 1673-629X. 2019. 05. 002]
点击复制

基于FastDFS的数字媒体系统设计与实现技术研究()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
29
期数:
2019年05期
页码:
6-11
栏目:
智能、算法、系统工程
出版日期:
2019-05-10

文章信息/Info

Title:
Research on Design and Implementation of Digital Media System Based on FastDFS
文章编号:
1673-629X(2019)05-0006-06
作者:
张祥俊1; 伍卫国2
1. 西安交通大学 软件学院,陕西 西安 710000;2. 西安交通大学 电信学院,陕西 西安 710000
Author(s):
ZHANG Xiang-jun1; WU Wei-guo2
1. School of Software,Xi’an Jiaotong University,Xi’an 710000,China;2. School of Electronics and Information Engineering,Xi’an Jiaotong University,Xi’an 710000,China
关键词:
FastDFS; Docker; 媒体资产管理; 分布式存储; 数字媒体制作
Keywords:
FastDFS;  Docker; media asset management; distributed storage; digital media production
分类号:
TP302. 1
DOI:
10. 3969 / j. issn. 1673-629X. 2019. 05. 002
摘要:
海量的数据存储一般采用分布式文件系统 HDFS,Hbase 等作为存储工具,但对海量媒体资产数据这种中小文件的存储,存在性能瓶颈,并且随着后期小文件越来越多,会造成系统元数据占用过大。 对于海量媒体资产管理而言,检索是关键,设计良好的查询体系很重要。 结合国家重点研发课题海量数字媒体与应用的实际应用需求,通过分析分布式文件系统 FastDFS 的原理与缺陷,提出了一种基于容器技术、适合于存储海量中小型文件的分布式存储系统,同时设计了集群服务器之间的同步目录组方案,确保在宕机的情况下,用户的数据不会丢失,且 Docker 支持多租户,确保了不同租户数据的隔离性。 最后对设计实现的原型系统进行了测试与分析,测试结果表明,基于 FastDFS 的数字媒体资产在线管理系统符合预期设计,性能和可靠性较好,可适用于超算环境下各种渲染任务的使用。
Abstract:
Distributed file systems such as HDFS and Hbase are commonly used as storage tools to store massive data. However,there exists bottlenecks in the storage of medium and small files for massive media asset data,and as small files increase,system metadata may occupy too much space. For mass media asset management,retrieval is the key,and well - designed query system is very important. Combined with actual application requirements of huge amounts of digital media and application of the national key project,based on the analysis of the principles and defects of distributed file system FastDFS,we propose a distributed storage system based on container technology which is suitable for storing large and medium- sized files. At the same time,we design a synchronized directory group scheme between cluster servers to ensure that users爷 data will not be lost in case of downtime. Docker supports multi-tenancy and can ensure the isolation of different tenant data. Finally,the prototype system designed and implemented is tested and analyzed. The test results show that the digital media asset online management system based on FastDFS is in line with the expected design with better performance and reliability,which is applicable for various rendering tasks in a supercomputing environment.

相似文献/References:

[1]鲁涛,陈杰,史军.Docker 安全性研究[J].计算机技术与发展,2018,28(06):115.[doi:10.3969/ j. issn.1673-629X.2018.06.026]
　LU Tao,CHEN Jie,SHI Jun.Research of Docker Security[J].,2018,28(05):115.[doi:10.3969/ j. issn.1673-629X.2018.06.026]
[2]赵乐乐,黄刚,马越. 基于Docker的Hadoop平台架构研究[J].计算机技术与发展,2016,26(09):99.
　ZHAO Le-le,HUANG Gang,MA Yue. Research on Hadoop Platform Based on Docker[J].,2016,26(05):99.
[3]张军,李泽平,王苍灵. 基于组合定权法的负载评价模型研究[J].计算机技术与发展,2017,27(07):10.
　ZHANG Jun,LI Ze-ping,WANG Cang-ling. Investigation on Load Evaluation Model with Combination Weighting Method[J].,2017,27(05):10.
[4]蒋园,阳许军.基于人脸识别的海量图片的存储和索引优化[J].计算机技术与发展,2019,29(03):85.[doi:10.3969/ j. issn.1673-629X.2019.03.018]
　JIANG Yuan,YANG Xu-jun.Storage and Index Optimization of Massive Images Based on Facial Recognition[J].,2019,29(05):85.[doi:10.3969/ j. issn.1673-629X.2019.03.018]
[5]刘钱超,董超群,张垚.基于容器技术的软件测试优化研究[J].计算机技术与发展,2019,29(04):13.[doi:10. 3969 / j. issn. 1673-629X. 2019. 04. 003]
　LIU Qian-chao,DONG Chao-qun,ZHANG Yao.Research on Optimization of Software Testing Based on Container Technology[J].,2019,29(05):13.[doi:10. 3969 / j. issn. 1673-629X. 2019. 04. 003]
[6]郝慧杰,肖 建,张 粮,等.VNF 生命周期管理系统设计与实现[J].计算机技术与发展,2020,30(07):12.[doi:10. 3969 / j. issn. 1673-629X. 2020. 07. 003]
　HAO Hui-jie,XIAO Jian,ZHANG Liang,et al.Design and Implementation of VNF Life Cycle Management System[J].,2020,30(05):12.[doi:10. 3969 / j. issn. 1673-629X. 2020. 07. 003]
[7]赵旭杰,粱正和.Kubernetes 可视化管理平台[J].计算机技术与发展,2021,31(02):106.[doi:10. 3969 / j. issn. 1673-629X. 2021. 02. 020]
　ZHAO Xu-jie,LIANG Zheng-he.Kubernetes Visual Management Platform[J].,2021,31(05):106.[doi:10. 3969 / j. issn. 1673-629X. 2021. 02. 020]
[8]万嘉龙,况立群*,熊风光,等.面向 Spring 的热点代码在线部署方法研究[J].计算机技术与发展,2023,33(05):105.[doi:10. 3969 / j. issn. 1673-629X. 2023. 05. 016]
　WAN Jia-long,KUANG Li-qun*,XIONG Feng-guang,et al.Research on Hot Code Online Deployment Method for Spring[J].,2023,33(05):105.[doi:10. 3969 / j. issn. 1673-629X. 2023. 05. 016]

更新日期/Last Update: 2019-05-10