[1]邢艳芳,周舒琪.基于 Hadoop 的分布式视频转码系统研究[J].计算机技术与发展,2022,32(05):58-62.[doi:10. 3969 / j. issn. 1673-629X. 2022. 05. 010]
　XING Yan-fang,ZHOU Shu-qi.Research on Distributed Video Transcoding System Based on Hadoop[J].,2022,32(05):58-62.[doi:10. 3969 / j. issn. 1673-629X. 2022. 05. 010]
点击复制

基于 Hadoop 的分布式视频转码系统研究()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
32
期数:
2022年05期
页码:
58-62
栏目:
系统工程
出版日期:
2022-05-10

文章信息/Info

Title:
Research on Distributed Video Transcoding System Based on Hadoop
文章编号:
1673-629X(2022)05-0058-05
作者:
邢艳芳; 周舒琪
南京传媒学院,江苏 南京 211172
Author(s):
XING Yan-fang; ZHOU Shu-qi
Communication University of China,Nanjing 211172,China
关键词:
Hadoop; 分布式处理; 视频转码; HDFS; FFmpeg
Keywords:
Hadoop; distributed processing; video transcoding; HDFS; FFmpeg
分类号:
TP391
DOI:
10. 3969 / j. issn. 1673-629X. 2022. 05. 010
摘要:
当下由于视频内容多样化的爆发式变革,产生了多种音视频封装格式和编码格式,为解决用户高清视频多端下载收看及相应格式转换需求,应对庞大的数据量计算作业,需整合高效计算机资源。 该文提出了一种基于 Hadoop 的分布式视频转码方案,采用分布式文件存储系统 HDFS 进行大型视频文件的存储,通过 MapReduce 编程框架结合 FFmpeg 开源软件,将视频数据处理划分为 Map 和 Reduce 两个阶段,把庞大的数据量分布到多处理节点分析。 调用转码模块,减少开发人员工作量,分布式完成视频转码功能。 该方案充分利用了数据集群的并行计算能力,突破了单机视频转码技术的发展瓶颈。 通过实验验证得出,相比于单一节点进行视频转码,此系统的转码速度仅在 2 台数据节点的分布式集群中就获得了 50% 的提升。 通过此系统可以为各类终端用户按各自需求提供易于使用、开放便捷、快速高效的视频转码服务。
Abstract:
At present,due to the diversified and explosive changes of video content,a variety of audio and video packaging formats and encoding formats have been produced. In order to meet the needs of users for multi-end downloading and viewing of high -definition videos and corresponding format conversion requirements,it is necessary to deal with huge data volume calculation tasks and integrate efficient computer resource assistance. We propose a Hadoop-based distributed video transcoding scheme,which uses the distributed file storage system HDFS to store large video files. Through the Map Reduce programming framework combined with the FFmpeg opensource software,the video data processing? is divided into two stages:Map and Reduce. Distribute the huge amount of data to multi -processing nodes for analysis. Call the transcoding module to reduce the workload of developers and complete the video transcoding function in a distributed manner. This solution makes full use? of the parallel computing capabilities of the data cluster and breaks the bottleneck of the development of stand-alone video transcoding technology. Experimental verification shows that compared to a single node for video transcoding,the transcoding speed of this system is only 50% improved in a distributed cluster of 2 data nodes. Through this system,it is possible to provide various end-users with easy-to-use, open and convenient,fast and efficient video trans coding services according to their needs.

相似文献/References:

[1]夏奇思 王汝传.基于属性约简的粗糙集海量数据分割算法研究[J].计算机技术与发展,2010,(04):5.
　XIA Qi-si,WANG Ru-chuan.Mass Data Partition for Rough Set on Attribute Reduction Algorithm[J].,2010,(05):5.
[2]李远方 邓世昆 闻玉彪 韩月阳.Hadoop-MapReduce下的PageRank矩阵分块算法[J].计算机技术与发展,2011,(08):6.
　LI Yuan-fang,DENG Shi-kun,WEN Yu-biao,et al.PageRank Matrix Partitioned Algorithm Using Hadoop-MapReduce[J].,2011,(05):6.
[3]李远方 贾时银 邓世昆 韩月阳.基于树结构的MapReduce模型[J].计算机技术与发展,2011,(08):149.
　LI Yuan-fang,JIA Shi-yin,DENG Shi-kun,et al.MapReduce Model Based on Tree Structure[J].,2011,(05):149.
[4]王梅,朱信忠,赵建民,等.基于 Hadoop 的海量图像检索系统[J].计算机技术与发展,2013,(01):204.
　WANG Mei,ZHU Xin-zhong,ZHAO Jian-min,et al.Massive Images Retrieval System Based on Hadoop[J].,2013,(05):204.
[5]王晓军,孙惠.基于MapReduce的多路连接优化方法研究[J].计算机技术与发展,2013,(06):59.
　WANG Xiao-jun,SUN Hui.Research of Optimizing Multiway Joins Based on MapReduce[J].,2013,(05):59.
[6]朱贤军,李敬兆.无加密模式下对云数据的隐私保密[J].计算机技术与发展,2013,(06):216.
　ZHU Xian-jun,LI Jing-zhao.Cloud Data Privacy under None Encryption[J].,2013,(05):216.
[7]周婷,张君瑛,罗成.基于Hadoop的K-means聚类算法的实现[J].计算机技术与发展,2013,(07):18.
　ZHOU Ting[],ZHANG Jun-ying[],LUO Cheng[].Realization of K-means Clustering Algorithm Based on Hadoop[J].,2013,(05):18.
[8]吕婉琪,钟诚,唐印浒,等.Hadoop分布式架构下大数据集的并行挖掘[J].计算机技术与发展,2014,24(01):22.
　L Wan-qi,ZHONG Cheng,TANG Yin-hu,et al.Parallel Mining of Large Dataset in Hadoop Distributed Computing Framework[J].,2014,24(05):22.
[9]王晓军,邹亮亮. Hadoop迭代优化技术的研究[J].计算机技术与发展,2014,24(09):98.
　WANG Xiao-jun,ZOU Liang-liang. Research on Optimizing Iterative Technology of Hadoop[J].,2014,24(05):98.
[10]徐源吾[][],王珣[][]. 基于Hadoop的智能家居信息处理平台[J].计算机技术与发展,2014,24(09):183.
　XU Yuan-wu[] [],WANG Xun[][]. nformation Processing Platform of Smart Home Based on Hadoop[J].,2014,24(05):183.

更新日期/Last Update: 2022-05-10