[1]李相桥,李晨,田丽华,等.卷积神经网络并行训练的优化研究[J].计算机技术与发展,2018,28(08):12-16.[doi:10.3969/ j. issn.1673-629X.2018.08.003]
　LI Xiang-qiao,LI Chen,TIAN Li-hua,et al.Research on Optimization of Parallel Training for Convolution Neural Network[J].,2018,28(08):12-16.[doi:10.3969/ j. issn.1673-629X.2018.08.003]
点击复制

卷积神经网络并行训练的优化研究()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
28
期数:
2018年08期
页码:
12-16
栏目:
智能、算法、系统工程
出版日期:
2018-08-10

文章信息/Info

Title:
Research on Optimization of Parallel Training for Convolution Neural Network
文章编号:
1673-629X(2018)08-0012-05
作者:
李相桥1  2 ; 李晨1 ; 田丽华1 ; 张玉龙1
1. 西安交通大学 软件学院,陕西 西安 710049;
2. 中航工业西安飞行自动控制研究所,陕西 西安 710065
Author(s):
LI Xiang-qiao 1  2 ; LI Chen 1 ; TIAN Li-hua 1 ; ZHANG Yu-long 1
1. School of Software Engineering,Xi’an Jiaotong University,Xi’an 710049,China;
2. AVIC Xi’an Flight Automatic Control Research Institute,Xi’an 710065,China
关键词:
卷积神经网络; 数据并行; 通讯优化; 数据加载优化
Keywords:
convolution neural network; data parallelism; communication optimization; data load optimization
分类号:
TP183
DOI:
10.3969/ j. issn.1673-629X.2018.08.003
文献标志码:
A
摘要:
由于卷积神经网络模型结构复杂且计算量大,实际应用中一般采用多 GPU 的方式对其进行并行训练,快速地完成卷积神经网络的快速训练。 为了提高卷积神经网络的并行训练效率,同时解决在并行训练时通讯缓慢以及数据加载时等待的问题,提出参数通讯以及数据加载两个方面的优化策略。 在参数通讯优化方面,将梯度计算和参数通讯同时执行,利用计算时间来覆盖通讯时间。 通过改变通讯方式,利用归约和冗余通讯方式减少参数通讯时的同步等待时间。 利用预加载和异步拷贝的方式将数据提前加载并拷贝到 GPU 显存空间,减少数据加载带来的时间消耗。 实验结果表明,优化后的方法能够有效地提高卷积神经网络的并行训练效率。
Abstract:
As the model of convolution neural network is complex in structure and large in computation,the parallel training is generally carried out in the way of multi-GPU in practical application,and the rapid training of convolution neural network is completed quickly.In order to improve the parallel training efficiency of convolution neural network and solve the problem of slow communication and the waiting caused by data loading in parallel training,we propose two aspects of optimization strategy in parameter communication and data loading. In the aspect of the parameter communication optimization,the gradient calculation and the parameter communication are executed simultaneously,and the computation time is utilized to cover the communication time. By changing the communication mode,the synchronous waiting time of communication is reduced by means of reduction and redundancy communication. Using preload and asynchronous copying,the data is loaded and copied to the GPU memory space in advance to reduce the time consumption of data loading.The experiment shows that the optimized method can effectively improve the parallel training efficiency of convolution neural network.

相似文献/References:

[1]崔凤焦.表情识别算法研究进展与性能比较[J].计算机技术与发展,2018,28(02):145.[doi:10．3969/j．issn．1673－629X．2018．02．031]
　CUI Feng-jiao.Ｒesearch and Performance Comparison of Facial Expression Ｒecognition Algorithm[J].,2018,28(08):145.[doi:10．3969/j．issn．1673－629X．2018．02．031]
[2]张丹丹,李雷. 基于PCANet-RF的人脸检测系统[J].计算机技术与发展,2016,26(02):31.
　ZHANG Dan-dan,LI Lei. Face Detection System Based on PCANet-RF[J].,2016,26(08):31.
[3]陈强锐,谢世朋.基于深度学习的肺部肿瘤检测方法[J].计算机技术与发展,2018,28(04):201.[doi:10.3969/ j. issn.1673-629X.2018.04.043]
　CHEN Qiang-rui,XIE Shi-peng.Lung Cancer Detection Method Based on Deep Learning[J].,2018,28(08):201.[doi:10.3969/ j. issn.1673-629X.2018.04.043]
[4]郭子琰,舒心,刘常燕,等.基于ReLU 函数的卷积神经网络的花卉识别算法[J].计算机技术与发展,2018,28(05):154.[doi:10．3969/j．issn．1673－629X．2018．05．035]
　GUO Ziyan,SHU Xin,LIU Changyan,et al.A Recognition Algorithm of Flower Based on Convolution Neural Network with ReLU Function[J].,2018,28(08):154.[doi:10．3969/j．issn．1673－629X．2018．05．035]
[5]缪宇杰,吴智钧,宫 婧.基于3D 卷积的视频错帧筛选方法[J].计算机技术与发展,2018,28(05):179.[doi:10.3969/ j. issn.1673-629X.2018.05.040]
　MIAO Yu-jie,WU Zhi-jun,GONG Jing.A Wrong Temporal-order Frames Identification Method Based on 3D Convolution[J].,2018,28(08):179.[doi:10.3969/ j. issn.1673-629X.2018.05.040]
[6]吴玉枝,吴志红,熊运余.基于卷积神经网络的小样本车辆检测与识别[J].计算机技术与发展,2018,28(06):1.[doi:10.3969/ j. issn.1673-629X.2018.06.001]
　WU Yu-zhi,WU Zhi-hong,XIONG Yun-yu.Vehicle Detection and Recognition of a Few Samples Based on Convolutional Neural Network[J].,2018,28(08):1.[doi:10.3969/ j. issn.1673-629X.2018.06.001]
[7]邓宗平,赵启军,陈虎. 基于深度学习的人脸姿态分类方法[J].计算机技术与发展,2016,26(07):11.
　DEND Zong-ping,ZHAO Qi-jun,CHEN Hu. Face Pose Classification Method Based on Deep Learning[J].,2016,26(08):11.
[8]河海大学 计算机与信息学院,江苏 南京 0098.卷积网络的无监督特征提取对人脸识别的研究[J].计算机技术与发展,2018,28(06):17.[doi:10.3969/ j. issn.1673-629X.2018.06.004]
　DU Bai-sheng.Research on Unsupervised Feature Extraction Based on Convolutional Neural Network for Face Recognition[J].,2018,28(08):17.[doi:10.3969/ j. issn.1673-629X.2018.06.004]
[9]高翔,陈志,岳文静,等.基于视频场景深度学习的人物语义识别模型[J].计算机技术与发展,2018,28(06):53.[doi:10.3969/ j. issn.1673-629X.2018.06.012]
　GAO Xiang,CHEN Zhi,YUE Wen-jing,et al.Human Semantic Recognition Model Based on Video Scene Deep Learning[J].,2018,28(08):53.[doi:10.3969/ j. issn.1673-629X.2018.06.012]
[10]戴晓薇,赵启军. 基于回归的指纹方向场估计[J].计算机技术与发展,2017,27(01):1.
　DAI Xiao-wei,ZHAO Qi-jun. Fingerprint Orientation Field Estimation Based on Regression[J].,2017,27(08):1.

更新日期/Last Update: 2018-10-15