[1]张佳钰,寇金桥,刘宁钟.基于滤波器分布拟合的神经网络剪枝算法[J].计算机技术与发展,2022,32(12):136-141.[doi:10. 3969 / j. issn. 1673-629X. 2022. 12. 021]
　ZHANG Jia-yu,KOU Jin-qiao,LIU Ning-zhong.Deep Convolutional Neural Networks Pruning Algorithm Based on Filter Pruning via Distribution Fitting[J].,2022,32(12):136-141.[doi:10. 3969 / j. issn. 1673-629X. 2022. 12. 021]
点击复制

基于滤波器分布拟合的神经网络剪枝算法()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
32
期数:
2022年12期
页码:
136-141
栏目:
人工智能
出版日期:
2022-12-10

文章信息/Info

Title:
Deep Convolutional Neural Networks Pruning Algorithm Based on Filter Pruning via Distribution Fitting
文章编号:
1673-629X(2022)12-0136-06
作者:
张佳钰1 ; 寇金桥2 ; 刘宁钟1
1. 南京航空航天大学 计算机科学与技术学院,江苏 南京 211106;
2. 北京计算机技术及应用研究所 方舟重点实验室,北京 100854
Author(s):
ZHANG Jia-yu1 ; KOU Jin-qiao2 ; LIU Ning-zhong1
1. School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China;
2. Fangzhou Key Laboratory,Beijing Institute of Computer Technology and Application,Beijing 100854,China
关键词:
深度学习; 模型压缩; 网络剪枝; 分布拟合; 滤波器剪枝
Keywords:
deep learning; model compression; network pruning; distribution fitting; filter pruning
分类号:
TP391. 4
DOI:
10. 3969 / j. issn. 1673-629X. 2022. 12. 021
摘要:
随着人工智能技术的迅猛发展,深度神经网络在不断地加深与变宽,模型的计算量快速增加,神经网络模型的高存储和高功耗的需求也随之产生。 网络剪枝是实现模型压缩和加速的一种有效方法。 常见的剪枝方法遵循“ 较小规范-不重要＂ 的标准来对滤波器进行修剪,认为范值较小的滤波器重要性较低,可以安全地修剪掉。 针对删去重要性较小的滤波器容易导致滤波器范数分布不均衡的问题,文中提出了一种拟合原始滤波器范数分布的剪枝算法。 该算法不仅可以筛选出拟合了原始范数分布的滤波器,还能删去冗余的滤波器。 实验表明该算法在两个数据集上的模型压缩效果均优于对比实验。 其中,在 CIFAR-10 数据集上压缩基于 ResNet110 的图像分类模型的效果明显,最终在减少了 62% 以上的 FLOPs的情况下,相对准确率仅降低了 0. 14% 。
Abstract:
With the rapid development of artificial intelligence technology,deep neural networks are constantly deepening and widening,and the computational amount of the model is increasing rapidly. Therefore,the demand of high storage and high power consumption ofthe neural network model is also generated. Network pruning is an effective way to achieve model compression and acceleration. Thefilters are usually pruned by following the " smaller norm-less important" criterion,where filters with smaller norm values are consideredless important and can be safely pruned out. And the deletion of filters with smaller importance easily leads to the problem of unbalanceddistribution of filter norms. In this regard,a pruning algorithm for fitting the original filter norm distribution is proposed,which not onlyretains the filters that can fit the distribution of filter weights of the original network, but also deletes the redundant filters. Theexperiments demonstrate that the proposed method outperforms the comparison experiments in terms of model compression on bothdatasets. Among them,the effect of compressing the ResNet110-based image classification model on the CIFAR-10 dataset is obvious,which ends up with a relative accuracy reduction of only 0. 14% with over 62% reduction in FLOPs.

相似文献/References:

[1]陈强锐,谢世朋.基于深度学习的肺部肿瘤检测方法[J].计算机技术与发展,2018,28(04):201.[doi:10.3969/ j. issn.1673-629X.2018.04.043]
　CHEN Qiang-rui,XIE Shi-peng.Lung Cancer Detection Method Based on Deep Learning[J].,2018,28(12):201.[doi:10.3969/ j. issn.1673-629X.2018.04.043]
[2]施泽浩,赵启军.基于全卷积网络的目标检测算法[J].计算机技术与发展,2018,28(05):55.[doi:10.3969/j.issn.1673－629X.2018.05.013]
　SHI Ze-hao,ZHAO Qi-jun.Object Detection Algorithm Based on Fully Convolutional Neural Network[J].,2018,28(12):55.[doi:10.3969/j.issn.1673－629X.2018.05.013]
[3]黄法秀,张世杰,吴志红,等.数据增广下的人脸识别研究[J].计算机技术与发展,2020,30(03):67.[doi:10. 3969 / j. issn. 1673-629X. 2020. 03. 013]
　HUANG Fa-xiu,ZHANG Shi-jie,WU Zhi-hong,et al.Research on Face Recognition Based on Data Augmentation[J].,2020,30(12):67.[doi:10. 3969 / j. issn. 1673-629X. 2020. 03. 013]
[4]陈浩翔,蔡建明,刘铿然,等. 手写数字深度特征学习与识别[J].计算机技术与发展,2016,26(07):19.
　CHEN Hao-xiang,CAI Jian-ming,LIU Keng-ran,et al. Deep Learning and Recognition of Handwritten Numeral Features[J].,2016,26(12):19.
[5]高翔,陈志,岳文静,等.基于视频场景深度学习的人物语义识别模型[J].计算机技术与发展,2018,28(06):53.[doi:10.3969/ j. issn.1673-629X.2018.06.012]
　GAO Xiang,CHEN Zhi,YUE Wen-jing,et al.Human Semantic Recognition Model Based on Video Scene Deep Learning[J].,2018,28(12):53.[doi:10.3969/ j. issn.1673-629X.2018.06.012]
[6]贺飞翔,赵启军. 基于深度学习的头部姿态估计[J].计算机技术与发展,2016,26(11):1.
　HE Fei-xiang,ZHAO Qi-jun. Head Pose Estimation Based on Deep Learning[J].,2016,26(12):1.
[7]徐 融,邱晓晖.一种改进的 YOLO V3 目标检测方法[J].计算机技术与发展,2020,30(07):30.[doi:10. 3969 / j. issn. 1673-629X. 2020. 07. 007]
　XU Rong,QIU Xiao-hui.An Improved YOLO V3 Object Detection[J].,2020,30(12):30.[doi:10. 3969 / j. issn. 1673-629X. 2020. 07. 007]
[8]曾志平[] [],萧海东[],张新鹏[]. 基于DBN的金融时序数据建模与决策[J].计算机技术与发展,2017,27(04):1.
　ZENG Zhi-ping[] [],XIAO Hai-dong[],ZHANG Xin-peng[]. Modeling and Decision-making of Financial Time Series Data with DBN[J].,2017,27(12):1.
[9]李全兵,文 钊*,田艳梅*,等.基于 WGAN 的音频关键词识别研究[J].计算机技术与发展,2021,31(08):26.[doi:10. 3969 / j. issn. 1673-629X. 2021. 08. 005]
　LI Quan-bing,WEN Zhao *,TIAN Yan-mei *,et al.Research on Audio Keywords Recognition Based on WassersteinGenerative Adversarial Network[J].,2021,31(12):26.[doi:10. 3969 / j. issn. 1673-629X. 2021. 08. 005]
[10]李宏林. 分析式纹理合成技术及其在深度学习的应用[J].计算机技术与发展,2017,27(11):7.
　LI Hong-lin. Analyzed Texture-synthesis Techniques and Their Applications in Deep Learning[J].,2017,27(12):7.
[11]陈莉君,李 卓.基于深度神经压缩的 YOLO 优化[J].计算机技术与发展,2019,29(12):72.[doi:10. 3969 / j. issn. 1673-629X. 2019. 12. 013]
　CHEN Li-jun,LI Zhuo.YOLO Optimization Based on Deep Neural Compression[J].,2019,29(12):72.[doi:10. 3969 / j. issn. 1673-629X. 2019. 12. 013]

更新日期/Last Update: 2022-12-10