[1]李一波,郭培宜,张森悦.深度卷积神经网络中激活函数的研究[J].计算机技术与发展,2021,31(09):61-66.[doi:10. 3969 / j. issn. 1673-629X. 2021. 09. 011]
　LI Yi-bo,GUO Pei-yi,ZHANG Sen-yue.Research on Activation Function in Deep Convolutional Neural Network[J].,2021,31(09):61-66.[doi:10. 3969 / j. issn. 1673-629X. 2021. 09. 011]
点击复制

深度卷积神经网络中激活函数的研究()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
31
期数:
2021年09期
页码:
61-66
栏目:
图形与图像
出版日期:
2021-09-10

文章信息/Info

Title:
Research on Activation Function in Deep Convolutional Neural Network
文章编号:
1673-629X(2021)09-0061-06
作者:
李一波; 郭培宜; 张森悦
沈阳航空航天大学 自动化学院,辽宁 沈阳 110000
Author(s):
LI Yi-bo; GUO Pei-yi; ZHANG Sen-yue
School of Automation Institute,Shenyang Aerospace University,Shenyang 110000,China
关键词:
深度卷积神经网络; 激活函数; 反正切函数; 对数函数; 图像分类
Keywords:
deep convolutional neural network; activation function; arctangent function; logarithmic function; image classification
分类号:
TP183
DOI:
10. 3969 / j. issn. 1673-629X. 2021. 09. 011
摘要:
针对深度卷积神经网络中经典的 AlexNet 网络模型中激活函数 ReLU 在网络模型训练时易产生神经元“死亡” 和均值偏移的问题进行研究以及改进,通过结合反正切函数和对数函数的优势,在传统激活函数 ReLU 基础上提出了一种新的激活函数 sArcReLU,并在后续训练过程中进一步调参。 并将文中改进后的激活函数 sArcReLU 用于 AlexNet 网络模型训练,将使用新激活函数训练的深度卷积神经网络模型应用于公开数据集进行分类实验以验证其性能。 实验结果表明:利用 sArcReLU 激活函数训练的深度卷积神经网络比利用 ReLU 以及 ArcReLU 训练的网络模型在分类精度上分别提升了1.7% 和 2.4% ,证明了改进方式经过大量数据充分微调的深度卷积神经网络可有效地提高图像分类精度,该方法同时也提升了深度卷积神经网络的实际应用价值。
Abstract:
The problem that the activation function ReLU in the classic AlexNet network model of deep convolutional neural network is prone to " death" of neurons and mean shift when the network model is trained is studied and improved. By combining the advantages of arctangent function and logarithmic function,we propose a new activation function sArcReLU based on the traditional activation function ReLU and further adjust the parameters in the subsequent training process. The improved activation function sArcReLU is used for AlexNet network model training,and the deep convolutional neural network model trained with the new activation function is applied to the public data set for classification experiments to verify its performance.? ?The experiment shows that the deep convolutional neural network trained with the sArcReLU activation function improves the classification accuracy by 1.7% and 2.4% ,respectively,than the network model trained with ReLU and ArcReLU,which proves that the proposed improved method is fully fine-tuned by a large amount of data. Neural network can effectively improve the accuracy of image classification, and this method also improves the practical application value of deep convolutional neural networks.

相似文献/References:

[1]宦如松,于舒娟,张昀.双Sigmoid Hopfield神经网络盲检测算法[J].计算机技术与发展,2014,24(04):100.
　HUAN Ru-song,YU Shu-juan,ZHANG Yun.Blind Detection Algorithm with Double Sigmoid Hopfield Neural Network[J].,2014,24(09):100.
[2]雷 飞,孙 康,王雪丽.基于改进 LeNet-5 的牛奶生产日期识别研究[J].计算机技术与发展,2020,30(07):96.[doi:10. 3969 / j. issn. 1673-629X. 2020. 07. 021]
　LEI Fei,SUN Kang,WANG Xue-li.Research on Milk Production Date Recognition Based on Improved LeNet-5[J].,2020,30(09):96.[doi:10. 3969 / j. issn. 1673-629X. 2020. 07. 021]
[3]曲之琳,胡晓飞.基于改进激活函数的卷积神经网络研究[J].计算机技术与发展,2017,27(12):77.[doi:10.3969/ j. issn.1673-629X.2017.12.017]
　QU Zhi-lin,HU Xiao-fei.Research on Convolutional Neural Network Based on Improved Activation Function[J].,2017,27(09):77.[doi:10.3969/ j. issn.1673-629X.2017.12.017]
[4]高友文,周本君,胡晓飞.基于数据增强的卷积神经网络图像识别研究[J].计算机技术与发展,2018,28(08):62.[doi:10.3969/ j. issn.1673-629X.2018.08.013]
　GAO You-wen,ZHOU Ben-jun,HU Xiao-fei.Research on Image Recognition of Convolution Neural Network Based on Data Enhancement[J].,2018,28(09):62.[doi:10.3969/ j. issn.1673-629X.2018.08.013]
[5]邓烜堃,万 良,马彦勤.深度稀疏修正神经网络在股票预测中的应用[J].计算机技术与发展,2018,28(09):199.[doi:10.3969/ j. issn.1673-629X.2018.09.041]
　DENG Xuan-kun,WAN Liang,MA Yan-qin.Application of Deep Sparse Modified Neural Network in Stock Forecasting[J].,2018,28(09):199.[doi:10.3969/ j. issn.1673-629X.2018.09.041]
[6]陈 辉,刘立群*.改进哈里斯鹰优化 PCNN 参数的图像融合应用[J].计算机技术与发展,2023,33(04):168.[doi:10. 3969 / j. issn. 1673-629X. 2023. 04. 025]
　CHEN Hui,LIU Li-qun.Image Fusion Application of Improved Harris Hawk Optimizing for PCNN Parameters[J].,2023,33(09):168.[doi:10. 3969 / j. issn. 1673-629X. 2023. 04. 025]

更新日期/Last Update: 2021-09-10