[1]张亚飞.基于注意力的权重分配机制[J].计算机技术与发展,2020,30(09):49-53.[doi:10. 3969 / j. issn. 1673-629X. 2020. 09. 009]
　ZHANG Ya-fei.Attention-based Weight Allocation Mechanism[J].,2020,30(09):49-53.[doi:10. 3969 / j. issn. 1673-629X. 2020. 09. 009]
点击复制

基于注意力的权重分配机制()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
30
期数:
2020年09期
页码:
49-53
栏目:
智能、算法、系统工程
出版日期:
2020-09-10

文章信息/Info

Title:
Attention-based Weight Allocation Mechanism
文章编号:
1673-629X(2020)09-0049-05
作者:
张亚飞
中国石油大学(华东) 计算机科学与技术学院,山东 青岛 266580
Author(s):
ZHANG Ya-fei
School of Computer Science and Technology,China University of Petroleum,Qingdao 266580,China
关键词:
深度学习; 神经网络; 网络优化; 注意力机制; 先验知识
Keywords:
deep learning; neural networks; network optimization; attention mechanism; priori knowledge
分类号:
TP302. 1
DOI:
10. 3969 / j. issn. 1673-629X. 2020. 09. 009
摘要:
目前,基于神经网络的深度学习技术得到了飞速的发展,已经广泛应用于日常生活,如行人检测、车牌识别、人脸识别等。 理论上可以通过不断扩大神经网络规模来提高算法准确度,然而这种方法并不可行。 原因在于单纯扩大网络规模会导致过拟合。 为了解决这个问题,通过人的先验知识来指导神经网络结构的设计以及明确神经网络每一个模块需要学习的目标,进而通过明确的模块分工来提升神经网络性能。 受注意力机制和正则化方法的启发,提出了一个基于注意力机制的自适应权重分配算法,通过对神经网络各模块进行合理的权重分配,强调或者弱化某些输入数据对于下一步处理的贡献并以可微分的方式进行设计,完成一个端对端的神经网络。 实验结果显示相比于其他方法,该算法达到了更好的效果。
Abstract:
Recently,the deep learning technology based on neural network has been developed rapidly and widely used in daily life,such as pedestrian detection, license plate recognition, face recognition and so on. Theoretically, the neural networks have the ability to approximate any complicated functions abstracting from real world. However,in practice,it is impossible to reach this target. The reason is that enlarging networks would cause over-fitting. To tackle this question,some researches design the neural networks structure guided by person’s priori knowledge to narrow down the function of every neural network module for increasing its performance. Inspired by attention mechanism and regularization methods,we propose an attention - based weight allocation mechanism to optimize the network structure. This method emphasizes or reduces contributions of some input data by distributing weights into different neural networks modules,which is designed as a differentiable end-to-end neural network. In a number of experiments on citation networks and on some public datasets,we demonstrate that the proposed algorithm has a better quality and outperforms other methods.

相似文献/References:

[1]路川 胡欣杰.区域航空市场航线客流量预测研究[J].计算机技术与发展,2010,(04):84.
　LU Chuan,HU Xin-jie.Analysis of Regional Airline Passenger Forecast Title[J].,2010,(09):84.
[2]高峥 陈蜀宇 李国勇.混合入侵检测系统的研究[J].计算机技术与发展,2010,(06):148.
　GAO Zheng,CHEN Shu-yu,LI Guo-yong.Research of a Hybrid Intrusion Detection System[J].,2010,(09):148.
[3]包力伟 周俊.铸锻企业生产质量控制系统的开发[J].计算机技术与发展,2008,(04):174.
　BAO Li-wei,ZHOU Jun.Development of a Manufacture Quality Control System in Casting Company[J].,2008,(09):174.
[4]李志俊 程家兴 金奎 饶玉佳.基于样本期望训练数的BP神经网络改进研究[J].计算机技术与发展,2009,(05):103.
　LI Zhi-jun,CHENG Jia-xing,JIN Kui,et al.BP Algorithm Improvement Based on Sample Expected Training Number[J].,2009,(09):103.
[5]李龙澍 葛瑞峰 王慧萍.基于神经网络的批强化学习在Robocup中的应用[J].计算机技术与发展,2009,(07):98.
　LI Long-shu,GE Rui-feng,WANG Hui-ping.Application of Batch Reinforcement Learning Based on NN to Robocup[J].,2009,(09):98.
[6]贾志先.神经网络在空白试卷识别中的应用[J].计算机技术与发展,2009,(08):208.
　JIA Zhi-xian.Application of Neural Network in Recognization Blank Examination Paper[J].,2009,(09):208.
[7]肖宜龙 路游 亓永刚.基于神经网络的NURBS曲面重建[J].计算机技术与发展,2009,(09):65.
　XIAO Yi-long,LU You,QI Yong-gang.NURBS Surface Reconstruction Based on Neural Network[J].,2009,(09):65.
[8]蔡秋茹 罗烨 柳益君 叶飞跃.企业资信的BP神经网络评估模型研究[J].计算机技术与发展,2009,(10):117.
　CAI Qiu-ru,LUO Ye,LIU Yi-jun,et al.Research on BP Neural Network Model for Corporation Credit Rating[J].,2009,(09):117.
[9]王晓敏 刘希玉 戴芬.BP神经网络预测算法的改进及应用[J].计算机技术与发展,2009,(11):64.
　WANG Xiao-min,LIU Xi-yu,DAI Fen.Improvement and Application of BP Neural Network Forecasting Algorithm[J].,2009,(09):64.
[10]崔海青 刘希玉.基于粒子群算法的RBF网络参数优化算法[J].计算机技术与发展,2009,(12):117.
　CUI Hai-qing,LIU Xi-yu.Parameter Optimization Algorithm of RBF Neural Network Based on PSO Algorithm[J].,2009,(09):117.
[11]王嘉文,王传栋,杨雁莹.一种中文人名识别的训练架构[J].计算机技术与发展,2018,28(07):53.[doi:10.3969/ j. issn.1673-629X.2018.07.012]
　WANG Jia-wen,WANG Chuan-dong,YANG Yan-ying.A Training Framework for Chinese Name Recognition[J].,2018,28(09):53.[doi:10.3969/ j. issn.1673-629X.2018.07.012]
[12]贾翻连,张丽红.基于改进的卷积神经网络的人群密度估计[J].计算机技术与发展,2019,29(02):77.[doi:10．3969/j．issn．1673－629X．2019．02．016]
　JIA Fanlian,ZHANG Lihong.Crowd Density Estimation Based on an Improved Convolution Neural Network[J].,2019,29(09):77.[doi:10．3969/j．issn．1673－629X．2019．02．016]
[13]叶 硕,褚 钰,王 祎,等.语音识别中声学模型研究综述[J].计算机技术与发展,2020,30(03):181.[doi:10. 3969 / j. issn. 1673-629X. 2020. 03. 035]
　YE Shuo,CHU Yu,WANG Yi,et al.Summary of Acoustic Models in Speech Recognition[J].,2020,30(09):181.[doi:10. 3969 / j. issn. 1673-629X. 2020. 03. 035]
[14]张明淘,韩 普.医疗实体识别研究进展[J].计算机技术与发展,2020,30(04):57.[doi:10. 3969 / j. issn. 1673-629X. 2020. 04. 011]
　ZHANG Ming-tao,HAN Pu.Research Progress on Medical Entity Recognition[J].,2020,30(09):57.[doi:10. 3969 / j. issn. 1673-629X. 2020. 04. 011]
[15]黄 璐,毛晓艳.基于深度学习的诊断心脑血管疾病分类方法[J].计算机技术与发展,2021,31(增刊):36.[doi:10. 3969 / j. issn. 1673-629X. 2021. S. 007]
　HUANG Lu,MAO Xiao-yan.Classification Method of Cardiovascular and Cerebrovascular Diseases Based on Deep Learning[J].,2021,31(09):36.[doi:10. 3969 / j. issn. 1673-629X. 2021. S. 007]
[16]王宇欣,方浩宇,张 伟,等.注意力机制在情感分析中的应用研究[J].计算机技术与发展,2022,32(04):193.[doi:10. 3969 / j. issn. 1673-629X. 2022. 04. 033]
　WANG Yu-xin,FANG Hao-yu,ZHANG Wei,et al.Application Research of Attention Mechanism in Sentiment Analysis[J].,2022,32(09):193.[doi:10. 3969 / j. issn. 1673-629X. 2022. 04. 033]
[17]姜丽莉,黄承宁.融合注意力机制改进残差网络的表情识别方法[J].计算机技术与发展,2022,32(05):42.[doi:10. 3969 / j. issn. 1673-629X. 2022. 05. 007]
　JIANG Li-li,HUANG Cheng-ning.An Expression Recognition Method Based on Fusion of Attention Mechanism and Improved Residual Network[J].,2022,32(09):42.[doi:10. 3969 / j. issn. 1673-629X. 2022. 05. 007]
[18]高新成,杜功鑫,王莉利,等.深度学习在地震初至拾取中的应用综述[J].计算机技术与发展,2022,32(08):1.[doi:10. 3969 / j. issn. 1673-629X. 2022. 08. 001]
　GAO Xin-cheng,DU Gong-xin,WANG Li-li,et al.Application of Deep Learning in Earthquake First Break Picking[J].,2022,32(09):1.[doi:10. 3969 / j. issn. 1673-629X. 2022. 08. 001]
[19]朱晓雨,曹自平,崔红涛.超声波穿金属无线通信噪声智能抑制方法研究[J].计算机技术与发展,2023,33(09):190.[doi:10. 3969 / j. issn. 1673-629X. 2023. 09. 028]
　ZHU Xiao-yu,CAO Zi-ping,CUI Hong-tao.Research on Intelligent Noise Suppression Method of Ultrasonic through Metal Wireless Communication[J].,2023,33(09):190.[doi:10. 3969 / j. issn. 1673-629X. 2023. 09. 028]

更新日期/Last Update: 2020-09-10