[1]郭 辉,郭静纯,张 甜.基于梯度优化的多任务混合学习方法[J].计算机技术与发展,2021,31(10):7-12.[doi:10. 3969 / j. issn. 1673-629X. 2021. 10. 002]
　GUO Hui,GUO Jing-chun,ZHANG Tian.An Approach of Mixed Multi-task Learning Based on Gradient Optimization[J].,2021,31(10):7-12.[doi:10. 3969 / j. issn. 1673-629X. 2021. 10. 002]
点击复制

基于梯度优化的多任务混合学习方法()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
31
期数:
2021年10期
页码:
7-12
栏目:
大数据分析与挖掘
出版日期:
2021-10-10

文章信息/Info

Title:
An Approach of Mixed Multi-task Learning Based on Gradient Optimization
文章编号:
1673-629X(2021)10-0007-06
作者:
郭 辉; 郭静纯; 张 甜
宁夏大学 信息工程学院,宁夏 银川 750021
Author(s):
GUO Hui; GUO Jing-chun; ZHANG Tian
School of Information Engineering,Ningxia University,Yinchuan 750021,China
关键词:
多任务学习; 硬参数共享; 特征提取; 混合训练; 梯度优化
Keywords:
multi-task learning; hard parameter sharing; feature extraction; mixed training; gradient optimization
分类号:
TP183
DOI:
10. 3969 / j. issn. 1673-629X. 2021. 10. 002
摘要:
多任务学习作为深度学习的一个分支,得到了广泛关注与深入研究,但仍然存在网络结构复杂、任务区分困难的问题。 据此,基于硬参数共享神经网络给出梯度优化的多任务混合学习方法。 首先,无需区分不同任务,将多任务训练数据一同送入网络进行混合训练,所有任务共用一个损失函数,前一次训练所得的网络共享层参数作为下次训练的共享层初始化参数;其次,根据不同共享层提取特征的差异和任务在深层梯度变化的不同,调节相应的激活值,优化网络参数,既保持了硬参数共享神经网络结构的简洁性特点,又利于解决多任务训练过程中数据的非平衡问题;最后,通过在 UCI 公开数据集中的鸢尾花和天平秤数据上的实际应用,以及与传统的硬参数共享神经网络的纵向对比,验证了该学习方法的可行性与有效性。
Abstract:
As a branch of deep learning,multi-task learning has received extensive attention and in-depth research. However,there are still some difficult problems such as overly complex network structure and indistinguishable multiple related tasks. An approach of mixed multi-task learning based on gradient optimization is proposed,that is established for the hard parameter sharing neural network. Firstly,the multi-task training data is fed into the network together for mixed training without distinguishing between different tasks,and all tasks share one loss function, during which the network sharing layer parameters obtained from the previous training are used as the initialization parameters of the sharing layer for the next training. Secondly,according to the difference of features extracted from different sharing layers and the different gradient changes of tasks in deep layers,the corresponding activation values are adjusted and the network parameters are optimized,which not only maintain the conciseness of the hard parameter sharing network structure,but also help to solve the non-equilibrium problem of data during the multi-task training. Finally, through the practical application of iris and balance data in the UCI open data set,as well as the longitudinal comparison with the traditional hard parameter sharing neural network,the feasibility and effectiveness of the proposed learning method is verified.

相似文献/References:

[1]许棣华 王志坚.基于多任务学习的邮件过滤系统的研究[J].计算机技术与发展,2010,(10):137.
　XU Di-hua,WANG Zhi-jian.Research of Spam Filter System Based on Multitask Learning[J].,2010,(10):137.
[2]沈佳敏,鲍秉坤.基于深度学习的广告布局图片美学属性评价[J].计算机技术与发展,2021,31(03):39.[doi:10. 3969 / j. issn. 1673-629X. 2021. 03. 007]
　SHEN Jia-min,BAO Bing-kun.Aesthetic Attribute Evaluation of Advertising Layout Images Based on Deep Learning[J].,2021,31(10):39.[doi:10. 3969 / j. issn. 1673-629X. 2021. 03. 007]
[3]万 苗,任 杰 *,马 苗,等.多任务学习在中国方言分类中的应用研究[J].计算机技术与发展,2022,32(04):109.[doi:10. 3969 / j. issn. 1673-629X. 2022. 04. 019]
　WAN Miao,REN Jie *,MA Miao,et al.Chinese Dialect Classification via Multi-task Learning[J].,2022,32(10):109.[doi:10. 3969 / j. issn. 1673-629X. 2022. 04. 019]

更新日期/Last Update: 2021-10-10