[1]罗 峦,夏骄雄.融合 ERNIE 与改进 Transformer 的中文 NER 模型[J].计算机技术与发展,2022,32(10):120-125.[doi:10. 3969 / j. issn. 1673-629X. 2022. 10. 020]
　LUO Luan,XIA Jiao-xiong.Research on Chinese Named Entity Recognition Combining ERNIE with Improved Transformer[J].,2022,32(10):120-125.[doi:10. 3969 / j. issn. 1673-629X. 2022. 10. 020]
点击复制

融合 ERNIE 与改进 Transformer 的中文 NER 模型()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
32
期数:
2022年10期
页码:
120-125
栏目:
人工智能
出版日期:
2022-10-10

文章信息/Info

Title:
Research on Chinese Named Entity Recognition Combining ERNIE with Improved Transformer
文章编号:
1673-629X(2022)10-0120-06
作者:
罗 峦; 夏骄雄
上海理工大学 光电信息与计算机工程学院,上海 200093
Author(s):
LUO Luan; XIA Jiao-xiong
School of Option-electrical and Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China
关键词:
自然语言处理; 命名实体识别; 深度学习; ERNIE; 注意力机制
Keywords:
NLP; named entity recognition; deep learning; ERNIE; attention mechanism
分类号:
TP391
DOI:
10. 3969 / j. issn. 1673-629X. 2022. 10. 020
摘要:
命名实体识别是信息抽取和关系提取基础的关键任务。 针对中文命名实体识别问题,提出了一种融合 ERNIE 和改进 Transformer 的中文命名实体识别深度学习模型——ERIT( combining ERNIE with Improved Transformer) 。 ERIT 使用ERNIE 训练词向量作为嵌入层,摆脱了模型对于分词预处理过程的依赖,避免出现因分词错误以及信息缺失引起错误传播而导致准确率降低的情况,在兼顾输入文本识别精度的同时进一步优化输入语句的词向量,利用 Transformer 获取输入序列的上下文信息并进行特征提取,结合自注意力层对权重参数进行更新,并在此基础上,通过在自注意力层上增加约束正则项提高对参数约束性以提高每个生成标签的准确性,并加入计划采样机制以解决模型训练与测试过程中存在的不匹配问题。 实验证明,ERNIE 作为嵌入层有效优化了词向量并提高了识别精度,且模型相较于其他实体识别模型取得了较好的效果。
Abstract:
Named entity recognition is the basic and key task of information extraction and relationship extraction. Aiming at the problem of Chinese named entity recognition,ERIT ( combining ERNIE and Improved Transformer) ,a Chinese named entity recognition deeplearning model combining ERNIE and improved Transformer,is proposed. ERIT uses Ernie training word vector as the embedding layer,which removes the dependence of the model on the word segmentation preprocessing process,avoids the situation that the accuracy is reduced due to the error of word segmentation and the lack of information. It further optimizes the word vector of the input sentencewhile taking into account the recognition accuracy of the input text. Transformer is used to obtain the context information of the input sequence and extract features,and the weight parameters are updated with the self-attention layer. On this basis,the attention regularizationis added to the self attention layer in order to improve the accuracy of each generated label by improving the constraint of parameters,and the scheduled sampling mechanism is added to solve the mismatch problem in the process of model training and testing. Experiments show that as the input of embedding layer, ERNIE effectively optimizes the word vector and improves the recognition accuracy, and compared with other entity recognition models,the model achieves better results.

相似文献/References:

[1]陈国华 赵克 李亚涛 易帅.自然语言处理系统中的事件类名词的耦合处理[J].计算机技术与发展,2008,(06):60.
　CHEN Guo-hua,ZHAO Ke,LI Ya-tao,et al.Coupling Processing of Event Noun in NLP Systems[J].,2008,(10):60.
[2]程节华.基于FAQ的智能答疑系统中分词模块的设计[J].计算机技术与发展,2008,(07):181.
　CHENG Jie-hua.Design of Words Module in Intelligent Q/A System Based on FAQ[J].,2008,(10):181.
[3]杨欢 许威 赵克 陈余.动词属性在自然语言处理当中的研究与应用[J].计算机技术与发展,2008,(07):233.
　YANG Huan,XU Wei,ZHAO Ke,et al.Research and Application of Verb Attributes in Natural Language Processing[J].,2008,(10):233.
[4]孙超 张仰森.面向综合语言知识库的知识融合与获取研究[J].计算机技术与发展,2010,(08):25.
　SUN Chao,ZHANG Yang-sen.Research of Knowledge Integration and Obtaining Oriented Comprehensive Language Knowledge System[J].,2010,(10):25.
[5]党建 亿珍珍 赵克 殷鸿.数学领域集体词结构形式化处理研究[J].计算机技术与发展,2007,(05):121.
　DANG Jian,YI Zhen-zhen,ZHAO Ke,et al.Research of Formalization Processing for Collective Structures in Mathematics Domain[J].,2007,(10):121.
[6]江有福 郑庆华.自然语言网络答疑系统中倒排索引技术的研究[J].计算机技术与发展,2006,(02):126.
　JIANG You-fu,ZHENG Qing-hua.Research of Inverted Index in NLWAS[J].,2006,(10):126.
[7]刘亚清 张瑾 于纯妍.基于义原同现频率的汉语词义排歧系统[J].计算机技术与发展,2006,(05):184.
　LIU Ya-qing,ZHANG Jin,YU Chun-yan.A Chinese Word Sense Disambiguation System Based on Primitive CO- Occurrence Data[J].,2006,(10):184.
[8]刘政怡 李炜 吴建国.基于IMM—IME的汉字键盘输入法编程技术研究[J].计算机技术与发展,2006,(12):43.
　LIU Zheng-yi,LI Wei,WU Jian-guo.Research of Programming Technology of Chinese Input Method Based on IMM- IME[J].,2006,(10):43.
[9]赵鹏 何留进 孙凯 方薇[].基于情感计算的网络中文信息分析技术[J].计算机技术与发展,2010,(11):146.
　ZHAO Peng,HE Liu-jin,SUN Kai,et al.Analyzing Technologies of Internet Chinese Information Based on Affective Computing[J].,2010,(10):146.
[10]徐远方 李成城.基于SVM和词间特征的新词识别研究[J].计算机技术与发展,2012,(05):134.
　XU Yuan-fang,LI Cheng-cheng.Research on New Word Identification Based on SVM and Word Characteristics[J].,2012,(10):134.
[11]王 俊,王修来*,栾伟先,等.基于 BERT 模型的科研人才领域命名实体识别[J].计算机技术与发展,2021,31(11):21.[doi:10. 3969 / j. issn. 1673-629X. 2021. 11. 004]
　WANG Jun,WANG Xiu-lai*,LUAN Wei-xian,et al.Research on Named Entity Recognition of Scientific Research Talents Field Based on BERT Model[J].,2021,31(10):21.[doi:10. 3969 / j. issn. 1673-629X. 2021. 11. 004]
[12]贵向泉,郭 亮,李 立.基于 MRC 和 ERNIE 的有色冶金命名实体识别模型[J].计算机技术与发展,2023,33(10):93.[doi:10. 3969 / j. issn. 1673-629X. 2023. 10. 015]
　GUI Xiang-quan,GUO Liang,LI Li.Nonferrous Metallurgical Named Entity Recognition Model Based on MRC and ERNIE[J].,2023,33(10):93.[doi:10. 3969 / j. issn. 1673-629X. 2023. 10. 015]
[13]李欣宇,赵 震*.命名实体消歧研究综述[J].计算机技术与发展,2024,34(02):1.[doi:10. 3969 / j. issn. 1673-629X. 2024. 02. 001]
　LI Xin-yu,ZHAO Zhen*.Review of Named Entity Disambiguation Studies[J].,2024,34(10):1.[doi:10. 3969 / j. issn. 1673-629X. 2024. 02. 001]

更新日期/Last Update: 2022-10-10