注意力机制的 LSTM-DBN 维语人称代词指代消解-《计算机技术与发展》

文章信息/Info

Title:: Attention Mechanism of LSTM-DBN Uyghur Personal Pronoun Anaphora Resolution

作者:: 李东欣¹ ; 禹龙² ; 田生伟¹ ; 李圃³; 赵建国⁴; 1. 新疆大学软件学院,新疆乌鲁木齐 830008; 2. 新疆大学网络中心,新疆乌鲁木齐 830008; 3. 新疆大学语言学院,新疆乌鲁木齐 830046; 4. 新疆大学人文学院,新疆乌鲁木齐 830046

Author(s):: LI Dong-xin 1 ; YU Long 2 ; TIAN Sheng-wei 1 ; LI Pu 3 ; ZHAO Jian-guo 4; 1. School of Software,Xinjiang University,Urumqi 830008,China; 2. Network Center,Xinjiang University,Urumqi 830008,China; 3. School of Languages,Xinjiang University,Urumqi 830046,China; 4. School of Humanities,Xinjiang University,Urumqi 830046,China

Keywords:: personal pronouns; anaphora resolution; word embedding; attention-based mechanism; deep belief- network; long short-term memory

摘要:: 针对维吾尔语中人称代词指代歧义问题,结合维吾尔语言的词法、语法、词间位置等关系,以及注意力机制、长短时记忆网络和深度置信网络,提出了一种维语人称代词指代消解模型。首先,分析维语中人称代词指代的特点和表达规律,提取出相应词向量特征;其次,借助长短时记忆网络挖掘维吾尔语人称代词的语义特征,并利用注意力机制的相似性度量、权重调节能力,避免信息在层间传递的丢失,实现特征编码向量的信息整合;最后利用深度置信网络(DBN)进一步挖掘出隐藏在维语上下文中的深层语义特征,完成维语人称代词指代消解。实验结果表明,所提模型在挖掘深层语义信息和识别效果上优于传统的深度学习模型,准确率达到了 81.14%, F 1 达到了 78.83%。

Abstract:: Aiming at the anaphora ambiguity of personal pronouns in Uyghur language,combining with the lexical,grammar and interword position of Uyghur language as well as the attention mechanism,long-short term memory and deep belief networks,an anaphora resolution model is proposed. Firstly,the characteristics and expression patterns of the personal pronouns in Uyghur language is analyzed and the corresponding word vector characteristics is extracted. Secondly,long short-term memory network is used to explore the semantic features of personal pronoun and the attention mechanism of similarity measurement and weights adjustment ability is used to avoid the loss of the information transfer between layers,to realize the information integration of the feature encoding vector. Finally,the deep belief network (DBN) is applied to further explore the deep semantic features hidden in Uyghur context and complete the Uyghur personal pro-nouns anaphora resolution. The experiment shows that the proposed model is superior to the traditional deep learning model in the mining of deep semantic information and recognition effect,with an accuracy of 81.14% and 78.83% of F 1 respectively.