«上一篇/Previous Article|本期目录/Table of Contents|下一篇/Next Article»

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:

32

期数:

2022年06期

页码:

21-27

栏目:

人工智能

出版日期:

2022-06-10

文章信息/Info

Title:

Relation Classification Model Based on Sliding Window Attention Network

文章编号:

1673-629X(2022)06-0021-07

作者:

潘理虎; 陈亭亭; 闫慧敏; 赵彭彭; 张 睿; 张英俊

1. 太原科技大学 计算机科学与技术学院,山西 太原 030024;

2. 中国科学院 地理科学与资源研究所,北京 100101

Author(s):

PAN Li-hu1 ; CHEN Ting-ting1 ; YAN Hui-min2 ; ZHAO Peng-peng1 ; ZHANG Rui1 ; ZHANG Ying-jun1

1. School of Computer Science and Technology,Taiyuan University of Science and Technology,Taiyuan 030024,China;

2. Institute of Geographic Science and Natural Resource Research,Chinese Academy of Sciences,Beijing 100101,China

关键词:

实体关系抽取; 滑动窗口; SBERT; 注意力机制; 局部信息; 全局信息

Keywords:

entity relation extraction; sliding window; SBERT; attention mechanism; local information; global information

分类号:

TP391. 1

DOI:

10. 3969 / j. issn. 1673-629X. 2022. 06. 004

摘要:

实体关系抽取是构建知识图谱过程中至关重要的一步。 将注意力机制引入卷积神经网络或循环神经网络是目前关系抽取任务中比较主流的解决方法,谷歌最新提出的 BERT 模型在多项自然语言处理任务中都取得了非常好的效果。为了充分融合局部信息和全局信息,并提高处理效率,该文提出了滑动窗口注意力网络模型( Sliding Window AttentionNetwork,SWAN) 。 该模型首先通过预训练的 word2vec 生成词向量,加入位置表示并使用 TransE 模型对实体进行表征以充分突出实体信息,再采用基于 BERT 的 SBERT 模型对句子进行表征,在此基础上采用多种滑动窗口注意力机制捕获局部信息,然后在聚集层对抽取到的局部信息进行聚合,最后利用 softmax 函数来实现实体关系的分类。 实验结果表明,提出的 SWAN 模型在 SemEval2010 Task 8 数据集上取得了较高的准确率,优于对比的现有关系抽取模型,同时模型训练效率也得到极大提升。

Abstract:

Relationship extraction is a crucial step in the process of constructing a knowledge graph. Introducing the attention mechanism into convolutional neural networks? ? or recurrent neural networks is the current mainstream solution for relation extraction tasks. The BERT model recently proposed by Google is efficient in a number of natural language processing tasks. In order to fully integrate local information and global information and improve processing efficiency,we propose a sliding window attention network model ( SWAN) .It firstly generates word vectors through pre-trained word2vec,adding location representation and using TransE to characterize the entity to fully highlight the entity information, and then uses the BERT - based SBERT model to characterize the sentence. On this basis,avariety of sliding window attention mechanisms are used to capture the local information, and then the extracted local information is aggregated in the aggregation layer,and finally the softmax function is used to classify entity relationships. The experimental results show that the proposed SWAN model achieves a higher accuracy rate on the SemEval2010 Task 8 data set,which is better than the existing relation extraction model for comparison,and the model training efficiency is also greatly improved.

常用功能

更新日期/Last Update: 2022-06-10