[1]宛艳萍,闫思聪,于海阔,等.结合 SS-GAN 和 BERT 的文本分类模型[J].计算机技术与发展,2023,33(02):187-194.[doi:10. 3969 / j. issn. 1673-629X. 2023. 02. 028]
　WAN Yan-ping,YAN Si-cong,YU Hai-kuo,et al.A Text Classification Model Based on Semi-supervised Generative Adversarial Networks and BERT[J].,2023,33(02):187-194.[doi:10. 3969 / j. issn. 1673-629X. 2023. 02. 028]
点击复制

结合 SS-GAN 和 BERT 的文本分类模型()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
33
期数:
2023年02期
页码:
187-194
栏目:
人工智能
出版日期:
2023-02-10

文章信息/Info

Title:
A Text Classification Model Based on Semi-supervised Generative Adversarial Networks and BERT
文章编号:
1673-629X(2023)02--0187-08
作者:
宛艳萍; 闫思聪; 于海阔; 许敏聪
河北工业大学 人工智能与数据科学学院,天津 300401
Author(s):
WAN Yan-ping; YAN Si-cong; YU Hai-kuo; XU Min-cong
School of Artificial Intelligence and Data Science,Hebei University of Technology,Tianjin 300401,China
关键词:
本分类; 半监督; BERT; 生成对抗网络; 模型压缩
Keywords:
text classification; semi-supervised; BERT; generative adversarial network; model compression
分类号:
TP391. 1
DOI:
10. 3969 / j. issn. 1673-629X. 2023. 02. 028
摘要:
BERT 是近年来提出的一种大型的预训练语言模型,在文本分类任务中表现优异,但原始 BERT 模型需要大量标注数据来进行微调训练,且参数规模大、时间复杂度高。 在许多真实场景中,大量的标注数据是不易获取的,而且模型参数规模过大不利于在真实场景的实际应用。 为了解决这一问题,提出了一种基于半监督生成对抗网络的 BERT 改进模型GT-BERT。 采用知识蒸馏的压缩方法将 BERT 模型进行压缩;引入半监督生成对抗网络的框架对 BERT 模型进行微调并选择最优生成器与判别器配置。 在半监督生成对抗网络的框架下增加无标签数据集对模型进行微调,弥补了标注数据较少的缺点。 在多个数据集上的实验结果表明,改进模型 GT-BERT 在文本分类任务中性能优异,可以有效利用原始模型不能使用的无标签数据,大大降低了模型对标注数据的需求,并且具有较低的模型参数规模与时间复杂度。
Abstract:
BERT is a large-scale pre-training language model proposed in recent years,which performs well in text classification tasks.However,the original BERT model requires a large amount of labeled data for fine-tuning training with large parameter scale and hightime complexity. In many real scenarios,a large amount of 　labeled data is not easy to obtain,and the large scale of model parameters isnot conducive to practical application in real scenarios. To solve this problem,an improved BERT model GT-BERT based on semi-supervised generative adversarial network is proposed. The BERT model is compressed by the compression method of knowledgedistillation,and the framework of semi-supervised generative adversarial network is introduced to fine-tune the BERT model and selectthe optimal generator and discriminator configurations. In the framework of semi-supervised generative adversarial network,an unlabeleddata set is added to fine-tune the model,which makes up for the shortcomings of less labeled data. The experimental results on multipledatasets show that the improved model GT-BERT has excellent performance in text classification tasks,can effectively use unlabeled datathat the original model cannot use,greatly reduces the model’s demand for labeled data,and has low model parameter size and time complexity.

相似文献/References:

[1]刘炯 徐同阁.基于NetFlow的应用协议半监督识别算法[J].计算机技术与发展,2010,(07):9.
　LIU Jiong,XU Tong-ge.A Semi-Supervised Clustering Algorithm for Application Protocol Recognition Based on NetFlow[J].,2010,(02):9.
[2]李亚娥,汪西莉.一种自适应的半监督图像分类算法[J].计算机技术与发展,2013,(02):112.
　LI Ya-e,WANG Xi-li.An Adaptive Semi-supervised Image Classification Algorithm[J].,2013,(02):112.
[3]罗 杨,万黎明,李 理,等.基于改进 U-Net 网络的半监督裂缝分割方法[J].计算机技术与发展,2022,32(12):179.[doi:10. 3969 / j. issn. 1673-629X. 2022. 12. 027]
　LUO Yang,WAN Li-ming,LI Li,et al.A Semi-supervised Crack Segmentation Method Based on Improved U-Net Network[J].,2022,32(02):179.[doi:10. 3969 / j. issn. 1673-629X. 2022. 12. 027]

更新日期/Last Update: 2023-02-10