[1]曹学飞,杨 帆,李济洪,等.基于 mx2 正则化交叉验证的神经网络超参数调优方法[J].计算机技术与发展,2024,34(04):168-173.[doi:10. 3969 / j. issn. 1673-629X. 2024. 04. 025]
　CAO Xue-fei,YANG Fan,LI Ji-hong,et al.A Method for Hyper-parameter Tuning of Neural Network Based on mx2 Regularized Cross-validation[J].,2024,34(04):168-173.[doi:10. 3969 / j. issn. 1673-629X. 2024. 04. 025]

点击复制

基于 mx2 正则化交叉验证的神经网络超参数调优方法()

分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:

34

期数:

2024年04期

页码:

168-173

栏目:

人工智能

出版日期:

2024-04-10

文章信息/Info

Title:

A Method for Hyper-parameter Tuning of Neural Network Based on mx2 Regularized Cross-validation

文章编号:

1673-629X(2024)04-0168-06

作者:

曹学飞¹ ; 杨 帆¹ ; 李济洪² ; 王瑞波² ; 牛 倩¹

1. 山西大学 自动化与软件学院,山西 太原 030006;

2. 山西大学 现代教育技术学院,山西 太原 030006

Author(s):

CAO Xue-fei1 ; YANG Fan1 ; LI Ji-hong2 ; WANG Rui-bo2 ; NIU Qian1

1. School of Automation and Software Engineering,Shanxi University,Taiyuan 030006,China;

2. School of Modern Educational Technology,Shanxi University,Taiyuan 030006,China

关键词:

m x 2 交叉验证; 正则化; 神经网络; 超参数调优; 信噪比

Keywords:

m x2 cross-validation; regularization; neural network; hyper-parameter tuning; signal-to-noise

分类号:

TP391

DOI:

10. 3969 / j. issn. 1673-629X. 2024. 04. 025

摘要:

超参数调优是神经网络建模的关键问题。 针对传统的超参数调优方法存在的问题,该文提出了一种基于 m x2 正则化交叉验证的超参数调优方法。 目的是给出一种适用于复杂模

型、大数据集背景下的计算开销较小且稳健的超参数调优方法。 该方法的思想是从完整的数据集上选取少部分数据进行调优,避免模型在数据集较大时非常耗时的超参数调优难题;在 m x 2 交叉验证的基础上设置正则化条件均衡训练集与验证集之间的分布差异,从而减少分布不一致带来的性能波动;使用信噪比作为调优的优化目标,从而可以综合考虑模型性能评价指标的均值和方差;并采用正交设计选择相关性较低的超参数组合以提高调优效率。 以命名实体任务为例进行实验,在 CoNLL 2003 数据集上的实验结果显示,提出的调优方法能够选到和网格搜索性能上没有显著差异的超参数组合,且调优时间可显著降低约 66% 。

Abstract:

Hyper-parameter tuning is a key issue in neural network modeling. From the viewpoint of the problems of traditional hyper-parameter tuning methods,we propose a hyper - parameter tuning method based on m x 2 regularized cross - validation. The goal is topresent a robust hyper-parameter tuning method with low computational cost suitable for complex models and large datasets. The idea ofthe proposed method is to select a small number of data from the complete dataset for tuning,so as to avoid the time-consuming problemof hyper-parameter tuning when the dataset is large. Then,on the basis of m x 2 cross-validation,regularization is adopted to balance thedistribution difference between the training set and the validation set to reduce the performance fluctuation caused by the distribution inconsistency. The signal-to-noise ratio is used as the metric of hyper-parameter tuning,so that the mean and variance of the model performance can be comprehensively considered. The orthogonal design is used to select a combination of hyper - parameters with lowcorrelation to improve the tuning efficiency. The experimental results on the CoNLL 2003 dataset show that the proposed method canobtain a combination of hyper-parameters that is not significantly different from the grid search,and the tuning time can be significantlyreduced by about 66% .

常用功能