基于正则化 KL 距离的交叉验证折数 K 的选择-《计算机技术与发展》

文章信息/Info

Title:: A Selection Criterion of Fold K in Cross-validation Based on Regularized KL Distance

作者:: 褚荣燕¹; 王钰²; 3 *; 杨杏丽¹; 李济洪³; 1. 山西大学数学科学学院,山西太原 030006;
2. 山西大学现代教育技术学院,山西太原 030006;
3. 山西大学软件学院,山西太原 030006

Author(s):: CHU Rong-yan¹; WANG Yu²; 3 *; YANG Xing-li¹; LI Ji-hong³; 1. School of Mathematical Sciences,Shanxi University,Taiyuan 030006,China;
2. School of Modern Educational Technology,Shanxi University,Taiyuan 030006,China;
3. School of Software,Shanxi University,Taiyuan 030006,China

Keywords:: K-fold cross-validation; selection of the fold K; KL distance (Kullback-Leibler distance); regularized; machine learning

摘要:: 在机器学习中,K 折交叉验证方法常常通过把数据分成多个训练集和测试集来进行模型评估与选择,然而其折数 K 的选择一直是一个公开的问题。注意到上述交叉验证数据划分的一个前提假定是训练集和测试集的分布一致,但是实际数据划分中,往往不是这样。因此,可以通过度量训练集和测试集的分布一致性来进行 K 折交叉验证折数 K 的选择。直观地,KL(Kullback-Leibler)距离是一种合适的度量方法,因为它度量了两个分布之间的差异。然而直接基于 KL 距离进行 K 的选择时,从多个数据实验结果发现随着 K 的增加 KL 距离也在增大,显然这是不合适的。为此,提出了一种基于正则化 KL 距离的 K 折交叉验证折数 K 的选择准则,通过最小化此正则 KL 距离来选择合适的折数 K。进一步多个真实数据实验验证了提出准则的有效性和合理性。

Abstract:: In machine learning,the K-fold cross-validation method often divides the data into multiple training and test sets for model evaluation and selection. However,the selection of the fold K is always an open problem. Note that one of the premises of the above crossvalidation data division assumes that the training set and the test set have the same distribution,but in actual data division,this is often not the case. Therefore,the selection of the fold K can be performed by measuring the distribution consistency of the training set and the testset in K-fold cross-validation. Intuitively,KL ( Kullback-Leibler) distance is a suitable measure because it measures the differencebetween two distributions. However,when selecting K directly based on the KL distance,it is found from multiple data experimentalresults that the KL distance also increases with the increase of K,which is obviously inappropriate. To this end,a selection criterion of thefold K in K-fold cross-validation based on regularized KL distance is proposed,and the appropriate fold K is selected by minimizing thisregular KL distance. Multiple real data experiments in a recent step have verified the effectiveness and rationality of the proposed criterion.