RFC-Net:基于残差结构的动作质量评估网络-《计算机技术与发展》

文章信息/Info

Title:: RFC-Net:Action Quality Assessment Network Based on Residual Structure

Author(s):: ZHOU Xian-wei; LAI Jian; CHEN Wei-tao; RUAN Le; LI Zhen-feng; YU Song-sen; School of Software,South China Normal University,Foshan 528010,China

Keywords:: action quality assessment; video feature extraction; video feature aggregation; neural networks; Spearman’s correlation coefficient

摘要:: 动作质量评估是视频分析中一个重要且具有挑战性的问题,动作质量评估是指对特定动作( 如跳水、体操等) 的完成质量进行评分,分数评估模型是通过将视频特征回归到该领域专家提供的真实分数来进行学习。现有的大多数方法是直接使用动作识别任务的模型如( C3D 和 I3D)来解决问题。为了增强网络模型的特征提取效果,从而提高分数回归的准确性,该文提出了一种基于残差结构的动作质量评估网络模型 RFC-Net。该网络由特征提取器和特征聚合器组成,在特征提取器中使用 I3D 网络对视频特征进行提取,在特征聚合器中对特征提取器最后一层卷积得到的视频特征分别进行平均的全局池化和残差卷积操作,对得到的结果进行特征融合,最后输出视频的分数表示。在动作质量评估领域公开的MTL-AQA 数据集上,该方法取得的斯皮尔曼相关性系数为 0. 946 3。为进一步验证模型在不同背景下、动作差异较大时的泛化能力,制作了羽毛球运动视频数据集,并在此基础上进行了不同模型之间的对比实验。

Abstract:: Action quality assessment is an important and challenging problem in video analysis. Action quality assessment refers to scoringthe completion quality of specific actions ( e. g. diving, gymnastics,etc. ) and score assessment models are learned by regressing videofeatures on the true scores provided by experts in the field. Most existing approaches use models for action recognition tasks such as( C3D and I3D) directly to solve the problem. To enhance the feature extraction effect of the network model and thus improve theaccuracy of the score regression,? we propose a residual structure-based action quality assessment network model RFC-Net. The networkconsists of a feature extractor and a feature aggregator,in which the video features are extracted using the I3D network in the feature extractor,and the video features obtained from the last layer of convolution in the feature extractor are subjected to average global poolingand residual convolution operations,respectively. The proposed method has achieved a Spearman correlation coefficient of 0. 946 3 on the MTL-AQA dataset,which is publicly available in the field of action quality assessment. In order to further validate the generalisationability of the model under different contexts and big action differences, a badminton video dataset was produced and a comparisonexperiment between different models was conducted on this basis