«上一篇/Previous Article|本期目录/Table of Contents|下一篇/Next Article»

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:

32

期数:

2022年07期

页码:

70-74

栏目:

图形与图像

出版日期:

2022-07-10

文章信息/Info

Title:

Study of Flower Image Classification Based on Spark

文章编号:

1673-629X(2022)07-0070-05

作者:

侯向宁; 徐草草; 杨井荣

成都理工大学 工程技术学院 电子信息与计算机工程系,四川 乐山 614000

Author(s):

HOU Xiang-ning; XU Cao-cao; YANG Jing-rong

Department of Electronic Information and Computer Engineering,School of Engineering and Technique,Chengdu University of Technology,Leshan 614000,China

关键词:

花卉分类; Hadoop; Spark; VGG16; TensorFlowOnSpark; SK 单元

Keywords:

flower classification; Hadoop; Spark; VGG16; TensorFlowOnSpark; SK unit

分类号:

TP391

DOI:

10. 3969 / j. issn. 1673-629X. 2022. 07. 012

摘要:

针对传统单机模式对海量花卉图像数据分类效率低下以及现有网络模型对花卉分类准确率不高的问题,首先通过搭建 Hadoop 及 Spark 分布式计算框架,利用 HDFS 存储海量花卉图像数据,Spark 进行分布式并行计算,HBASE 存储相关的集群参数及网络模型参数。 其次在对现有的 VGG16 网络模型进行研究的基础上, 将选择性软注意力机制引入VGG16 网络对其进行改进,使 VGG16 网络可以从不同的感受野获取信息,并使网络泛化能力变得更强。 最终在 Spark 分布式计算框架中采用 TensorFlowOnSpark 技术,实现花卉图像特征提取、模型训练及分类测试的并行化,既降低了模型训练的时间,同时也提高了花卉分类的准确率。 实验表明,与未引入 SK( 选择性内核) 单元的 VGG16 模型相比,花卉分类的准确率提高了近 15. 3 个百分点。 实验还表明,分布式计算有利于负载均衡,极大地降低了模型训练及分类测试的耗时,能进一步提高海量花卉数据分类的效率。

Abstract:

In view of the low efficiency of the traditional single-machine mode in the classification of massive flower image data? ? ?and the low accuracy of the existing network model in the classification of flowers,firstly by building Hadoop and Spark distributed computing framework,HDFS is used to store massive flower image data,and Spark is used for distributed parallel computing,and HBASE to store cluster parameters and network model parameters. Secondly,on the basis of the research of existing VGG16 network model,the selective soft attention mechanism is introduced into the VGG16 network to improve it,so that the VGG16 network can obtain information from different receptive fields, and make the network generalization ability become stronger. Finally, TensorFlowOnSpark technology was adopted in the Spark distributed computing framework to realize the parallelization of flower image feature extraction,model training and classification test,which not only reduced the time of model training, but also improved the accuracy of flower classification. The experiment shows that compared with the VGG16 model without SK ( selective kernel) unit, the accuracy of flower classification is improved by 15. 3 percentage points. The experiment also shows that distributed computing is beneficial to load balance,greatly reduces the time of model training and classification test,and can further improve the efficiency of massive flower data classification.

常用功能

更新日期/Last Update: 2022-07-10