[1]李星,李涛.基于 Spark 的推荐系统的设计与实现[J].计算机技术与发展,2018,28(10):194-198.[doi:10.3969/ j. issn.1673-629X.2018.10.040]
　LI Xing,LI Tao.Design and Implementation of Recommendation System Based on Spark[J].,2018,28(10):194-198.[doi:10.3969/ j. issn.1673-629X.2018.10.040]
点击复制

基于 Spark 的推荐系统的设计与实现()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
28
期数:
2018年10期
页码:
194-198
栏目:
智能、算法、系统工程
出版日期:
2018-10-10

文章信息/Info

Title:
Design and Implementation of Recommendation System Based on Spark
文章编号:
1673-629X(2018)10-0194-05
作者:
李星; 李涛
南京邮电大学 通信与信息工程学院,江苏 南京 210003
Author(s):
LI Xing; LI Tao
chool of Communication and Information Technology,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
关键词:
大数据; Spark 平台; 推荐系统; 协同过滤(CF); 数据挖掘
Keywords:
big data; Spark; recommendation system; collaborative filtering (CF); data mining
分类号:
TP302
DOI:
10.3969/ j. issn.1673-629X.2018.10.040
文献标志码:
A
摘要:
推荐系统是数据挖掘的一个重要部分,能够实现海量数据信息的快速、全面、准确过滤。 然而基于以往传统单个主机模式实现的推荐算法其计算过程耗费的时间过长,已经不能满足当前商业时代快速可靠的技术追求。 大数据平台Spark 分布式计算框架通过引入 RDD(弹性分布式数据集)的概念以及基于内存的运算模式,能够更好地适应大数据挖掘这一应用场景。 推荐算法在实现过程中存在多次迭代计算,Spark 计算框架的使用可以极大提升推荐系统的运算效率。文中利用 Spark 平台设计了一个基于物品的协同过滤(Item-CF)算法的商品推荐系统,并将其应用在 MovieLens 数据集上运行测试。 实验结果表明,该系统能够提高推荐精确度并降低运算时间。
Abstract:
The recommendation system is an important part of data mining,which can realize the rapid,comprehensive and accurate filtering for a large number of data. However,it takes a lot of time to realize the proposed algorithm based on the traditional single-machine model,which cannot meet the fast and reliable business needs in today’s business era. The Spark distributed computing framework of big data platform can better adapt to big data mining by introducing the concept of RDD (resilient distributed datasets) and based on memory computing mode. The recommendation algorithm has many iterative calculations in the implementation process,and the use of the Spark calculation framework can greatly enhance the efficiency of the recommended system. We use the Spark platform to design a product recommendation system based on item-based collaborative filtering (Item-CF) algorithm,which is applied to run a test on the MovieLens data set. The experiment shows that the system can improve the recommendation accuracy and reduce the operation time.

相似文献/References:

[1]严霄凤,张德馨.大数据研究[J].计算机技术与发展,2013,(04):168.
　YAN Xiao-feng,ZHANG De-xin.Big Data Research[J].,2013,(10):168.
[2]王雷,陈彦先,袁哲,等. 面向预拌混凝土行业的云计算[J].计算机技术与发展,2014,24(08):14.
　WANG Lei,CHEN Yan-xian,YUAN Zhe JI Xu. Research on Cloud Computing for Ready-mixed Concrete Industry[J].,2014,24(10):14.
[3]金宗泽,冯亚丽,文必龙,等. 大数据分析流程框架的研究[J].计算机技术与发展,2014,24(08):117.
　JIN Zong-ze,FENG Ya-l,WEN Bi-long,et al. Research on Framework of Big Data Analytic Process[J].,2014,24(10):117.
[4]张也弛,周文钦,石润华. 一种面向云的大数据完整性检测协议[J].计算机技术与发展,2014,24(09):68.
　ZHANG Ye-chi,ZHOU Wen-qin,SHI Run-hua. A Big Data Integrity Checking Protocol for Cloud[J].,2014,24(10):68.
[5]谢怡,王航,刘新瀚,等. 大数据环境下数据读取关键技术研究[J].计算机技术与发展,2015,25(02):113.
　XIE Yi,WANG Hang,LIU Xin-han,et al. Research on Data Reading Techniques Based on Big Data Environment[J].,2015,25(10):113.
[6]付燕平,罗明宇,刘其军. 大数据三维模型快速显示技术研究[J].计算机技术与发展,2015,25(05):87.
　FU Yan-ping,LUO Ming-yu,LIU Qi-jun. Research on Fast Display Technology for Big Data Three-dimensional Model[J].,2015,25(10):87.
[7]赵震,任永昌. 大数据时代基于云计算的电子政务平台研究[J].计算机技术与发展,2015,25(10):145.
　ZHAO Zhen,REN Yong-chang. Research on E-government Platform Based on Cloud Computing in Big Data Era[J].,2015,25(10):145.
[8]胡存刚,程莹. 基于粒子群算法的大数据智能搜索引擎的研究[J].计算机技术与发展,2015,25(12):14.
　HU Cun-gang,CHENG Ying. Research on Big Data Intelligent Search Engine Based on PSO[J].,2015,25(10):14.
[9]孔钦,叶长青,孙赟.大数据下数据预处理方法研究[J].计算机技术与发展,2018,28(05):1.[doi:10．3969/j．issn．1673－629X．2018．05．001]
　KONG Qin,YE Changqing,SUN Yun.Research on Data Preprocessing Methods for Big Data[J].,2018,28(10):1.[doi:10．3969/j．issn．1673－629X．2018．05．001]
[10]杨明,李铁冰,姜茸,等.基于AHP 的大数据可用性及挖掘方案模型研究[J].计算机技术与发展,2018,28(05):51.[doi:10.3969/j.issn.1673－629X.2018.05.012]
　YANG Ming,LI Tie-bing,JIANG Rong,et al.Research on Model of Big Data Usability and Mining Strategy Based on AHP[J].,2018,28(10):51.[doi:10.3969/j.issn.1673－629X.2018.05.012]

更新日期/Last Update: 2018-10-10