[1]付剑生[] .徐林龙[]。 林文斌[]. 分布式全网职位搜索引擎的研究与实现[J].计算机技术与发展,2015,25(05):6-9.
　FU Jian-sheng[],XU Lin-long[],LIN Wen-bin[]. Research and Implementation of Distributed Network-wide Job Search Engine[J].,2015,25(05):6-9.
点击复制

 分布式全网职位搜索引擎的研究与实现()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
25
期数:
2015年05期
页码:
6-9
栏目:
智能、算法、系统工程
出版日期:
2015-05-10

文章信息/Info

Title:
 Research and Implementation of Distributed Network-wide Job Search Engine
文章编号:
1673-629X（2015)05-0006-04
作者:
 付剑生[1] .徐林龙[2]。 林文斌[1]
1. 西南交通大学 物理科学与技术学院；2.西南交通大学 数学学院
Author(s):
 FU Jian-sheng[1];  XU Lin-long[2];  LIN Wen-bin[1]
关键词:
 ; Solrcloud; 职位搜索; 网络爬虫; 分布式搜索引擎
Keywords:
 ; Solrcloud; job search; web crawlers; distributed search engine
分类号:
TP39
文献标志码:
A
摘要:
 传统招聘网站所查寻的信息仅限于站内搜索,而且每个招聘网站往往都会有重复的招聘信息,导致重复投递,对求职者和招聘者都造成了资源浪费。文中研究和分析了基于Lucene的分布式全文搜索引擎Solrcloud,设计了全网职位搜索引擎系统。该系统采用Bloom Filter进行数据及URL的去重,通过使用Zookeeper提供分布式同步服务,并通过多线程来实现网页并发抓取。通过对系统的测试表明,该系统具有良好的可靠性和应用性,并在大数据量的情况下保证了搜索的效率及准确性。
Abstract:
 The information traditional recruitment websites queried is restricted to site search,and each recruitment website may have du-plicate information,which will lead to a waste of resource for both job hunters and recruiters. Based on research and analysis about Solr-cloud,a distributed full text search engine based on Lucene,design a network-wide job search engine. This system uses a method for date and URL duplication removal based on Bloom Filter,provides distributed synchronization service through Zookeeper,and realizes web-page crawling through multithreading. Experiments indicate that this system has a high reliability and applicability,and improves the effi-ciency and accuracy on large data sets.

相似文献/References:

[1]张志宏,吴庆波,邵立松,等.基于飞腾平台TOE协议栈的设计与实现[J].计算机技术与发展,2014,24(07):1.
　ZHANG Zhi-hong,WU Qing-bo,SHAO Li-song,et al. Design and Implementation of TCP/IP Offload Engine Protocol Stack Based on FT Platform[J].,2014,24(05):1.
[2]梁文快,李毅. 改进的基因表达算法对航班优化排序问题研究[J].计算机技术与发展,2014,24(07):5.
　LIANG Wen-kuai,LI Yi. Research on Optimization of Flight Scheduling Problem Based on Improved Gene Expression Algorithm[J].,2014,24(05):5.
[3]黄静,王枫,谢志新,等. EAST文档管理系统的设计与实现[J].计算机技术与发展,2014,24(07):13.
　HUANG Jing,WANG Feng,XIE Zhi-xin,et al. Design and Implementation of EAST Document Management System[J].,2014,24(05):13.
[4]侯善江[],张代远[][][]. 基于样条权函数神经网络P2P流量识别方法[J].计算机技术与发展,2014,24(07):21.
　HOU Shan-jiang[],ZHANG Dai-yuan[][][]. P2P Traffic Identification Based on Spline Weight Function Neural Network[J].,2014,24(05):21.
[5]李璨,耿国华,李康,等. 一种基于三维模型的文物碎片线图生成方法[J].计算机技术与发展,2014,24(07):25.
　LI Can,GENG Guo-hua,LI Kang,et al. A Method of Obtaining Cultural Debris’ s Line Chart Based on Three-dimensional Model[J].,2014,24(05):25.
[6]翁鹤,皮德常. 混沌RBF神经网络异常检测算法[J].计算机技术与发展,2014,24(07):29.
　WENG He,PI De-chang. Chaotic RBF Neural Network Anomaly Detection Algorithm[J].,2014,24(05):29.
[7]刘茜[],荆晓远[],李文倩[],等. 基于流形学习的正交稀疏保留投影[J].计算机技术与发展,2014,24(07):34.
　LIU Qian[],JING Xiao-yuan[,LI Wen-qian[],et al. Orthogonal Sparsity Preserving Projections Based on Manifold Learning[J].,2014,24(05):34.
[8]尚福华,李想,巩淼. 基于模糊框架-产生式知识表示及推理研究[J].计算机技术与发展,2014,24(07):38.
　SHANG Fu-hua,LI Xiang,GONG Miao. Research on Knowledge Representation and Inference Based on Fuzzy Framework-production[J].,2014,24(05):38.
[9]叶偲,李良福,肖樟树. 一种去除运动目标重影的图像镶嵌方法研究[J].计算机技术与发展,2014,24(07):43.
　YE Si,LI Liang-fu,XIAO Zhang-shu. Research of an Image Mosaic Method for Removing Ghost of Moving Targets[J].,2014,24(05):43.
[10]余松平[][],蔡志平[],吴建进[],等. GSM-R信令监测选择录音系统设计与实现[J].计算机技术与发展,2014,24(07):47.
　YU Song-ping[][],CAI Zhi-ping[] WU Jian-jin[],GU Feng-zhi[]. Design and Implementation of an Optional Voice Recording System Based on GSM-R Signaling Monitoring[J].,2014,24(05):47.

更新日期/Last Update: 2015-06-23