[1]范孟可,王攀. 基于Hadoop的固网宽带终端识别技术研究和实现[J].计算机技术与发展,2017,27(11):171-175.
　FAN Meng-ke,WANG Pan. Research and Implementation of Terminal Identification Technology of Fixed-line Broadband Based on Hadoop[J].,2017,27(11):171-175.
点击复制

 基于Hadoop的固网宽带终端识别技术研究和实现()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
27
期数:
2017年11期
页码:
171-175
栏目:
应用开发研究
出版日期:
2017-11-10

文章信息/Info

Title:
 Research and Implementation of Terminal Identification Technology of Fixed-line Broadband Based on Hadoop
文章编号:
1673-629X（2017)11-0171-05
作者:
 范孟可; 王攀
 南京邮电大学 物联网学院
Author(s):
 FAN Meng-ke; WANG Pan
关键词:
 ; 终端识别; Hadoop; UserDefinedFunction(UDF) ; 分布式爬虫; 固网宽带; 大数据运营
Keywords:
 ; terminal identification; Hadoop; User Defined Function ( UDF); distributed crawler; fixed-line broadband; big data operations
分类号:
TP31
文献标志码:
A
摘要:
 
随着大数据时代的来临,大数据在各个行业应用越来越广泛.大数据在运营商行业的应用也很普遍,但同时也遇到了很多技术问题,其中家庭画像的塑造是运营商大数据的一个核心问题.如何提取和识别固网宽带下的终端类型是一个有待解决的问题.不像移动网,固网宽带由于没有信令通道,所以不携带任何准确的终端信息,因而对固网下的终端类型识别比较困难.传统方法都是采用解析和匹配HTTP GET报文中的UA字段进行识别.但这种方法由于UA的非标准化,以及终端数量和种类众多的缘故而导致终端类型的识别准确率低下.文中采用Hadoop框架,利用Hive中UDF的方法,结合分布式爬虫获取终端库,可以更加快速准确地识别出用户上网终端信息.实验结果表明,终端识别准确率可以达到92%以上,相比传统方法有了大幅提升.
Abstract:
 With the coming of the era of big data,big data is more and more widely applied in various industries,which is also done in op-erators industry,but many technical problems are found simultaneously,of which family portraits of shaping is a core for operators of large data. How to extract and identify the terminal type of fixed-line broadband is a problem needed to be solved. Unlike mobile net-work,fixed-line broadband don’ t take any accurate terminal information due to lack of signaling channel,so it is hard to conduct termi-nal type identification in fixed-line. The traditional method adopts UA fields of HTTP GET message parsing and matching for identifica-tion,but it is low in identification accuracy because of UA non-standardized and the large amounts of terminal number and varieties. Based on the Hadoop framework,the UDF of Hive is used,and combined with the distributed crawler for obtainment of terminal library, the user terminal information online is identified more quickly and accurately. According to the experiment,the accuracy of terminal iden-tification can reach above 92%,a substantial increase compared with the traditional method.

相似文献/References:

[1]张志宏,吴庆波,邵立松,等.基于飞腾平台TOE协议栈的设计与实现[J].计算机技术与发展,2014,24(07):1.
　ZHANG Zhi-hong,WU Qing-bo,SHAO Li-song,et al. Design and Implementation of TCP/IP Offload Engine Protocol Stack Based on FT Platform[J].,2014,24(11):1.
[2]梁文快,李毅. 改进的基因表达算法对航班优化排序问题研究[J].计算机技术与发展,2014,24(07):5.
　LIANG Wen-kuai,LI Yi. Research on Optimization of Flight Scheduling Problem Based on Improved Gene Expression Algorithm[J].,2014,24(11):5.
[3]黄静,王枫,谢志新,等. EAST文档管理系统的设计与实现[J].计算机技术与发展,2014,24(07):13.
　HUANG Jing,WANG Feng,XIE Zhi-xin,et al. Design and Implementation of EAST Document Management System[J].,2014,24(11):13.
[4]侯善江[],张代远[][][]. 基于样条权函数神经网络P2P流量识别方法[J].计算机技术与发展,2014,24(07):21.
　HOU Shan-jiang[],ZHANG Dai-yuan[][][]. P2P Traffic Identification Based on Spline Weight Function Neural Network[J].,2014,24(11):21.
[5]李璨,耿国华,李康,等. 一种基于三维模型的文物碎片线图生成方法[J].计算机技术与发展,2014,24(07):25.
　LI Can,GENG Guo-hua,LI Kang,et al. A Method of Obtaining Cultural Debris’ s Line Chart Based on Three-dimensional Model[J].,2014,24(11):25.
[6]翁鹤,皮德常. 混沌RBF神经网络异常检测算法[J].计算机技术与发展,2014,24(07):29.
　WENG He,PI De-chang. Chaotic RBF Neural Network Anomaly Detection Algorithm[J].,2014,24(11):29.
[7]刘茜[],荆晓远[],李文倩[],等. 基于流形学习的正交稀疏保留投影[J].计算机技术与发展,2014,24(07):34.
　LIU Qian[],JING Xiao-yuan[,LI Wen-qian[],et al. Orthogonal Sparsity Preserving Projections Based on Manifold Learning[J].,2014,24(11):34.
[8]尚福华,李想,巩淼. 基于模糊框架-产生式知识表示及推理研究[J].计算机技术与发展,2014,24(07):38.
　SHANG Fu-hua,LI Xiang,GONG Miao. Research on Knowledge Representation and Inference Based on Fuzzy Framework-production[J].,2014,24(11):38.
[9]叶偲,李良福,肖樟树. 一种去除运动目标重影的图像镶嵌方法研究[J].计算机技术与发展,2014,24(07):43.
　YE Si,LI Liang-fu,XIAO Zhang-shu. Research of an Image Mosaic Method for Removing Ghost of Moving Targets[J].,2014,24(11):43.
[10]余松平[][],蔡志平[],吴建进[],等. GSM-R信令监测选择录音系统设计与实现[J].计算机技术与发展,2014,24(07):47.
　YU Song-ping[][],CAI Zhi-ping[] WU Jian-jin[],GU Feng-zhi[]. Design and Implementation of an Optional Voice Recording System Based on GSM-R Signaling Monitoring[J].,2014,24(11):47.

更新日期/Last Update: 2018-01-02