[1]李聪颖[],王瑞刚[],梁小江[]. 基于Hadoop的交互式大数据分析查询处理方法[J].计算机技术与发展,2016,26(08):134-137.
　LI Cong-ying[],WANG Rui-gang[],LIANG Xiao-jiang[]. An Interactive Processing Method of Analysis and Query for Big Data Based on Hadoop[J].,2016,26(08):134-137.
点击复制

 基于Hadoop的交互式大数据分析查询处理方法()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
26
期数:
2016年08期
页码:
134-137
栏目:
应用开发研究
出版日期:
2016-08-10

文章信息/Info

Title:
 An Interactive Processing Method of Analysis and Query for Big Data Based on Hadoop
文章编号:
1673-629X（2016)08-0134-04
作者:
 李聪颖[1]; 王瑞刚[1]; 梁小江[2]
 1.西安邮电大学；2.陕西省信息化工程研究院
Author(s):
 LI Cong-ying[1]; WANG Rui-gang[1]; LIANG Xiao-jiang[2]
关键词:
 ; Hadoop集群; 大数据处理; 交互式查询; 快速; SQL
Keywords:
 ; Hadoop clustering; big data processing; interactive query; fast; SQL
分类号:
TP302.1
文献标志码:
A
摘要:
 基于Hadoop的交互式大数据分析查询处理方法旨在快速分析查询大数据集的信息，最重要的特征就是查询速度快。该方法能够运行在上千节点的集群上，适于半结构化/嵌套数据的分析、兼容现有的SQL环境和Apache Hive。文中主要利用此方法实现连接HDFS、Hive以及Hbase进行查询测试，还完成了同时从不同数据源上关联查询数据。在同一Ha-doop集群环境中，将该方法与Spark SQL对于10万、20万、50万、100万、500万条数据进行查询速度对比测试。经过多次实验后得出，基于Hadoop的交互式大数据分析查询处理方法速度快、效率高，能够帮助企业用户快速、高效地进行Hadoop数据查询和企业级大数据分析。
Abstract:
 An interactive processing method of analysis and query of big data based on Hadoop aims to analyze and query large data fast, whose important feature is the rapid query speed. The method is able to run on a cluster with thousands of nodes,suitable for analyzing semi-structured or nested data,combining with existing SQL environment and Apache Hive. The main purpose is to use the method to connect HDFS,Hive and Hbase for query,also achieving to query data from different data sources. Furthermore,in the same Hadoop clus-tering environment,the method and Spark SQL is compared in the query speed for data with 100 000,200 000,500 000,one million and five million. Several experiments show the method is fast and efficient,and enables business users to query data and analyze enterprise Ha-doop big data quickly and efficiently.

相似文献/References:

[1]张志宏,吴庆波,邵立松,等.基于飞腾平台TOE协议栈的设计与实现[J].计算机技术与发展,2014,24(07):1.
　ZHANG Zhi-hong,WU Qing-bo,SHAO Li-song,et al. Design and Implementation of TCP/IP Offload Engine Protocol Stack Based on FT Platform[J].,2014,24(08):1.
[2]梁文快,李毅. 改进的基因表达算法对航班优化排序问题研究[J].计算机技术与发展,2014,24(07):5.
　LIANG Wen-kuai,LI Yi. Research on Optimization of Flight Scheduling Problem Based on Improved Gene Expression Algorithm[J].,2014,24(08):5.
[3]黄静,王枫,谢志新,等. EAST文档管理系统的设计与实现[J].计算机技术与发展,2014,24(07):13.
　HUANG Jing,WANG Feng,XIE Zhi-xin,et al. Design and Implementation of EAST Document Management System[J].,2014,24(08):13.
[4]侯善江[],张代远[][][]. 基于样条权函数神经网络P2P流量识别方法[J].计算机技术与发展,2014,24(07):21.
　HOU Shan-jiang[],ZHANG Dai-yuan[][][]. P2P Traffic Identification Based on Spline Weight Function Neural Network[J].,2014,24(08):21.
[5]李璨,耿国华,李康,等. 一种基于三维模型的文物碎片线图生成方法[J].计算机技术与发展,2014,24(07):25.
　LI Can,GENG Guo-hua,LI Kang,et al. A Method of Obtaining Cultural Debris’ s Line Chart Based on Three-dimensional Model[J].,2014,24(08):25.
[6]翁鹤,皮德常. 混沌RBF神经网络异常检测算法[J].计算机技术与发展,2014,24(07):29.
　WENG He,PI De-chang. Chaotic RBF Neural Network Anomaly Detection Algorithm[J].,2014,24(08):29.
[7]刘茜[],荆晓远[],李文倩[],等. 基于流形学习的正交稀疏保留投影[J].计算机技术与发展,2014,24(07):34.
　LIU Qian[],JING Xiao-yuan[,LI Wen-qian[],et al. Orthogonal Sparsity Preserving Projections Based on Manifold Learning[J].,2014,24(08):34.
[8]尚福华,李想,巩淼. 基于模糊框架-产生式知识表示及推理研究[J].计算机技术与发展,2014,24(07):38.
　SHANG Fu-hua,LI Xiang,GONG Miao. Research on Knowledge Representation and Inference Based on Fuzzy Framework-production[J].,2014,24(08):38.
[9]叶偲,李良福,肖樟树. 一种去除运动目标重影的图像镶嵌方法研究[J].计算机技术与发展,2014,24(07):43.
　YE Si,LI Liang-fu,XIAO Zhang-shu. Research of an Image Mosaic Method for Removing Ghost of Moving Targets[J].,2014,24(08):43.
[10]余松平[][],蔡志平[],吴建进[],等. GSM-R信令监测选择录音系统设计与实现[J].计算机技术与发展,2014,24(07):47.
　YU Song-ping[][],CAI Zhi-ping[] WU Jian-jin[],GU Feng-zhi[]. Design and Implementation of an Optional Voice Recording System Based on GSM-R Signaling Monitoring[J].,2014,24(08):47.
[11]秦军[],冯亮亮[],孙蒙[]. 基于异构Hadoop集群的负载均衡策略研究[J].计算机技术与发展,2017,27(06):110.
　QIN Jun[],FENG Liang-liang[],SUN Meng[]. Research on Load Balancing Strategy with Heterogeneous Hadoop Clustering[J].,2017,27(08):110.

更新日期/Last Update: 2016-09-29