[1]刘梦青,王少辉. 基于蚁群算法的Storm集群资源感知任务调度[J].计算机技术与发展,2017,27(09):92-96.
　LIU Meng-qing,WANG Shao-hui. Research on Storm Resource-aware Task Scheduling with Ant Colony Algorithm[J].,2017,27(09):92-96.
点击复制

 基于蚁群算法的Storm集群资源感知任务调度()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
27
期数:
2017年09期
页码:
92-96
栏目:
智能、算法、系统工程
出版日期:
2017-09-10

文章信息/Info

Title:
 Research on Storm Resource-aware Task Scheduling with Ant Colony Algorithm
文章编号:
1673-629X（2017)09-0092-05
作者:
 刘梦青; 王少辉
 南京邮电大学 计算机学院；江苏省无线传感网高技术研究重点实验室
Author(s):
 LIU Meng-qing; WANG Shao-hui
关键词:
 ; Storm; 资源感知; 蚁群算法; 负载均衡
Keywords:
 ; Storm; resource-aware; ant colony algorithm; load balance
分类号:
TP39
文献标志码:
A
摘要:
 实时计算系统Storm是当前十分流行的开源流式系统,在处理流式数据时具有明显的优势,但也存在默认调度器在任务调度时难以将节点资源与任务需求相结合、节点资源利用率不高、节点内存不足以及网络堵塞等问题.为了解决这些问题,提出了一种基于蚁群算法的Storm集群资源感知任务调度算法及其实现方案.该算法将节点的资源动态变化表示为蚂蚁运动所需的信息素,将任务调度过程模拟为蚂蚁觅食过程,以此对任务调度进行优化,保证了Storm任务调度的有效性.实验结果表明,该算法能够找到与当前任务所需资源最匹配的节点,从而实现资源的合理分配;与默认调度相比,具有更优的任务调度效率、更少的平均处理时间和更高的集群吞吐量,有利于集群负载均衡,优化集群的性能.
Abstract:
 Storm is the popular open source real-time computing system,which has a great advantage in handling data stream. However, there are some problems in its default scheduler when scheduling tasks,such as difficultly in combining node resources and mission re-quirements,ineffectiveness in node resource utilization,lack of memory and network congestion and so on. In order to solve them,a re-source-aware scheduler based on ant colony algorithm and its implementation scheme is proposed,in which the dynamic changes of the node resource can be expressed as the pheromones of ant movement required and the task scheduling process is similar to ant foraging process,to optimize the task scheduling and ensure the effectiveness of the Storm task scheduling. Experimental results show that it has found the most suitable node for the current task and achieved the reasonable allocation of resources and that compared with the default scheduling,it has better task scheduling efficiency,less average processing time and higher throughput of the cluster,which can benefit the load balance and optimize the performance for the cluster.

相似文献/References:

[1]张志宏,吴庆波,邵立松,等.基于飞腾平台TOE协议栈的设计与实现[J].计算机技术与发展,2014,24(07):1.
　ZHANG Zhi-hong,WU Qing-bo,SHAO Li-song,et al. Design and Implementation of TCP/IP Offload Engine Protocol Stack Based on FT Platform[J].,2014,24(09):1.
[2]梁文快,李毅. 改进的基因表达算法对航班优化排序问题研究[J].计算机技术与发展,2014,24(07):5.
　LIANG Wen-kuai,LI Yi. Research on Optimization of Flight Scheduling Problem Based on Improved Gene Expression Algorithm[J].,2014,24(09):5.
[3]黄静,王枫,谢志新,等. EAST文档管理系统的设计与实现[J].计算机技术与发展,2014,24(07):13.
　HUANG Jing,WANG Feng,XIE Zhi-xin,et al. Design and Implementation of EAST Document Management System[J].,2014,24(09):13.
[4]侯善江[],张代远[][][]. 基于样条权函数神经网络P2P流量识别方法[J].计算机技术与发展,2014,24(07):21.
　HOU Shan-jiang[],ZHANG Dai-yuan[][][]. P2P Traffic Identification Based on Spline Weight Function Neural Network[J].,2014,24(09):21.
[5]李璨,耿国华,李康,等. 一种基于三维模型的文物碎片线图生成方法[J].计算机技术与发展,2014,24(07):25.
　LI Can,GENG Guo-hua,LI Kang,et al. A Method of Obtaining Cultural Debris’ s Line Chart Based on Three-dimensional Model[J].,2014,24(09):25.
[6]翁鹤,皮德常. 混沌RBF神经网络异常检测算法[J].计算机技术与发展,2014,24(07):29.
　WENG He,PI De-chang. Chaotic RBF Neural Network Anomaly Detection Algorithm[J].,2014,24(09):29.
[7]刘茜[],荆晓远[],李文倩[],等. 基于流形学习的正交稀疏保留投影[J].计算机技术与发展,2014,24(07):34.
　LIU Qian[],JING Xiao-yuan[,LI Wen-qian[],et al. Orthogonal Sparsity Preserving Projections Based on Manifold Learning[J].,2014,24(09):34.
[8]尚福华,李想,巩淼. 基于模糊框架-产生式知识表示及推理研究[J].计算机技术与发展,2014,24(07):38.
　SHANG Fu-hua,LI Xiang,GONG Miao. Research on Knowledge Representation and Inference Based on Fuzzy Framework-production[J].,2014,24(09):38.
[9]叶偲,李良福,肖樟树. 一种去除运动目标重影的图像镶嵌方法研究[J].计算机技术与发展,2014,24(07):43.
　YE Si,LI Liang-fu,XIAO Zhang-shu. Research of an Image Mosaic Method for Removing Ghost of Moving Targets[J].,2014,24(09):43.
[10]余松平[][],蔡志平[],吴建进[],等. GSM-R信令监测选择录音系统设计与实现[J].计算机技术与发展,2014,24(07):47.
　YU Song-ping[][],CAI Zhi-ping[] WU Jian-jin[],GU Feng-zhi[]. Design and Implementation of an Optional Voice Recording System Based on GSM-R Signaling Monitoring[J].,2014,24(09):47.
[11]张发扬,李玲娟,陈煜. VFDT算法基于Storm平台的实现方案[J].计算机技术与发展,2016,26(09):192.
　ZHANG Fa-yang,LI Ling-juan,CHEN Yu. Implementation Scheme of VFDT Algorithm on Storm Platform[J].,2016,26(09):192.
[12]孙杜靖,李玲娟. 面向Redis的数据序列化算法研究[J].计算机技术与发展,2017,27(05):77.
　SUN Du-jing,LI Ling-juan. Investigation on Data Serialization Algorithm for Redis[J].,2017,27(09):77.
[13]孙杜靖,李玲娟,马可. 面向流数据的DPFP-Stream算法的设计与实现[J].计算机技术与发展,2017,27(07):29.
　SUN Du-jing,LI Ling-juan,MA Ke. Realization and Implementation of Distributed Parallel Mining of Frequent Patterns for Data Streams[J].,2017,27(09):29.
[14]万新贵,李玲娟,马可. 分布式数据流聚类算法及其基于Storm的实现[J].计算机技术与发展,2017,27(07):150.
　WAN Xin-gui,LI Ling-juan,MA Ke. Distributed Data Stream Clustering Algorithm and Its Implementation with Storm[J].,2017,27(09):150.

更新日期/Last Update: 2017-10-20