FP—Growth算法MapReduce化研究-《计算机技术与发展》

文章信息/Info

Author(s):: LO Xue-ji; LI Long-shu; College of Computer Science and Technology, Anhui University

Keywords:: MapReduce; FP-Growth ; MR-FP; cloud computing ; distributed data mining

摘要:: 随着云计算概念的盛行，以及数据挖掘技术在分布式环境下的应用问题，该文献针对当前业界中流行的大规模并行计算模型MapReduce，将其引入数据挖掘领域关联规则算法的并行化改进中，提出基于FP-Growth算法并行化改进的MR—FP算法，为并行化关联规则挖掘提供节点可扩展、可容错、故障可恢复的运行保证。并通过案例分析得出系统在事务数呈数量级级别增长下仍可保持较高的性能。通过理论分析和案例实验表明，数据挖掘理论和方法在云计算环境下可以充分发挥能力，具有广阔的、有价值的研究空间

Abstract:: Nowadays the massively parallel computing model MapReduce is very popular in the current industry. It will introduce it into the improvement of association rules of data mining algorithms in parallelization, propose the improved MR-FP algorithm based on par alleled FP-Growth algorithm, and provide the parallelization association rules mining with node scalable, fault tolerance and operation. Draw a conclusion that the system can still maintain pretty high performance when the transactions are under the orders of magnitude level. The theoretical analysis and the case studies demonstrate that data mining theory and methods can show their full abilities based on the cloud computing. It deserves more valuable research