[1]刘晓峰 *,刘智斌,董兆安.基于记忆启发的强化学习方法研究[J].计算机技术与发展,2023,33(06):168-172.[doi:10. 3969 / j. issn. 1673-629X. 2023. 06. 025]
　LIU Xiao-feng *,LIU Zhi-bin,DONG Zhao-an.Research on Memory Heuristic Reinforcement Learning[J].,2023,33(06):168-172.[doi:10. 3969 / j. issn. 1673-629X. 2023. 06. 025]

点击复制

基于记忆启发的强化学习方法研究()

分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:

33

期数:

2023年06期

页码:

168-172

栏目:

人工智能

出版日期:

2023-06-10

文章信息/Info

Title:

Research on Memory Heuristic Reinforcement Learning

文章编号:

1673-629X(2023)06-0168-05

作者:

刘晓峰^{1 *} ; 刘智斌² ; 董兆安²

1. 曲阜师范大学 图书馆,山东 日照 276826;

2. 曲阜师范大学 计算机学院,山东 日照 276826

Author(s):

LIU Xiao-feng1 * ; LIU Zhi-bin2 ; DONG Zhao-an2

1. Liberary,Qufu Normal University,Rizhao 276826,China;

2. School of Computer Science,Qufu Normal University,Rizhao 276826,China

关键词:

强化学习; Q 学习; 启发式搜索; Shaping 函数; 路径规划

Keywords:

reinforcement learning; Q-learning; heuristic search; Shaping function; path planning

分类号:

TP301

DOI:

10. 3969 / j. issn. 1673-629X. 2023. 06. 025

摘要:

该文旨在研究人工智能领域的强化学习问题。 在处理优化问题的过程中,强化学习具有不依赖于模型信息的特点,在信息产业和生产领域逐步获得应用,并取得了较好的效果。 然而,传统的强化

学习算法通过随机探索获得优化行为,存在学习速度慢、收敛不及时的问题。 为了提高强化学习的效率,提出一种方法,让 Agent 利用自身学习得到的知识,指导和加速其以后的学习过程。 将 Q?

学习和启发式 Shaping 回报函数结合起来,利用记忆的知识加速了 Agent 的学习过程。 另外,证明了采用启发函数与不使用启发函数在策略优化上的一致性。 针对一个路径规划问题,采用了学

习过程中生成的势场函数作为启发函数,通过启发函数对强化学习的探索过程给予指导。 在实验中对该方法进行了验证,分析了采用不同参数带来的不同效果,并提出了一个解决死点问题的方法。 结果表明,该方法对强化学习过程有明显的加速作用,并能取得优化的搜索路径。

Abstract:

We aim to research on the reinforcement learning problems in the field of artificial intelligence. In the process of dealing withoptimization problems,reinforcement learning has the feature?

of not relying on model information,which gradually gains applications inthe areas of information and production,achieving better results. However,the traditional reinforcement learning algorithm obtains the optimization behavior by random exploration,which has the problems of slow learning speed and untimely convergence. In order to improvethe efficiency of reinforcement learning,we propose a method that allows an agent to use the knowledge obtained from its own learning toguide and accelerate its subsequent learning process. Q-learning and heuristic Shaping reward function are combined to accelerate thelearning process of the agent by utilizing the knowledge of memory. In addition, we demonstrate the consistency of?

using heuristicfunction and not using heuristic function in policy optimization. For a path planning problem, we adopt the potential field functiongenerated during the learning process as a heuristic function,which gives guidance to the exploration process of reinforcement learning.The method is validated in experiments,the different effects brought by using different parameters are analyzed,and a method to solve thedead point problem is proposed. The results show that adopting the proposed method has a significant acceleration effect on thereinforcement learning process and can obtain an optimized searching path.

常用功能