Murim Pak,
Kangsong Ro,
Choljin Wang,
Jusong Pak,
- Professor, Department of Mechanical Engineering, Kim Chaek University of Technology, Pyongyang, DPR Korea, Korea
- Professor, Department of Mechanical Engineering, Kim Chaek University of Technology, Pyongyang, DPR Korea, Korea
- Professor, Department of Mechanical Engineering, Kim Chaek University of Technology, Pyongyang, DPR Korea, Korea
- Professor, Department of Mechanical Engineering, Kim Chaek University of Technology, Pyongyang, DPR Korea, India
Abstract document.addEventListener(‘DOMContentLoaded’,function(){frmFrontForm.scrollToID(‘frm_container_abs_127901’);});Edit Abstract & Keyword
Path planning is fundamental and important task of mobile robot. There are many attempts to adopt reinforcement learning (RL) in mobile robot path planning. RL based path planning is effective in path planning of intelligent mobile robot, especially in unknown environment because it doesn’t require environmental information and finds optimal path through trial-and-error process. Q-learning is one of RL algorithm widely used in path planning of mobile robots. The main drawback of Q-learning is slow convergence speed. Several methods to improve convergence speed has been proposed but remarkable success has not been achieved. In this paper, some methods to improve convergence speed are proposed. This new algorithm includes; (1) new method of initializing Q-values before learning, (2) method of using searched obstacle information on improving learning efficiency, and (3) new way of adjusting greedy factor. First, The Q-values near the target are initialized considerably higher than the Q values far from the target. Second, we decreased the number of bumping into obstacles iteratively in the early stages of learning by modifying state values around obstacles by using partially searched obstacle information. Finally, we decreased greedy factor exponentially based on number of episodes reached at target correctly. In the early stages of learning, greedy factor is relatively high but it decreases dramatically as number of episodes reached at target correctly increases. It is expected that proposed algorithm has better convergence speed and learning efficiency than existing algorithms. Experiment results show that proposed algorithm has higher convergence speed and better learning efficiency than existing algorithms
Keywords: Reinforcement learning (RL), Q-learning, path planning, convergence speed
[This article belongs to International Journal of Advanced Robotics and Automation Technology (ijarat)]
Murim Pak, Kangsong Ro, Choljin Wang, Jusong Pak. Improvement of Convergence Speed of Q-learning based Path Planning Algorithm. International Journal of Advanced Robotics and Automation Technology. 2024; 02(02):19-28.
Murim Pak, Kangsong Ro, Choljin Wang, Jusong Pak. Improvement of Convergence Speed of Q-learning based Path Planning Algorithm. International Journal of Advanced Robotics and Automation Technology. 2024; 02(02):19-28. Available from: https://journals.stmjournals.com/ijarat/article=2024/view=0
References
document.addEventListener(‘DOMContentLoaded’,function(){frmFrontForm.scrollToID(‘frm_container_ref_127901’);});Edit
- Chao Y. and Wang H.: Developed Dijkstra shortest path search algorithm and simulation (2010). 2010 International Conference On Computer Design and Applications. IEEE. Qinhuangdao, China.
- Bingbing Xu, Rongfei Hao: Current Situation and Development of Robot Path Planning Technology. Electronic Technology & Software Engineering (2019)
- Toolika Arora, Yogita Gigras, Vijay Arora: Robotic Path Planning using Genetic Algorithm in Dynamic Environment. International Journal of Computer Applications (2014). Vol 89 (11).
- Xianxia Liang, Zhaoying Liu, Xueling Song, Yngkun Zhang: Research on Improved Artificial Potential Field Approach in Local Path Planning for Mobile Robot. Computer Simulation (2018)
- Phalgun Chintala, Rolf Dornberger, Thomas Hanne: Robotic Path Planning by Q learning and a performance Comparation with Classical Path Finding Algorithms (2022). IJMERR 2022 Vol.11(6): 373-378
- Li S., Xin X., and Lei Z.: Dynamic path planning of a mobile robot with improved Q-learning algorithm (2015). 2015 IEEE International Conference on Information and Automation. IEEE. Lijiang, China
- Wen S., Chen J., Li Z., Rad A. B., and Othman K. M.: Fuzzy Q-learning obstacle avoidance algorithm of humanoid robot in unknown environment (2018). 2018 37th Chinese Control Conference (CCC). IEEE. Wuhan, China
- Zhen Shi, Keyin Wang, Jianhui Zhang: Improved reinforcement learning path planning algorithm integrating prior knowledge. Plos One (2023).
| Volume | 02 |
| Issue | 02 |
| Received | 12/07/2024 |
| Accepted | 10/10/2024 |
| Published | 16/12/2024 |