Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios
Collision avoidance is one of the most fundamental and challenging aspects of socially aware robot navigation. Recently, numerous works based on reinforcement learning (RL) have been proposed for collision avoidance in experimental settings. However, reinforcement learning approaches face two challe...
Published in: | IEEE Access |
---|---|
Main Authors: | , |
Format: | Article in Journal/Newspaper |
Language: | English |
Published: |
IEEE
2024
|
Subjects: | |
Online Access: | https://doi.org/10.1109/ACCESS.2024.3448292 https://doaj.org/article/a36ac01ef97c494ba5b27a06f1f1ea82 |
id |
ftdoajarticles:oai:doaj.org/article:a36ac01ef97c494ba5b27a06f1f1ea82 |
---|---|
record_format |
openpolar |
spelling |
ftdoajarticles:oai:doaj.org/article:a36ac01ef97c494ba5b27a06f1f1ea82 2024-09-15T18:28:58+00:00 Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios Jian Ban Gongyan Li 2024-01-01T00:00:00Z https://doi.org/10.1109/ACCESS.2024.3448292 https://doaj.org/article/a36ac01ef97c494ba5b27a06f1f1ea82 EN eng IEEE https://ieeexplore.ieee.org/document/10643540/ https://doaj.org/toc/2169-3536 2169-3536 doi:10.1109/ACCESS.2024.3448292 https://doaj.org/article/a36ac01ef97c494ba5b27a06f1f1ea82 IEEE Access, Vol 12, Pp 116956-116967 (2024) Collision avoidance reinforcement learning machine learning mobile robots Electrical engineering. Electronics. Nuclear engineering TK1-9971 article 2024 ftdoajarticles https://doi.org/10.1109/ACCESS.2024.3448292 2024-09-02T15:34:37Z Collision avoidance is one of the most fundamental and challenging aspects of socially aware robot navigation. Recently, numerous works based on reinforcement learning (RL) have been proposed for collision avoidance in experimental settings. However, reinforcement learning approaches face two challenges: managing lengthy training processes and generalizing in volatile environments. These issues arise because reinforcement learning must initiate training whenever it encounters new environments or environmental changes. In this work, we present an algorithm that executes without prior training and adaptively optimizes itself when the environment changes. Its training is its execution. The novelty of this work is twofold: 1. developing a fluctuating epsilon model, which is entirely new and designed from scratch. It adjusts the exploration probability based on the performance of the algorithm. 2. developing a velocity obstacle-based exploration model, which innovatively combines Reinforcement Learning (RL) with the Velocity Obstacles (VO) method by replacing random exploration in RL with the classic collision avoidance algorithm, the Optimal Reciprocal Collision Avoidance (ORCA) algorithm, to create our innovative exploration model. Our proposed algorithm is evaluated in simulation scenarios. Experimental results demonstrate that our comprehensive model outperforms the state-of-the-art (SOTA) baseline algorithms, achieving a 3.23% increase in success rate. Moreover, our model continues to excel even when the environment changes, consistently improving its performance. Specifically, after environmental changes, our model’s success rate exceeds that of the baseline by 17.49%. Article in Journal/Newspaper Orca Directory of Open Access Journals: DOAJ Articles IEEE Access 12 116956 116967 |
institution |
Open Polar |
collection |
Directory of Open Access Journals: DOAJ Articles |
op_collection_id |
ftdoajarticles |
language |
English |
topic |
Collision avoidance reinforcement learning machine learning mobile robots Electrical engineering. Electronics. Nuclear engineering TK1-9971 |
spellingShingle |
Collision avoidance reinforcement learning machine learning mobile robots Electrical engineering. Electronics. Nuclear engineering TK1-9971 Jian Ban Gongyan Li Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios |
topic_facet |
Collision avoidance reinforcement learning machine learning mobile robots Electrical engineering. Electronics. Nuclear engineering TK1-9971 |
description |
Collision avoidance is one of the most fundamental and challenging aspects of socially aware robot navigation. Recently, numerous works based on reinforcement learning (RL) have been proposed for collision avoidance in experimental settings. However, reinforcement learning approaches face two challenges: managing lengthy training processes and generalizing in volatile environments. These issues arise because reinforcement learning must initiate training whenever it encounters new environments or environmental changes. In this work, we present an algorithm that executes without prior training and adaptively optimizes itself when the environment changes. Its training is its execution. The novelty of this work is twofold: 1. developing a fluctuating epsilon model, which is entirely new and designed from scratch. It adjusts the exploration probability based on the performance of the algorithm. 2. developing a velocity obstacle-based exploration model, which innovatively combines Reinforcement Learning (RL) with the Velocity Obstacles (VO) method by replacing random exploration in RL with the classic collision avoidance algorithm, the Optimal Reciprocal Collision Avoidance (ORCA) algorithm, to create our innovative exploration model. Our proposed algorithm is evaluated in simulation scenarios. Experimental results demonstrate that our comprehensive model outperforms the state-of-the-art (SOTA) baseline algorithms, achieving a 3.23% increase in success rate. Moreover, our model continues to excel even when the environment changes, consistently improving its performance. Specifically, after environmental changes, our model’s success rate exceeds that of the baseline by 17.49%. |
format |
Article in Journal/Newspaper |
author |
Jian Ban Gongyan Li |
author_facet |
Jian Ban Gongyan Li |
author_sort |
Jian Ban |
title |
Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios |
title_short |
Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios |
title_full |
Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios |
title_fullStr |
Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios |
title_full_unstemmed |
Training Is Execution: A Reinforcement Learning-Based Collision Avoidance Algorithm for Volatile Scenarios |
title_sort |
training is execution: a reinforcement learning-based collision avoidance algorithm for volatile scenarios |
publisher |
IEEE |
publishDate |
2024 |
url |
https://doi.org/10.1109/ACCESS.2024.3448292 https://doaj.org/article/a36ac01ef97c494ba5b27a06f1f1ea82 |
genre |
Orca |
genre_facet |
Orca |
op_source |
IEEE Access, Vol 12, Pp 116956-116967 (2024) |
op_relation |
https://ieeexplore.ieee.org/document/10643540/ https://doaj.org/toc/2169-3536 2169-3536 doi:10.1109/ACCESS.2024.3448292 https://doaj.org/article/a36ac01ef97c494ba5b27a06f1f1ea82 |
op_doi |
https://doi.org/10.1109/ACCESS.2024.3448292 |
container_title |
IEEE Access |
container_volume |
12 |
container_start_page |
116956 |
op_container_end_page |
116967 |
_version_ |
1810470393362776064 |