Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing
Reinforcement Learning (RL) methods have been successfully demonstrated in robotic tasks, however, their application to continuous state-action systems with fast dynamics is challenging. In this work, we investigate RL solutions for the autonomous racing problem on the ORCA miniature race car platfo...
Main Author: | |
---|---|
Other Authors: | , , , |
Language: | English |
Published: |
ETH Zurich
2020
|
Subjects: | |
Online Access: | https://hdl.handle.net/20.500.11850/486540 https://doi.org/10.3929/ethz-b-000486540 |
id |
ftethz:oai:www.research-collection.ethz.ch:20.500.11850/486540 |
---|---|
record_format |
openpolar |
spelling |
ftethz:oai:www.research-collection.ethz.ch:20.500.11850/486540 2023-05-15T17:53:30+02:00 Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing Chisari, Eugenio Rupenyan, Alisa Liniger, Alexander Balula, Samuel Lygeros, John 2020-03-23 application/application/pdf https://hdl.handle.net/20.500.11850/486540 https://doi.org/10.3929/ethz-b-000486540 en eng ETH Zurich http://hdl.handle.net/20.500.11850/486540 doi:10.3929/ethz-b-000486540 info:eu-repo/semantics/openAccess http://rightsstatements.org/page/InC-NC/1.0/ In Copyright - Non-Commercial Use Permitted 2020 ftethz https://doi.org/20.500.11850/486540 https://doi.org/10.3929/ethz-b-000486540 2022-04-25T14:27:28Z Reinforcement Learning (RL) methods have been successfully demonstrated in robotic tasks, however, their application to continuous state-action systems with fast dynamics is challenging. In this work, we investigate RL solutions for the autonomous racing problem on the ORCA miniature race car platform. When training a deep neural network policy using RL methods only using simulations, we observe poor performance, due to model mismatch also known as reality gap. We propose three different methods to reduce this gap, first we propose a policy regularization in the policy optimization step, second, we use model randomization. These two methods allow learning a policy that can race the car without any real environment interactions. Our third method improves this policy, by running the RL algorithm online while driving the car. The achieved performance on the ORCA platform is comparable to that achieved previously by a model-based controller, in terms of lap time, and improved with respect to track constraint violations. Other/Unknown Material Orca ETH Zürich Research Collection |
institution |
Open Polar |
collection |
ETH Zürich Research Collection |
op_collection_id |
ftethz |
language |
English |
description |
Reinforcement Learning (RL) methods have been successfully demonstrated in robotic tasks, however, their application to continuous state-action systems with fast dynamics is challenging. In this work, we investigate RL solutions for the autonomous racing problem on the ORCA miniature race car platform. When training a deep neural network policy using RL methods only using simulations, we observe poor performance, due to model mismatch also known as reality gap. We propose three different methods to reduce this gap, first we propose a policy regularization in the policy optimization step, second, we use model randomization. These two methods allow learning a policy that can race the car without any real environment interactions. Our third method improves this policy, by running the RL algorithm online while driving the car. The achieved performance on the ORCA platform is comparable to that achieved previously by a model-based controller, in terms of lap time, and improved with respect to track constraint violations. |
author2 |
Rupenyan, Alisa Liniger, Alexander Balula, Samuel Lygeros, John |
author |
Chisari, Eugenio |
spellingShingle |
Chisari, Eugenio Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing |
author_facet |
Chisari, Eugenio |
author_sort |
Chisari, Eugenio |
title |
Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing |
title_short |
Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing |
title_full |
Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing |
title_fullStr |
Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing |
title_full_unstemmed |
Learning from Simulation, Racing in Reality: Sim2Real Methods for Autonomous Racing |
title_sort |
learning from simulation, racing in reality: sim2real methods for autonomous racing |
publisher |
ETH Zurich |
publishDate |
2020 |
url |
https://hdl.handle.net/20.500.11850/486540 https://doi.org/10.3929/ethz-b-000486540 |
genre |
Orca |
genre_facet |
Orca |
op_relation |
http://hdl.handle.net/20.500.11850/486540 doi:10.3929/ethz-b-000486540 |
op_rights |
info:eu-repo/semantics/openAccess http://rightsstatements.org/page/InC-NC/1.0/ In Copyright - Non-Commercial Use Permitted |
op_doi |
https://doi.org/20.500.11850/486540 https://doi.org/10.3929/ethz-b-000486540 |
_version_ |
1766161212070952960 |