Reinforcement learning for in silico determination of adsorbate—substrate structures

Abstract Reinforcement learning (RL) methods have helped to define the state of the art in the field of modern artificial intelligence, mostly after the breakthrough involving AlphaGo and the discovery of novel algorithms. In this work, we present a RL method, based on Q‐learning, for the structural...

Full description

Bibliographic Details
Published in:Journal of Computational Chemistry
Main Authors: Lourenço, Maicon Pierre, Hostaš, Jiří, Bellinger, Colin, Tchagang, Alain, Salahub, Dennis R.
Other Authors: Fundação de Amparo à Pesquisa e Inovação do Espírito Santo, Conselho Nacional de Desenvolvimento Científico e Tecnológico, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, Natural Sciences and Engineering Research Council of Canada
Format: Article in Journal/Newspaper
Language:English
Published: Wiley 2024
Subjects:
Online Access:http://dx.doi.org/10.1002/jcc.27322
https://onlinelibrary.wiley.com/doi/pdf/10.1002/jcc.27322
id crwiley:10.1002/jcc.27322
record_format openpolar
spelling crwiley:10.1002/jcc.27322 2024-06-02T08:12:49+00:00 Reinforcement learning for in silico determination of adsorbate—substrate structures Lourenço, Maicon Pierre Hostaš, Jiří Bellinger, Colin Tchagang, Alain Salahub, Dennis R. Fundação de Amparo à Pesquisa e Inovação do Espírito Santo Conselho Nacional de Desenvolvimento Científico e Tecnológico Coordenação de Aperfeiçoamento de Pessoal de Nível Superior Natural Sciences and Engineering Research Council of Canada 2024 http://dx.doi.org/10.1002/jcc.27322 https://onlinelibrary.wiley.com/doi/pdf/10.1002/jcc.27322 en eng Wiley http://onlinelibrary.wiley.com/termsAndConditions#vor Journal of Computational Chemistry volume 45, issue 15, page 1289-1302 ISSN 0192-8651 1096-987X journal-article 2024 crwiley https://doi.org/10.1002/jcc.27322 2024-05-03T11:56:30Z Abstract Reinforcement learning (RL) methods have helped to define the state of the art in the field of modern artificial intelligence, mostly after the breakthrough involving AlphaGo and the discovery of novel algorithms. In this work, we present a RL method, based on Q‐learning, for the structural determination of adsorbate@substrate models in silico, where the minimization of the energy landscape resulting from adsorbate interactions with a substrate is made by actions on states (translations and rotations) chosen from an agent's policy. The proposed RL method is implemented in an early version of the reinforcement learning software for materials design and discovery (RLMaterial), developed in Python3.x. RLMaterial interfaces with deMon2k, DFTB+, ORCA, and Quantum Espresso codes to compute the adsorbate@substrate energies. The RL method was applied for the structural determination of (i) the amino acid glycine and (ii) 2‐amino‐acetaldehyde, both interacting with a boron nitride (BN) monolayer, (iii) host‐guest interactions between phenylboronic acid and β ‐cyclodextrin and (iv) ammonia on naphthalene. Density functional tight binding calculations were used to build the complex search surfaces with a reasonably low computational cost for systems (i)–(iii) and DFT for system (iv). Artificial neural network and gradient boosting regression techniques were employed to approximate the Q‐matrix or Q‐table for better decision making (policy) on next actions. Finally, we have developed a transfer‐learning protocol within the RL framework that allows learning from one chemical system and transferring the experience to another, as well as from different DFT or DFTB levels. Article in Journal/Newspaper Orca Wiley Online Library Journal of Computational Chemistry 45 15 1289 1302
institution Open Polar
collection Wiley Online Library
op_collection_id crwiley
language English
description Abstract Reinforcement learning (RL) methods have helped to define the state of the art in the field of modern artificial intelligence, mostly after the breakthrough involving AlphaGo and the discovery of novel algorithms. In this work, we present a RL method, based on Q‐learning, for the structural determination of adsorbate@substrate models in silico, where the minimization of the energy landscape resulting from adsorbate interactions with a substrate is made by actions on states (translations and rotations) chosen from an agent's policy. The proposed RL method is implemented in an early version of the reinforcement learning software for materials design and discovery (RLMaterial), developed in Python3.x. RLMaterial interfaces with deMon2k, DFTB+, ORCA, and Quantum Espresso codes to compute the adsorbate@substrate energies. The RL method was applied for the structural determination of (i) the amino acid glycine and (ii) 2‐amino‐acetaldehyde, both interacting with a boron nitride (BN) monolayer, (iii) host‐guest interactions between phenylboronic acid and β ‐cyclodextrin and (iv) ammonia on naphthalene. Density functional tight binding calculations were used to build the complex search surfaces with a reasonably low computational cost for systems (i)–(iii) and DFT for system (iv). Artificial neural network and gradient boosting regression techniques were employed to approximate the Q‐matrix or Q‐table for better decision making (policy) on next actions. Finally, we have developed a transfer‐learning protocol within the RL framework that allows learning from one chemical system and transferring the experience to another, as well as from different DFT or DFTB levels.
author2 Fundação de Amparo à Pesquisa e Inovação do Espírito Santo
Conselho Nacional de Desenvolvimento Científico e Tecnológico
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Natural Sciences and Engineering Research Council of Canada
format Article in Journal/Newspaper
author Lourenço, Maicon Pierre
Hostaš, Jiří
Bellinger, Colin
Tchagang, Alain
Salahub, Dennis R.
spellingShingle Lourenço, Maicon Pierre
Hostaš, Jiří
Bellinger, Colin
Tchagang, Alain
Salahub, Dennis R.
Reinforcement learning for in silico determination of adsorbate—substrate structures
author_facet Lourenço, Maicon Pierre
Hostaš, Jiří
Bellinger, Colin
Tchagang, Alain
Salahub, Dennis R.
author_sort Lourenço, Maicon Pierre
title Reinforcement learning for in silico determination of adsorbate—substrate structures
title_short Reinforcement learning for in silico determination of adsorbate—substrate structures
title_full Reinforcement learning for in silico determination of adsorbate—substrate structures
title_fullStr Reinforcement learning for in silico determination of adsorbate—substrate structures
title_full_unstemmed Reinforcement learning for in silico determination of adsorbate—substrate structures
title_sort reinforcement learning for in silico determination of adsorbate—substrate structures
publisher Wiley
publishDate 2024
url http://dx.doi.org/10.1002/jcc.27322
https://onlinelibrary.wiley.com/doi/pdf/10.1002/jcc.27322
genre Orca
genre_facet Orca
op_source Journal of Computational Chemistry
volume 45, issue 15, page 1289-1302
ISSN 0192-8651 1096-987X
op_rights http://onlinelibrary.wiley.com/termsAndConditions#vor
op_doi https://doi.org/10.1002/jcc.27322
container_title Journal of Computational Chemistry
container_volume 45
container_issue 15
container_start_page 1289
op_container_end_page 1302
_version_ 1800759370784440320