These are preliminary reports that have not been peer-reviewed. They should not be regarded as conclusive, guide clinical practice/health-related behavior, or be reported in news media as established information. For more information, please see our FAQs.
2019_04_15_Manuscript_Submitted.docx (4.71 MB)

An Exploration Strategy Improves the Diversity of de novo Ligands Using Deep Reinforcement Learning – A Case for the Adenosine A2A Receptor

revised on 16.04.2019, 09:12 and posted on 16.04.2019, 16:21 by Xuhan Liu, Kai Ye, Herman Van Vlijmen, Adriaan P. IJzerman, Gerard JP Van westen

Over the last five years deep learning has progressed tremendously in both image recognition and natural language processing. Now it is increasingly applied to other data rich fields. In drug discovery, recurrent neural networks (RNNs) have been shown to be an effective method to generate novel chemical structures in the form of SMILES. However, ligands generated by current methods have so far provided relatively low diversity and do not fully cover the whole chemical space occupied by known ligands. Here, we propose a new method (DrugEx) to discover de novo drug-like molecules. DrugEx is an RNN model (generator) trained through reinforcement learning which was integrated with a special exploration strategy. As a case study we applied our method to design ligands against the adenosine A2A receptor. From ChEMBL data, a machine learning model (predictor) was created to predict whether generated molecules are active or not. Based on this predictor as the reward function, the generator was trained by reinforcement learning without any further data. We then compared the performance of our method with two previously published methods, REINVENT and ORGANIC. We found that candidate molecules our model designed, and predicted to be active, had a larger chemical diversity, and better covered the chemical space of known ligands compared to the state-of-the-art.


Dutch Scientific Council (NWO) Applied and engineering Sciences (AES) VENI # 14410

Chinese Scholarship Council


Email Address of Submitting Author


Leiden Academic Center for Drug Research (Leiden University)



ORCID For Submitting Author


Declaration of Conflict of Interest

No conflict of interest