Real-World Molecular Out-Of-Distribution: Specification and Investigation

26 July 2023, Version 2
This content is a preprint and has not undergone peer review at the time of posting.


This study presents a rigorous framework for investigating Molecular Out-Of-Distribution (MOOD) generalization in drug discovery. The concept of MOOD is first clarified through a problem specification that demonstrates how the covariate shifts encountered during real-world deployment can be characterized by the distribution of sample distances to the training set. We find that these shifts can cause performance to drop by up to 60% and uncertainty calibration by up to 40%. This leads us to propose a splitting protocol that aims to close the gap between deployment and testing. Then, using this protocol, a thorough investigation is conducted to assess the impact of model design, model selection and dataset characteristics on MOOD performance and uncertainty calibration. We find that appropriate representations and algorithms with built-in uncertainty estimation are crucial to improve performance and uncertainty calibration. This study sets itself apart by its exhaustiveness and opens an exciting avenue to benchmark meaningful, algorithmic progress in molecular scoring. All related code can be found on Github at


Molecular Scoring
Applicability Domain
Drug Discovery

Supplementary materials

MOOD: Supplementary Material
Provides a variety of additional figures to support the results from the main text.

Supplementary weblinks


Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy [opens in a new tab] - please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here [opens in a new tab] .
This site is protected by reCAPTCHA and the Google Privacy Policy [opens in a new tab] and Terms of Service [opens in a new tab] apply.
Comment number 1, Rafael Lameiro: Dec 04, 2023, 20:39

Excellent work, although I found it a bit confusing because less known concepts are discussed before they are defined. To name a few: uncertainty calibration, covariate shifts, IID/OOD algorithms, RCT. Moving the Methods section up, after the Introduction, would greatly improve it, IMO.