We present a model-agnostic method that gives structure-activity explanations of black-box models. Machine learning models are now common for molecular property prediction and chemical design. They typically are black boxes -- having no explanation for predictions. Our method uses surrogate models to attribute predictions to chemical descriptors and molecular substructures, independent of the black box model inputs. Our approach provides explanations consistent with chemical reasoning, like connecting existence of a functional group or molecular polarity.
Fixed problem with MACCS descriptors in figures 5-6