CrabNet for explainable deep learning in materials science: bridging the gap between academia and industry

21 October 2021, Version 1
This content is a preprint and has not undergone peer review at the time of posting.

Abstract

Despite recent breakthroughs in deep learning for materials informatics, there exists a disparity between their popularity in academic research and their limited adoption in the industry. A significant contributor to this “interpretability-adoption gap” is the prevalence of black-box models and the lack of built-in methods for model interpretation. While established methods for evaluating model performance exist, an intuitive understanding of the modeling and decision-making processes in models is nonetheless desired in many cases. In this work, we demonstrate several ways of incorporating model interpretability to the structure-agnostic Compositionally Restricted Attention-Based network, CrabNet. We show that CrabNet learns meaningful, material property-specific element representations based solely on the data with no additional supervision. These element representations can then be used to explore element identity, similarity, behavior, and interactions within different chemical environments. Chemical compounds can also be uniquely represented and examined to reveal clear structures and trends within the chemical space. Additionally, visualizations of the attention mechanism can be used in conjunction to further understand the modeling process, identify potential modeling or dataset errors, and hint at further chemical insights leading to a better understanding of the phenomena governing material properties. We feel confident that the interpretability methods introduced in this work for CrabNet will be of keen interest to materials informatics researchers as well as industrial practitioners alike.

Keywords

Materials informatics
Deep learning
Self-attention
Interpretability
Explainable AI
XAI

Supplementary materials

Title
Description
Actions
Title
Supplementary Information
Description
Supplementary Information file, including plots
Actions
Title
ESM1 - Element correlations
Description
ESM1 - Plots of element correlations from CBFV feature sets (static and interactive plots)
Actions
Title
ESM2 - Element correlations in CrabNet & HotCrab
Description
ESM2 - Plots of element correlations from CrabNet & HotCrab (static and interactive plots)
Actions
Title
ESM3 - Element prevalence and Shannon entropy
Description
ESM3 - Plots of element prevalence and Shannon entropy as calculated from the datasets
Actions
Title
ESM4 - Element vector representations
Description
ESM4 - Plots of element vector representations of silicon and chromium (static and interactive plots)
Actions
Title
ESM5 - Attention videos
Description
ESM5 - Example attention videos obtained during model training
Actions

Comments

Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy [opens in a new tab] - please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here [opens in a new tab] .
This site is protected by reCAPTCHA and the Google Privacy Policy [opens in a new tab] and Terms of Service [opens in a new tab] apply.