Theoretical and Computational Chemistry

Compositionally-Restricted Attention-Based Network for Materials Property Prediction

Anthony Wang Technische Universität Berlin, Chair of Advanced Ceramic Materials


In this paper, we demonstrate a novel application of the Transformer self-attention mechanism. Our network, the Compositionally-Restricted Attention-Based network, referred to as CrabNet, explores the area of structure-agnostic materials property predictions when only a chemical formula is provided.
Our results show that CrabNet's performance matches or exceeds current best practice methods on nearly all of 28 total benchmark datasets. We also demonstrate how CrabNet's architecture lends itself towards model interpretability by showing different visualization approaches that are made possible by its design.
We feel confident that CrabNet, and its attention-based framework, will be of keen interest to future materials informatics researchers.

For trained model weights, please see:

Version notes

Update manuscript revision addressing reviewer comments


Thumbnail image of CrabNet revision2_submitted.pdf
download asset CrabNet revision2_submitted.pdf 5 MB [opens in a new tab]

Supplementary material

Thumbnail image of CrabNet - SI
download asset CrabNet - SI 14 MB [opens in a new tab]
CrabNet - SI revision2

Supplementary weblinks