MolAR: Bringing Chemical Structures to Life with Augmented Reality and Machine Learning

06 September 2021, Version 1
This content is a preprint and has not undergone peer review at the time of posting.


Visualizing three-dimensional molecular structures is crucial to understanding and predicting their chemical behavior. Existing visualization software, however, can be cumbersome to use, and, for many, hand-drawn skeletal structures remain the preferred method of chemical communication. Although convenient, the static, two-dimensional nature of these drawings can be misleading in conveying the molecule’s 3D structure, not to mention that dynamic movement is completely disregarded. Here, we combine machine learning and augmented reality (AR) to develop MolAR, an immersive mobile application for visualizing molecules in real-world scenes. The application uses deep learning to recognize hand-drawn hydrocarbons structures which it converts into interactive 3D molecules in AR. Users can also “hunt” for chemicals in food and drink to uncover molecules in their real-life environment. A variety of interesting molecules are pre-loaded into the application, and users can visualize molecules in PubChem by providing their name or SMILES string and proteins in the Protein Data Bank by providing their PDB ID. MolAR was designed to be used in both research and education settings, providing an almost barrierless platform to visualize and interact with 3D molecular structures in a uniquely immersive way.


augmented reality
machine learning
optical chemical structure recognition
chemical education

Supplementary materials

Supplementary Information
Supplementary Information for "MolAR: Bringing Chemical Structures to Life with Augmented Reality and Machine Learning"


Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy [opens in a new tab] - please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here [opens in a new tab] .
This site is protected by reCAPTCHA and the Google Privacy Policy [opens in a new tab] and Terms of Service [opens in a new tab] apply.