An artificial intelligence's interpretation of complex high-resolution in situ transmission electron microscopy data

12 June 2023, Version 2
This content is a preprint and has not undergone peer review at the time of posting.


In situ transmission electron microscopy (TEM) has enabled researchers to visualize complicated nano- and atomic-scale processes with sub-Angstrom spatial resolution and millisecond time resolution. These processes are often highly dynamical and can be time-consuming to analyze and interpret. Here, we report how variational autoencoders (VAEs), a deep learning algorithm, can provide an artificial intelligence’s interpretation of high-resolution in situ TEM data by condensing and deconvoluting complicated atomic-scale dynamics into a latent space with reduced dimensionality. In this work, we designed a VAEs model with high latent dimensions capable of deconvoluting information from complex high-resolution TEM data. We demonstrate how this model with high latent dimensions trained on atomically resolved TEM images of lead sulfide (PbS) nanocrystals is able to capture movements and perturbations of periodic lattices in both simulated and real in situ TEM data. The VAEs model shows capability of detecting and deconvoluting dynamical nanoscale physical processes, such as the rotation of crystal lattices and intraparticle ripening during the annealing of semiconductor nanocrystals. With the help of the VAEs model, we can identify an in situ observation that can serve as a direct experimental evidence of the existence of intraparticle ripening. The VAEs model provides a potent tool for facilitating the analysis and interpretation of complex in situ TEM data as a part of an autonomous experimental workflow.


Machine learning
In situ TEM
Transmission electron microscopy

Supplementary materials

Supporting information
Detailed description of the procedures of preparing the training dataset for the VAEs model. Additional discussions on expanding the applications of VAEs model to a different dataset, and the optimization of the model.


Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy [opens in a new tab] - please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here [opens in a new tab] .
This site is protected by reCAPTCHA and the Google Privacy Policy [opens in a new tab] and Terms of Service [opens in a new tab] apply.