Inverse Material Search and Synthesis Verification by Hand Drawings via Transfer Learning and Contour Detection

Nanomaterials of various morphologies and chemistry have an extensive use as photonic devices, advanced catalysts, sorbents for water purification, agrochemicals, platforms for drug delivery as well as imaging systems to name a few. However, search for synthesis routes giving custom nanomaterials for particular needs with the desired structure, shape, and size remains a challenge and is often implemented by manual research articles screening. Here, we develop for the first time scanning and transmission electron microscopy (SEM/TEM) reverse image search and hand drawing-based search via transfer learning (TL), namely, VGG16 convolutional neural network (CNN) repurposing for image features extraction and subsequent image similarity determination. Moreover, we demonstrate case use of this platform on calcium carbonate system, where sufficient amount of data was acquired by random high throughput multiparametric synthesis, as well as on Au nanoparticles (NPs) data extracted from the articles. This approach can be not only used for advanced nanomaterials search and synthesis procedure verification, but also can be further combined with machine learning (ML) solutions to provide data-driven novel nanomaterials discovery.