These are preliminary reports that have not been peer-reviewed. They should not be regarded as conclusive, guide clinical practice/health-related behavior, or be reported in news media as established information. For more information, please see our FAQs.
V3 ChemArxive Computer vision for recognition of materials and vessels in chemistry lab settings and the Vector-LabPics dataset.pdf (1.85 MB)

Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Dataset

revised on 18.04.2020 and posted on 20.04.2020 by Sagi Eppel, Haoping Xu, Mor Bismuth, Alan Aspuru-Guzik
This work presents a machine learning approach for computer vision-based recognition of materials inside vessels in a chemistry lab and other settings. In addition, we release a dataset associated with the training of the model for further model development. The task to learn is finding the region, boundaries, and category for each material phase and vessel in an image. Handling materials inside mostly transparent containers is the main activity performed by human and robotic chemists in the laboratory. Visual recognition of vessels and their content is essential for performing this task. Modern machine vision methods learn recognition tasks by using datasets containing a large number of annotated images. This work presents the Vector-LabPics dataset, which consists of 2187 images of materials within mostly transparent vessels in a chemistry lab and other general settings. The images are annotated for both the vessels and the individual material phases inside them, and each instance is assigned one or more classes (liquid, solid, foam, suspension, powder,...). The fill level, labels, corks, and parts of the vessel are also annotated. Several convolutional nets for semantic and instance segmentation were trained on this dataset. The trained neural networks achieved good accuracy in detecting and segmenting vessels and material phases, and in classifying liquids and solids, but relatively low accuracy in segmenting multiphase systems such as phase-separating liquids.




Email Address of Submitting Author


University of Toronto, Vector institute



ORCID For Submitting Author


Declaration of Conflict of Interest