Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set
- 10 September 2020
- journal article
- research article
- Published by American Chemical Society (ACS) in ACS Central Science
- Vol. 6 (10) , 1743-1752
Abstract
This work presents a machine learning approach for the computer vision-based recognition of materials inside vessels in the chemistry lab and other settings. In addition, we release a data set associated with the training of the model for further model development. The task to learn is finding the region, boundaries, and category for each material phase and vessel in an image. Handling materials inside mostly transparent containers is the main activity performed by human and robotic chemists in the laboratory. Visual recognition of vessels and their contents is essential for performing this task. Modern machine-vision methods learn recognition tasks by using data sets containing a large number of annotated images. This work presents the Vector-LabPics data set, which consists of 2187 images of materials within mostly transparent vessels in a chemistry lab and other general settings. The images are annotated for both the vessels and the individual material phases inside them, and each instance is assigned one or more classes (liquid, solid, foam, suspension, powder, ...). The fill level, labels, corks, and parts of the vessel are also annotated. Several convolutional nets for semantic and instance segmentation were trained on this data set. The trained neural networks achieved good accuracy in detecting and segmenting vessels and material phases, and in classifying liquids and solids, but relatively low accuracy in segmenting multiphase systems such as phase-separating liquids.Keywords
All Related Versions
Funding Information
- Defense Advanced Research Projects Agency (HR00111920027)
This publication has 35 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- Fully convolutional networks for semantic segmentation2015
- Organic Synthesis: March of the MachinesAngewandte Chemie International Edition, 2015
- Camera-enabled techniques for organic synthesisBeilstein Journal of Organic Chemistry, 2013
- A prototype continuous-flow liquid–liquid extraction system using open-source technologyOrganic & Biomolecular Chemistry, 2012
- Liquid-level measurement using a single digital cameraMeasurement, 2009
- Measurement of liquid interface based on vision2004
- An experimental comparison of min-cut/max- flow algorithms for energy minimization in visionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- A Computational Approach to Edge Detection1987
- A Computational Approach to Edge DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1986