Abstract
We propose a machine learning based approach to real-time detection and classification assistance for images from unknown environments. While systems for detecting and classifying regular structures like faces in still images are well established, the task of e. g. detecting new morphotypes/objects in an environment is much more complex. The morphotypes/objects are not guaranteed to have apriori known characteristics such as facial landmarks, thus adaptive and data-driven approaches are needed that are trained online with a growing number of available labels provided by a human observer. Our approach combines an effective points of interest computation with an online learning network for real-time proposition of classes. In an example application using images from an arctic deep-sea underwater observatory we report a maximum accuracy of 0.87 and a top-3 accuracy of 0.97 in the range from 10 to 1499 available labels. The average accuracy in that range is 0.74, with the top-3 accuracy being 0.93. Additionally, a convolutional neural network achieves an accuracy of 0.965 at 1500 labels. Points of interest are computed in approximately 3.7 seconds, right after loading a new image. Proposing a class for a computationally determined point of interest or a manually selected one takes about 0.2 seconds. If the image is already in cache the time is reduced to 0.1 seconds. The suggestion of points of interest combined with a proposition of putative classes allows for significantly faster labeling times in new environments.

This publication has 16 references indexed in Scilit: