On Sensor Bias in Experimental Methods for Comparing Interest-Point, Saliency, and Recognition Algorithms
Authors:
- 12 May 2011
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 34 (1) , 110-126
Abstract
Most current algorithm evaluation protocols use large image databases, but give little consideration to imaging characteristics used to create the data sets. This paper evaluates the effects of camera shutter speed and voltage gain under simultaneous changes in illumination and demonstrates significant differences in the sensitivities of popular vision algorithms under variable illumination, shutter speed, and gain. These results show that offline data sets used to evaluate vision algorithms typically suffer from a significant sensor specific bias which can make many of the experimental methodologies used to evaluate vision algorithms unable to provide results that generalize in less controlled environments. We show that for typical indoor scenes, the different saturation levels of the color filters are easily reached, leading to the occurrence of localized saturation which is not exclusively based on the scene radiance but on the spectral density of individual colors present in the scene. Even under constant illumination, foreshortening effects due to surface orientation can affect feature detection and saliency. Finally, we demonstrate that active and purposive control of the shutter speed and gain can lead to significantly more reliable feature detection under varying illumination and nonconstant viewpoints.Keywords
This publication has 35 references indexed in Scilit:
- A Computational Perspective on Visual Attention2011
- Illumination robust interest point detectionComputer Vision and Image Understanding, 2009
- Saliency, attention, and visual search: An information theoretic approachJournal of Vision, 2009
- A Comparison of Affine Region DetectorsInternational Journal of Computer Vision, 2005
- A performance evaluation of local descriptorsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Common principles of image acquisition systems and biological visionProceedings of the IEEE, 2002
- Evaluation of Interest Point DetectorsInternational Journal of Computer Vision, 2000
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- Background light and the contrast gain of primate P and M retinal ganglion cells.Proceedings of the National Academy of Sciences, 1988
- Light adaptation of primate cones: An analysis based on extracellular dataVision Research, 1983