FoveaNet: Perspective-Aware Urban Scene Parsing

Authors:
  • XLXin Li
    Xin Li
  • ZJZequn Jie
    Zequn Jie
  • WWWei Wang
    Wei Wang
  • CLChangsong Liu
    Changsong Liu
  • JYJimei Yang
    Jimei Yang
  • XSXiaohui Shen
    Xiaohui Shen
  • ZLZhe Lin
    Zhe Lin
  • QCQiang Chen
    Qiang Chen
  • SYShuicheng Yan
    Shuicheng Yan
  • JFJiashi Feng
    Jiashi Feng
Publisher website
Google Scholar
Abstract
Parsing urban scene images benefits many applications, especially self-driving. Most of the current solutions employ generic image parsing models that treat all scales and locations in the images equally and do not consider the geometry property of car-captured urban scene images. Thus, they suffer from heterogeneous object scales caused by perspective projection of cameras on actual scenes and inevitably encounter parsing failures on distant objects as well as other boundary and recognition errors. In this work, we propose a new FoveaNet model to fully exploit the perspective geometry of scene images and address the common failures of generic parsing models. FoveaNet estimates the perspective geometry of a scene image through a convolutional network which integrates supportive evidence from contextual objects within the image. Based on the perspective geometry information, FoveaNet “undoes” the camera perspective projection - analyzing regions in the space of the actual scene, and thus provides much more reliable parsing results. Furthermore, to effectively address the recognition errors, FoveaNet introduces a new dense CRFs model that takes the perspective geometry as a prior potential. We evaluate FoveaNet on two urban scene parsing datasets, Cityspaces and CamVid, which demonstrates that FoveaNet can outperform all the well-established baselines and provide new state-of-the-art performance.