FCENet: An Instance Segmentation Model for Extracting Figures and Captions From Material Documents

Authors:
Open Access
Publisher website
Google Scholar
Abstract
A critical ideology of the existing Material Genome Project refers to the application of data and artificial intelligence to facilitate material innovation. The lack of data hinders the development of novel materials. The figures and captions in the material literature cover essential information regarding the entire document and have sufficient image sample data for research. Accordingly, how to extract figures and captions from the literature is critical to solve the lack of data. Though some PDF parsing tools are capable of extracting information from documents, they generally identify a document’s figures by parsing the document into a concrete structure. As impacted by the inconsistency of the form of different journals, they commonly achieve wrong recognition results. Thus, an efficient figure and caption extraction network FCENet is proposed in the present study. Inconsistent with other extraction tools, this study first attempts to adopt instance segmentation models to detect figures and their captions, and then extract them. FCENet developed in this study builds upon BlendMask and introduces a horizontal and vertical attention module. This study splits the BlendMask detection head into two branches, i.e., figure detection and caption detection, which increases final detection accuracy and speed. This study collects nearly 3000 material documents for model training and testing. As revealed from the last experiments and results, the performance of FCENet is significantly compared with that of other existing instance segmentation models. Its box and mask mAP (mean Average Precision) are 8.51% and 12.59% higher than those of BlendMask, respectively. This study hopes that considerable material image data can be acquired via FCENet and sufficiently support image data for machine learning and data mining in the material area.
Funding Information
  • National Natural Science Foundation of China (52061020, 61971208, 61671225)
  • Yunnan Applied Basic Research Projects (2018FA034)
  • Yunnan Reserve Talents of Young and Middle-Aged Academic and Technical Leaders (Shen Tao, 2018);
  • Yunnan Young Top Talents of Ten Thousands Plan (2018 73)
  • Scientific Research Foundation of Kunming University of Science and Technology (KKSY201703016)
  • Open Fund of Yunnan Key Laboratory of Computer Technology Application

This publication has 27 references indexed in Scilit: