End-to-end Document Recognition and Understanding with Dessurt

Authors:
  • BDBrian Davis
    Brian Davis
  • BMBryan Morse
    Bryan Morse
  • BPBryan Price
    Bryan Price
  • CTChris Tensmeyer
    Chris Tensmeyer
  • CWCurtis Wigington
    Curtis Wigington
  • VMVlad Morariu
    Vlad Morariu

Abstract
We introduce Dessurt, a relatively simple document understanding transformer capable of being fine-tuned on a greater variety of document tasks than prior methods. It receives a document image and task string as input and generates arbitrary text autoregressively as output. Because Dessurt is an end-to-end architecture that performs text recognition in addition to the document understanding, it does not require an external recognition model as prior methods do, making it easier to fine-tune to new visual domains. We show that this model is effective at 9 different dataset-task combinations.