Towards Discrete Object Representations in Vision Transformers with Tensor Products
Open Access
- 8 December 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 190-194
Abstract
In this work, we explore the use of Tensor Product Representations (TPRs) in a Vision Transformer model to form image representations that can later be used for symbolic manipulation in a neurosymbolic model. We propose the Tensor Product Vision Transformer (TP-ViT), an enhancement of a Vision Transformer that incorporates TPRs, an object representation methodology that utilizes filler and role vectors to represent objects. TP-ViT is the first application of TPRs on visual input, and we report qualitative and quantitative results which show that the use of TPRs allows for the formation of more targeted and diverse object representations when compared to a standard Vision Transformer.Keywords
Funding Information
- Ministry of Higher Education Malaysia (FRGS/1/2020/ICT02/MUSM/03/1)
This publication has 6 references indexed in Scilit:
- Delving Deep into the Generalization of Vision Transformers under Distribution Shifts2022
- Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet2021
- Attentive Tensor Product LearningProceedings of the AAAI Conference on Artificial Intelligence, 2019
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Transforming Auto-Encoders2011
- ImageNet: A large-scale hierarchical image database2009