Clayton Fields
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Clayton Fields has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
Casey Kennington
Department of Computer Science Boise State University
2 shared publications- Andrew McMains
Department of Computer Science Boise State University
1 shared publication - Casey Kennington
Boise State University 1910 W University Dr 83725 Boise ID
1 shared publication - Catherine Henry
Department of Computer Science Boise State University Boise Idaho 83725
1 shared publication Osama Natouf
1 shared publication
Latest Publications
- Open Access
Vision Language Transformers: A Survey
Vision language tasks, such as answering questions about or generating captions that describe an image, are difficult tasks for computers to perform. A relatively recent body of research has adapted the pretrained transformer architecture introduced in \citet{vaswani2017attention} to vision language modeling. Transformer models have greatly improved performance and versatility over previous vision language models. They do so by pretraining models on a large generic datasets and transferring their learning to new tasks with minor changes in architecture and parameter values. This type of transfer learning has become the standard modeling practice in both natural language processing and computer vision. Vision language transformers offer the promise of producing similar advancements in tasks which require both vision and language. In this paper, we provide a broad synthesis of the currently available research on vision language transformer models and offer some analysis of their strengths, limitations and some open questions that remain.
Open AccessVision Language Transformers: A Survey
Vision language tasks, such as answering questions about or generating captions that describe an image, are difficult tasks for computers to perform. A relatively recent body of research has adapted the pretrained transformer architecture introduced in \citet{vaswani2017attention} to vision language modeling. Transformer models have greatly improved performance and versatility over previous vision language models. They do so by pretraining models on a large generic datasets and transferring their learning to new tasks with minor changes in architecture and parameter values. This type of transfer learning has become the standard modeling practice in both natural language processing and computer vision. Vision language transformers offer the promise of producing similar advancements in tasks which require both vision and language. In this paper, we provide a broad synthesis of the currently available research on vision language transformer models and offer some analysis of their strengths, limitations and some open questions that remain.
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 521-531
Exploring Transformers as Compact, Data-efficient Language Models
Clayton Fields, Casey Kennington. Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL). 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 521-531
Open AccessExploring Transformers as Compact, Data-efficient Language Models
Clayton Fields, Casey Kennington. Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL). 2023.
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 19-29
Tiny Language Models Enriched with Multimodal Knowledge from Multiplex Networks
Clayton Fields, Osama Natouf, Andrew McMains, Catherine Henry, Casey Kennington. Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning. 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 19-29
Open AccessTiny Language Models Enriched with Multimodal Knowledge from Multiplex Networks
Clayton Fields, Osama Natouf, Andrew McMains, Catherine Henry, Casey Kennington. Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning. 2023.