T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings
Authors:
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 21829-21851
Abstract
Björn Deiseroth, Manuel Brack, Patrick Schramowski, Kristian Kersting, Samuel Weinbach. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Tokenizer Choice For LLM Training: Negligible or Crucial?2024
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- Canine: Pre-training an Efficient Tokenization-Free Encoder for Language RepresentationTransactions of the Association for Computational Linguistics, 2022
- HashFormers: Towards Vocabulary-independent Pre-trained Transformers2022
- UDPipe 2.0 Prototype at CoNLL 2018 UD Shared Task2018
- SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing2018
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs2017