Byte Latent Transformer: Patches Scale Better Than Tokens
Authors:
- Artidoro PagnoniAPArtidoro Pagnoni
- Ramakanth PasunuruRPRamakanth Pasunuru
Top Cited Papers
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 9238-9258
Abstract
Artidoro Pagnoni, Ramakanth Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason E Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman, Srini Iyer. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings2024
- MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling2024
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understanding2023
- Efficient Transformers with Dynamic Token Pooling2023
- The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine TranslationTransactions of the Association for Computational Linguistics, 2022
- ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte ModelsTransactions of the Association for Computational Linguistics, 2022
- Canine: Pre-training an Efficient Tokenization-Free Encoder for Language RepresentationTransactions of the Association for Computational Linguistics, 2022
- Hierarchical Transformers Are More Efficient Language Models2022
- Learning to rank with (a lot of) word featuresInformation Retrieval Journal, 2009