Bridging Information-Theoretic and Geometric Compression in Language Models
Authors:
Open Access
- 1 January 2023
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 12397-12420
Abstract
Emily Cheng, Corentin Kervadec, Marco Baroni. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- The generalized ratios intrinsic dimension estimatorScientific Reports, 2022
- On Information Plane Analyses of Neural Network Classifiers—A ReviewIEEE Transactions on Neural Networks and Learning Systems, 2021
- Information-Theoretic Probing with Minimum Description Length2020
- Humans store about 1.5 megabytes of information during language acquisitionRoyal Society Open Science, 2019
- Estimating the intrinsic dimension of datasets by a minimal neighborhood informationScientific Reports, 2017
- Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books2015
- Deep learning and the information bottleneck principle2015
- Manifold-adaptive dimension estimation2007
- Measuring the strangeness of strange attractorsPhysica D: Nonlinear Phenomena, 1983
- An Algorithm for Finding Intrinsic Dimensionality of DataIEEE Transactions on Computers, 1971