Daphne Ippolito
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Daphne Ippolito has a total of 1 co-authors. Here are the top co-authors by number of shared publications:
- Harshita Diddee
Carnegie Mellon University
2 shared publications
Latest Publications
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 1943-1957
Chasing Random: Instruction Selection Strategies Fail to Generalize
Harshita Diddee, Daphne Ippolito. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 1943-1957
Open AccessChasing Random: Instruction Selection Strategies Fail to Generalize
Harshita Diddee, Daphne Ippolito. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
- Open Access
Chasing Random: Instruction Selection Strategies Fail to Generalize
Prior work has shown that language models can be tuned to follow user instructions using only a small set of high-quality instructions. This has accelerated the development of methods that filter a large, noisy instruction-tuning datasets down to high-quality subset which works just as well. However, typically, the performance of these methods is not demonstrated across a uniform experimental setup and thus their generalization capabilities are not well established. In this work, we analyze popular selection strategies across different source datasets, selection budgets and evaluation benchmarks: Our results indicate that selection strategies generalize poorly, often failing to consistently outperform even random baselines. We also analyze the cost-performance trade-offs of using data selection. Our findings reveal that data selection can often exceed the cost of fine-tuning on the full dataset, yielding only marginal and sometimes no gains compared to tuning on the full dataset or a random subset.
Open AccessChasing Random: Instruction Selection Strategies Fail to Generalize
Prior work has shown that language models can be tuned to follow user instructions using only a small set of high-quality instructions. This has accelerated the development of methods that filter a large, noisy instruction-tuning datasets down to high-quality subset which works just as well. However, typically, the performance of these methods is not demonstrated across a uniform experimental setup and thus their generalization capabilities are not well established. In this work, we analyze popular selection strategies across different source datasets, selection budgets and evaluation benchmarks: Our results indicate that selection strategies generalize poorly, often failing to consistently outperform even random baselines. We also analyze the cost-performance trade-offs of using data selection. Our findings reveal that data selection can often exceed the cost of fine-tuning on the full dataset, yielding only marginal and sometimes no gains compared to tuning on the full dataset or a random subset.