Neha Srikanth
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Neha Srikanth has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Rachel Rudinger
University of Maryland College Park USA
3 shared publications Ani Nenkova
University of Maryland ♣ Adobe Research
2 shared publications- Marine Carpuat
✦ University of Maryland ✿ Johns Hopkins University
2 shared publications Puneet Mathur
University of Maryland ♣ Adobe Research
2 shared publicationsVictor Bursztyn
University of Maryland ♣ Adobe Research
2 shared publications- Abby Sun
Public Health Science Program, University of Maryland School, College Park, MD 20742, USA
1 shared publication
Latest Publications
- Open Access
SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps
We introduce SQLSpace, a human-interpretable, generalizable, compact representation for text-to-SQL examples derived with minimal human intervention. We demonstrate the utility of these representations in evaluation with three use cases: (i) closely comparing and contrasting the composition of popular text-to-SQL benchmarks to identify unique dimensions of examples they evaluate, (ii) understanding model performance at a granular level beyond overall accuracy scores, and (iii) improving model performance through targeted query rewriting based on learned correctness estimation. We show that SQLSpace enables analysis that would be difficult with raw examples alone: it reveals compositional differences between benchmarks, exposes performance patterns obscured by accuracy alone, and supports modeling of query success.
Open AccessSQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps
We introduce SQLSpace, a human-interpretable, generalizable, compact representation for text-to-SQL examples derived with minimal human intervention. We demonstrate the utility of these representations in evaluation with three use cases: (i) closely comparing and contrasting the composition of popular text-to-SQL benchmarks to identify unique dimensions of examples they evaluate, (ii) understanding model performance at a granular level beyond overall accuracy scores, and (iii) improving model performance through targeted query rewriting based on learned correctness estimation. We show that SQLSpace enables analysis that would be difficult with raw examples alone: it reveals compositional differences between benchmarks, exposes performance patterns obscured by accuracy alone, and supports modeling of query success.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 1533-1559
SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps
Neha Srikanth, Victor Bursztyn, Puneet Mathur, Ani Nenkova. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 1533-1559
Open AccessSQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps
Neha Srikanth, Victor Bursztyn, Puneet Mathur, Ani Nenkova. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 3182-3199
No Questions are Stupid, but some are Poorly Posed: Understanding Poorly-Posed Information-Seeking Questions
Neha Srikanth, Rachel Rudinger, Jordan Lee Boyd-Graber. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 3182-3199
Open AccessNo Questions are Stupid, but some are Poorly Posed: Understanding Poorly-Posed Information-Seeking Questions
Neha Srikanth, Rachel Rudinger, Jordan Lee Boyd-Graber. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 2574-2589
NLI under the Microscope: What Atomic Hypothesis Decomposition Reveals
Neha Srikanth, Rachel Rudinger. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 2574-2589
Open AccessNLI under the Microscope: What Atomic Hypothesis Decomposition Reveals
Neha Srikanth, Rachel Rudinger. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.
- Open Access
- 18 September 2024
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 12, 1143-1162
How Often Are Errors in Natural Language Reasoning Due to Paraphrastic Variability?
Large language models have been shown to behave inconsistently in response to meaning-preserving paraphrastic inputs. At the same time, researchers evaluate the knowledge and reasoning abilities of these models with test evaluations that do not disaggregate the effect of paraphrastic variability on performance. We propose a metric, PC, for evaluating the paraphrastic consistency of natural language reasoning models based on the probability of a model achieving the same correctness on two paraphrases of the same problem. We mathematically connect this metric to the proportion of a model’s variance in correctness attributable to paraphrasing. To estimate PC, we collect ParaNlu, a dataset of 7,782 human-written and validated paraphrased reasoning problems constructed on top of existing benchmark datasets for defeasible and abductive natural language inference.1 Using ParaNlu, we measure the paraphrastic consistency of several model classes and show that consistency dramatically increases with pretraining but not fine-tuning. All models tested exhibited room for improvement in paraphrastic consistency.
- 18 September 2024
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 12, 1143-1162
Open AccessHow Often Are Errors in Natural Language Reasoning Due to Paraphrastic Variability?
Large language models have been shown to behave inconsistently in response to meaning-preserving paraphrastic inputs. At the same time, researchers evaluate the knowledge and reasoning abilities of these models with test evaluations that do not disaggregate the effect of paraphrastic variability on performance. We propose a metric, PC, for evaluating the paraphrastic consistency of natural language reasoning models based on the probability of a model achieving the same correctness on two paraphrases of the same problem. We mathematically connect this metric to the proportion of a model’s variance in correctness attributable to paraphrasing. To estimate PC, we collect ParaNlu, a dataset of 7,782 human-written and validated paraphrased reasoning problems constructed on top of existing benchmark datasets for defeasible and abductive natural language inference.1 Using ParaNlu, we measure the paraphrastic consistency of several model classes and show that consistency dramatically increases with pretraining but not fine-tuning. All models tested exhibited room for improvement in paraphrastic consistency.
- Open Access
How often are errors in natural language reasoning due to paraphrastic variability?
Large language models have been shown to behave inconsistently in response to meaning-preserving paraphrastic inputs. At the same time, researchers evaluate the knowledge and reasoning abilities of these models with test evaluations that do not disaggregate the effect of paraphrastic variability on performance. We propose a metric for evaluating the paraphrastic consistency of natural language reasoning models based on the probability of a model achieving the same correctness on two paraphrases of the same problem. We mathematically connect this metric to the proportion of a model's variance in correctness attributable to paraphrasing. To estimate paraphrastic consistency, we collect ParaNLU, a dataset of 7,782 human-written and validated paraphrased reasoning problems constructed on top of existing benchmark datasets for defeasible and abductive natural language inference. Using ParaNLU, we measure the paraphrastic consistency of several model classes and show that consistency dramatically increases with pretraining but not finetuning. All models tested exhibited room for improvement in paraphrastic consistency.
Open AccessHow often are errors in natural language reasoning due to paraphrastic variability?
Large language models have been shown to behave inconsistently in response to meaning-preserving paraphrastic inputs. At the same time, researchers evaluate the knowledge and reasoning abilities of these models with test evaluations that do not disaggregate the effect of paraphrastic variability on performance. We propose a metric for evaluating the paraphrastic consistency of natural language reasoning models based on the probability of a model achieving the same correctness on two paraphrases of the same problem. We mathematically connect this metric to the proportion of a model's variance in correctness attributable to paraphrasing. To estimate paraphrastic consistency, we collect ParaNLU, a dataset of 7,782 human-written and validated paraphrased reasoning problems constructed on top of existing benchmark datasets for defeasible and abductive natural language inference. Using ParaNLU, we measure the paraphrastic consistency of several model classes and show that consistency dramatically increases with pretraining but not finetuning. All models tested exhibited room for improvement in paraphrastic consistency.
- Open Access
- 1 September 2023
- Published by Wolters Kluwer Health in Journal of Public Health Management & Practice
- Vol. 29 (5) , 663-670
Practical Guidance for the Development of Rosie, a Health Education Question-and-Answer Chatbot for New Mothers
Communities of color experience higher maternal and infant mortality, as well as a host of other adverse outcomes, during pregnancy and postpartum. To address this, our team is developing a free, user-friendly, question-answering chatbot called Rosie . Chatbots have gained significant popularity due to their scalability and success in individualizing resources. In recent years, scientific communities and researchers have started recognizing this technology's potential to inform communities, promote health outcomes, and address health disparities. The development of Rosie is an interdisciplinary project, with teams focused on the technical build of the application (app), the development of machine learning models, and community outreach, making Rosie a chatbot built with the input from the communities it aims to serve. From June to October 2022, more than 20 demonstration sessions were conducted in Washington, District of Columbia, Maryland, and Virginia, where a total of 109 pregnant women and new mothers of color could interact with Rosie . Results from the live demonstrations showed that 75% of mothers searched for maternity and baby-related information at least once a week and more than 90% of participants expressed the likelihood to use the app. Most of the participants inquired about their baby's development, nutrition for babies, and identifying and addressing the causes of certain symptoms and conditions, accounting for about 80% of the total questions asked. Mother-related questions in the community demonstrations were mainly about pregnancy. The high level of interest in the chatbot is a clear indication of the need for more resources. Rosie aims to help close the racial gap in maternal and infant health disparities by providing new mothers with easy access to reliable health information.
- 1 September 2023
- Published by Wolters Kluwer Health in Journal of Public Health Management & Practice
- Vol. 29 (5) , 663-670
Open AccessPractical Guidance for the Development of Rosie, a Health Education Question-and-Answer Chatbot for New Mothers
Communities of color experience higher maternal and infant mortality, as well as a host of other adverse outcomes, during pregnancy and postpartum. To address this, our team is developing a free, user-friendly, question-answering chatbot called Rosie . Chatbots have gained significant popularity due to their scalability and success in individualizing resources. In recent years, scientific communities and researchers have started recognizing this technology's potential to inform communities, promote health outcomes, and address health disparities. The development of Rosie is an interdisciplinary project, with teams focused on the technical build of the application (app), the development of machine learning models, and community outreach, making Rosie a chatbot built with the input from the communities it aims to serve. From June to October 2022, more than 20 demonstration sessions were conducted in Washington, District of Columbia, Maryland, and Virginia, where a total of 109 pregnant women and new mothers of color could interact with Rosie . Results from the live demonstrations showed that 75% of mothers searched for maternity and baby-related information at least once a week and more than 90% of participants expressed the likelihood to use the app. Most of the participants inquired about their baby's development, nutrition for babies, and identifying and addressing the causes of certain symptoms and conditions, accounting for about 80% of the total questions asked. Mother-related questions in the community demonstrations were mainly about pregnancy. The high level of interest in the chatbot is a clear indication of the need for more resources. Rosie aims to help close the racial gap in maternal and infant health disparities by providing new mothers with easy access to reliable health information.