ALAntonio Laverghetta

Antonio Laverghetta

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Antonio Laverghetta has a total of 21 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Old Test, New Tricks: Using Large Language Models to Generate New Divergent Thinking Test Items

    by Averie Elizabeth Linnell, Simone A Luchini, Austin Swanson, Roni Reiter-Palmon, Roger Beaty, Antonio Laverghetta

    Automated item generation (AIG) using large language models (LLMs) is emerging as a promising solution for scalable creativity assessments, yet the potential of AIG to create psychometrically rigorous creativity items has thus far received little attention. In the present study, we evaluated the psychometric properties of LLM-generated items for the classic Consequences task of divergent thinking, comparing them to human-written items. A sample of 192 undergraduates completed six Consequences tasks—two human-written, two standard LLM-generated, and two genre-inspired LLM-generated, drawing on sci-fi and fantasy themes—along with measures of personality, cognitive ability, and creative behavior. We found that LLM-generated items elicited significantly higher fluency and flexibility. Genre-inspired items (fantasy and sci-fi) additionally elicited higher originality, enjoyment, and positive valence ratings; LLM-generated items were similar to human-written items. Genre-inspired items did not unfairly advantage participants with more exposure to genre content, demonstrating measurement invariance. Together, these findings suggest that LLMs can generate consequences items with psychometric properties comparable to those of human-written items. We discuss the implications for creativity assessment, item bank development, and best practices for AIG in creativity research, and we provide the prompts we used for item generation to enable researchers to generate new Consequences items at scale.

  • Open Access

    Automated Detection of Invalid Responses to Creativity Assessments

    by Antonio Laverghetta, Simone A Luchini, Jimmy Pronchick, Roger Beaty

    Participants in creativity studies sometimes produce invalid data that is unusable for analysis,such as nonsensical or incomplete responses to idea generation tasks. Identifying such responsesis a time-consuming yet necessary process to ensure robust results but also remains challengingto automate. We explore the efficacy of transformer language models (TLMs) for automaticallydetecting invalid creativity responses, using only the text prompt and response and no othermetadata about the experimental session. We train a suite of transformers to detect invalid datafor two creativity assessments: the alternate uses task (AUT) and a design problems task (DPT).We compare TLMs against zero and few-shot large language models (LLMs) and other machinelearning models trained using word and sentence embeddings. We find that TLMs and LLMsgenerally outperform other baselines for both tasks. Further, we show that TLMs’ predictions arewell calibrated with respect to the quality of the participant response, ensuring that modelfailures will occur in a predictable way and that high-quality responses are unlikely to be labeledinvalid. Finally, we assess how well our models perform under an adversarial setting—using aprotocol where participants attempt to “break” the model by coming up with invalid responsesthat are nonetheless labeled valid. Our results demonstrate the potential of deep learning methodsfor cleaning creativity assessment data.

  • Open Access

    Artificial Intelligence Enhances Human Creativity Through Real-Time Evaluative Feedback

    by Pier Luc de Chantal, Roger Beaty, Antonio Laverghetta, Jimmy Pronchick, John Patterson, Peter Organisciak, Katarzyna Potega Vel Zabik, Baptiste Barbot, Maciej Karwowski

    The integration of artificial intelligence (AI) into creative work continues to expand, yet its impact on human creativity itself—beyond simply providing ideas—remains uncertain. We reposition AI’s role from idea generator to idea evaluator, using trained models to provide real-time feedback on human-generated ideas. Across two studies—a preregistered online experiment involving individuals with varying levels of expertise (N = 554) and a large-scale naturalistic experiment during a year-long museum exhibit (N = 36,198)—participants generated solutions to real-world problems or created visual sketches. AI feedback significantly improved participant originality in both verbal and visual creative domains. Mediation analyses revealed these gains were partly driven by changes in individuals’ self-evaluation of their own originality, implying a key role for metacognition—the ability to monitor, control and regulate one’s thinking. These findings suggest that AI’s potential extends beyond generation to include idea evaluation, helping humans assess and refine their ideas through real-time feedback.


  • Open Access

    No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference

    by Animesh Nighojkar, Antonio Laverghetta, John Licato

    Natural Language Inference (NLI) has been a cornerstone task in evaluating language models' inferential reasoning capabilities. However, the standard three-way classification scheme used in NLI has well-known shortcomings in evaluating models' ability to capture the nuances of natural human reasoning. In this paper, we argue that the operationalization of the neutral label in current NLI datasets has low validity, is interpreted inconsistently, and that at least one important sense of neutrality is often ignored. We uncover the detrimental impact of these shortcomings, which in some cases leads to annotation datasets that actually decrease performance on downstream tasks. We compare approaches of handling annotator disagreement and identify flaws in a recent NLI dataset that designs an annotator study based on a problematic operationalization. Our findings highlight the need for a more refined evaluation framework for NLI, and we hope to spark further discussion and action in the NLP community.

  • Open Access

    Generating Better Items for Cognitive Assessments Using Large Language Models

    by Antonio Laverghetta, John Licato

    Writing high-quality test questions (items) is critical to building educational measures but has traditionally also been a time-consuming process. One promising avenue for alleviating this is automated item generation, whereby methods from artificial intelligence (AI) are used to generate new items with minimal human intervention. Researchers have explored using large language models (LLMs) to generate new items with equivalent psychometric properties to human-written ones. But can LLMs generate items with \textit{improved} psychometric properties, even when existing items have poor validity evidence? We investigate this using items from a natural language inference (NLI) dataset. We develop a novel prompting strategy based on selecting items with both the best and worst properties to use in the prompt and use GPT-3 to generate new NLI items. We find that the GPT-3 items show improved psychometric properties in many cases, whilst also possessing good content, convergent and discriminant validity evidence. Collectively, our results demonstrate the potential of employing LLMs to ease the item development process and suggest that the careful use of prompting may allow for iterative improvement of item quality.


  • A survey of fatigue measures and models

    by Antonio Laverghetta, Minh Tran, Alec Braynen, Stephen Steinle, Bekhzodbek Moydinboyev, Heba Daas, John Licato

    In long, stressful operational periods, military personnel face numerous challenges that may compromise their performance, an especially important one being fatigue. Current literature supports the view that behavioral, physiological, and cognitive factors are all predictive of the level of fatigue in individuals. However, much of the work on modeling fatigue has taken a narrow approach, relying only on a handful of modalities to measure fatigue. This paper aims to fill the void by providing an extensive overview of the current literature on both computationally measuring and modeling fatigue. We provide up-to-date and practical advice on which models are best suited for different situations and highlight directions for future work.


  • Open Access

    No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference

    by Animesh Nighojkar, Antonio Laverghetta Jr., John Licato

    Animesh Nighojkar, Antonio Laverghetta Jr., John Licato. Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII). 2023.


  • Predicting Human Psychometric Properties Using Computational Language Models

    by Antonio Laverghetta, Animesh Nighojkar, Jamshidbek Mirzakhalov, John Licato


  • Open Access

    Towards a Task-Agnostic Model of Difficulty Estimation for Supervised Learning Tasks

    by Antonio Laverghetta Jr., Jamshidbek Mirzakhalov, John Licato

    Antonio Laverghetta Jr., Jamshidbek Mirzakhalov, John Licato. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: Student Research Workshop. 2020.