Jimmy Pronchick
Jimmy Pronchick has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Roger E. Beaty
Department of Psychology, Pennsylvania State University, USA
4 shared publications - Antonio Laverghetta
Advancing Machine and Human Reasoning (AMHR) Lab University of South Florida
2 shared publications - Dan Richard Johnson
Department of Cognitive and Behavioral Science, Washington and Lee University, Lexington, USA
2 shared publications - Simone A. Luchini
Pennsylvania State University
2 shared publications - Baptiste Barbot
University of Louvain, Ottignies-Louvain-la-Neuve, Belgium, Belgium
1 shared publication - James C. Kaufman
University of Connecticut
1 shared publication
Latest Publications
- Open Access
- 6 February 2026
- Published by Center for Open Science in PsyArXiv
Automated Detection of Invalid Responses to Creativity Assessments
Participants in creativity studies sometimes produce invalid data that is unusable for analysis,such as nonsensical or incomplete responses to idea generation tasks. Identifying such responsesis a time-consuming yet necessary process to ensure robust results but also remains challengingto automate. We explore the efficacy of transformer language models (TLMs) for automaticallydetecting invalid creativity responses, using only the text prompt and response and no othermetadata about the experimental session. We train a suite of transformers to detect invalid datafor two creativity assessments: the alternate uses task (AUT) and a design problems task (DPT).We compare TLMs against zero and few-shot large language models (LLMs) and other machinelearning models trained using word and sentence embeddings. We find that TLMs and LLMsgenerally outperform other baselines for both tasks. Further, we show that TLMs’ predictions arewell calibrated with respect to the quality of the participant response, ensuring that modelfailures will occur in a predictable way and that high-quality responses are unlikely to be labeledinvalid. Finally, we assess how well our models perform under an adversarial setting—using aprotocol where participants attempt to “break” the model by coming up with invalid responsesthat are nonetheless labeled valid. Our results demonstrate the potential of deep learning methodsfor cleaning creativity assessment data.
- 6 February 2026
- Published by Center for Open Science in PsyArXiv
Open AccessAutomated Detection of Invalid Responses to Creativity Assessments
Participants in creativity studies sometimes produce invalid data that is unusable for analysis,such as nonsensical or incomplete responses to idea generation tasks. Identifying such responsesis a time-consuming yet necessary process to ensure robust results but also remains challengingto automate. We explore the efficacy of transformer language models (TLMs) for automaticallydetecting invalid creativity responses, using only the text prompt and response and no othermetadata about the experimental session. We train a suite of transformers to detect invalid datafor two creativity assessments: the alternate uses task (AUT) and a design problems task (DPT).We compare TLMs against zero and few-shot large language models (LLMs) and other machinelearning models trained using word and sentence embeddings. We find that TLMs and LLMsgenerally outperform other baselines for both tasks. Further, we show that TLMs’ predictions arewell calibrated with respect to the quality of the participant response, ensuring that modelfailures will occur in a predictable way and that high-quality responses are unlikely to be labeledinvalid. Finally, we assess how well our models perform under an adversarial setting—using aprotocol where participants attempt to “break” the model by coming up with invalid responsesthat are nonetheless labeled valid. Our results demonstrate the potential of deep learning methodsfor cleaning creativity assessment data.
- Open Access
- 27 June 2025
- Published by Center for Open Science in PsyArXiv
CAP: The Creativity Assessment Platform for Online Testing and Automated Scoring
Creativity is increasingly recognized as a core competency for the 21st century, making its development a priority in education, research, and industry. To effectively cultivate creativity, researchers and educators need reliable and accessible assessment tools. Recent software developments have significantly enhanced the administration and scoring of creativity measures; however, existing software often requires expertise in experiment design and computer programming, limiting their accessibility to many educators and researchers. In the current work, we introduce CAP—the Creativity Assessment Platform—a free web application for building creativity assessments, collecting data, and automatically scoring responses (cap.ist.psu.edu). CAP allows users to create custom creativity assessments in 10 languages using a simple, point-and-click interface, selecting from tasks such as the Short Story Task, Drawing Task, and Scientific Creative Thinking Test. Users can automatically score task responses using machine learning models trained to match human creativity ratings—with multilingual capabilities, including the new Cross-Lingual Alternate Uses Scoring (CLAUS), a large language model achieving strong prediction of human creativity ratings in 10 languages. CAP also provides a centralized dashboard to monitor data collection, score assessments, and automatically generate text for a Method section based on the study’s tasks, metrics, and instructions—with a single click—promoting transparency and reproducibility in creativity assessment. Designed for ease-of-use, CAP aims to democratize creativity measurement, for researchers, educators, and everyone in-between.
- 27 June 2025
- Published by Center for Open Science in PsyArXiv
Open AccessCAP: The Creativity Assessment Platform for Online Testing and Automated Scoring
Creativity is increasingly recognized as a core competency for the 21st century, making its development a priority in education, research, and industry. To effectively cultivate creativity, researchers and educators need reliable and accessible assessment tools. Recent software developments have significantly enhanced the administration and scoring of creativity measures; however, existing software often requires expertise in experiment design and computer programming, limiting their accessibility to many educators and researchers. In the current work, we introduce CAP—the Creativity Assessment Platform—a free web application for building creativity assessments, collecting data, and automatically scoring responses (cap.ist.psu.edu). CAP allows users to create custom creativity assessments in 10 languages using a simple, point-and-click interface, selecting from tasks such as the Short Story Task, Drawing Task, and Scientific Creative Thinking Test. Users can automatically score task responses using machine learning models trained to match human creativity ratings—with multilingual capabilities, including the new Cross-Lingual Alternate Uses Scoring (CLAUS), a large language model achieving strong prediction of human creativity ratings in 10 languages. CAP also provides a centralized dashboard to monitor data collection, score assessments, and automatically generate text for a Method section based on the study’s tasks, metrics, and instructions—with a single click—promoting transparency and reproducibility in creativity assessment. Designed for ease-of-use, CAP aims to democratize creativity measurement, for researchers, educators, and everyone in-between.
- Open Access
- 26 May 2025
- Published by Center for Open Science in PsyArXiv
The Roles of Idea Generation and Elaboration in Human-AI Collaborative Creativity
Creativity is a fundamental ability of humans, vital to most domains of work and art. With the rising popularity of generative AI, research is needed to determine its costs and benefits for human creativity. We conducted a naturalistic study with university students (N = 263) who wrote short-stories with and without the help of ChatGPT. Stories written with AI assistance were significantly more original, but also more semantically similar to each other (i.e., less diverse). Critically, using a novel analysis of participant-AI interactions, we found that benefits in story originality were specifically predicted by engagement in idea generation with ChatGPT (prompting the AI to provide new story concepts, plot elements, or character ideas), but not idea elaboration (asking the AI to refine, reframe, or evaluate previously generated ideas). Additionally, we found that incorporating AI-generated content into the stories directly correlated with increased originality. Our findings suggest that AI-driven benefits depend on active engagement specifically focused on idea generation, although AI collaboration can limit idea diversity.
- 26 May 2025
- Published by Center for Open Science in PsyArXiv
Open AccessThe Roles of Idea Generation and Elaboration in Human-AI Collaborative Creativity
Creativity is a fundamental ability of humans, vital to most domains of work and art. With the rising popularity of generative AI, research is needed to determine its costs and benefits for human creativity. We conducted a naturalistic study with university students (N = 263) who wrote short-stories with and without the help of ChatGPT. Stories written with AI assistance were significantly more original, but also more semantically similar to each other (i.e., less diverse). Critically, using a novel analysis of participant-AI interactions, we found that benefits in story originality were specifically predicted by engagement in idea generation with ChatGPT (prompting the AI to provide new story concepts, plot elements, or character ideas), but not idea elaboration (asking the AI to refine, reframe, or evaluate previously generated ideas). Additionally, we found that incorporating AI-generated content into the stories directly correlated with increased originality. Our findings suggest that AI-driven benefits depend on active engagement specifically focused on idea generation, although AI collaboration can limit idea diversity.
- Open Access
- 8 May 2025
- Published by Center for Open Science in OSF Preprints
Artificial Intelligence Enhances Human Creativity Through Real-Time Evaluative Feedback
The integration of artificial intelligence (AI) into creative work continues to expand, yet its impact on human creativity itself—beyond simply providing ideas—remains uncertain. We reposition AI’s role from idea generator to idea evaluator, using trained models to provide real-time feedback on human-generated ideas. Across two studies—a preregistered online experiment involving individuals with varying levels of expertise (N = 554) and a large-scale naturalistic experiment during a year-long museum exhibit (N = 36,198)—participants generated solutions to real-world problems or created visual sketches. AI feedback significantly improved participant originality in both verbal and visual creative domains. Mediation analyses revealed these gains were partly driven by changes in individuals’ self-evaluation of their own originality, implying a key role for metacognition—the ability to monitor, control and regulate one’s thinking. These findings suggest that AI’s potential extends beyond generation to include idea evaluation, helping humans assess and refine their ideas through real-time feedback.
- 8 May 2025
- Published by Center for Open Science in OSF Preprints
Open AccessArtificial Intelligence Enhances Human Creativity Through Real-Time Evaluative Feedback
The integration of artificial intelligence (AI) into creative work continues to expand, yet its impact on human creativity itself—beyond simply providing ideas—remains uncertain. We reposition AI’s role from idea generator to idea evaluator, using trained models to provide real-time feedback on human-generated ideas. Across two studies—a preregistered online experiment involving individuals with varying levels of expertise (N = 554) and a large-scale naturalistic experiment during a year-long museum exhibit (N = 36,198)—participants generated solutions to real-world problems or created visual sketches. AI feedback significantly improved participant originality in both verbal and visual creative domains. Mediation analyses revealed these gains were partly driven by changes in individuals’ self-evaluation of their own originality, implying a key role for metacognition—the ability to monitor, control and regulate one’s thinking. These findings suggest that AI’s potential extends beyond generation to include idea evaluation, helping humans assess and refine their ideas through real-time feedback.