JPJimmy Pronchick

Jimmy Pronchick

*This is an algorithmically generated author record based on publicly available information.

Jimmy Pronchick has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Automated Detection of Invalid Responses to Creativity Assessments

    by Antonio Laverghetta, Simone A Luchini, Jimmy Pronchick, Roger Beaty

    Participants in creativity studies sometimes produce invalid data that is unusable for analysis,such as nonsensical or incomplete responses to idea generation tasks. Identifying such responsesis a time-consuming yet necessary process to ensure robust results but also remains challengingto automate. We explore the efficacy of transformer language models (TLMs) for automaticallydetecting invalid creativity responses, using only the text prompt and response and no othermetadata about the experimental session. We train a suite of transformers to detect invalid datafor two creativity assessments: the alternate uses task (AUT) and a design problems task (DPT).We compare TLMs against zero and few-shot large language models (LLMs) and other machinelearning models trained using word and sentence embeddings. We find that TLMs and LLMsgenerally outperform other baselines for both tasks. Further, we show that TLMs’ predictions arewell calibrated with respect to the quality of the participant response, ensuring that modelfailures will occur in a predictable way and that high-quality responses are unlikely to be labeledinvalid. Finally, we assess how well our models perform under an adversarial setting—using aprotocol where participants attempt to “break” the model by coming up with invalid responsesthat are nonetheless labeled valid. Our results demonstrate the potential of deep learning methodsfor cleaning creativity assessment data.

  • Open Access

    CAP: The Creativity Assessment Platform for Online Testing and Automated Scoring

    by John D. Patterson, Jimmy Pronchick, Ruchi Panachanadikar, Mark Fuge, Janet van Hell, Scarlett Miller, Dan Richard Johnson, Roger Beaty

    Creativity is increasingly recognized as a core competency for the 21st century, making its development a priority in education, research, and industry. To effectively cultivate creativity, researchers and educators need reliable and accessible assessment tools. Recent software developments have significantly enhanced the administration and scoring of creativity measures; however, existing software often requires expertise in experiment design and computer programming, limiting their accessibility to many educators and researchers. In the current work, we introduce CAP—the Creativity Assessment Platform—a free web application for building creativity assessments, collecting data, and automatically scoring responses (cap.ist.psu.edu). CAP allows users to create custom creativity assessments in 10 languages using a simple, point-and-click interface, selecting from tasks such as the Short Story Task, Drawing Task, and Scientific Creative Thinking Test. Users can automatically score task responses using machine learning models trained to match human creativity ratings—with multilingual capabilities, including the new Cross-Lingual Alternate Uses Scoring (CLAUS), a large language model achieving strong prediction of human creativity ratings in 10 languages. CAP also provides a centralized dashboard to monitor data collection, score assessments, and automatically generate text for a Method section based on the study’s tasks, metrics, and instructions—with a single click—promoting transparency and reproducibility in creativity assessment. Designed for ease-of-use, CAP aims to democratize creativity measurement, for researchers, educators, and everyone in-between.

  • Open Access

    The Roles of Idea Generation and Elaboration in Human-AI Collaborative Creativity

    by Simone A Luchini, Jimmy Pronchick, Simon M. Ceh, James C. Kaufman, Dan Richard Johnson, Janet Rafner, Roger Beaty

    Creativity is a fundamental ability of humans, vital to most domains of work and art. With the rising popularity of generative AI, research is needed to determine its costs and benefits for human creativity. We conducted a naturalistic study with university students (N = 263) who wrote short-stories with and without the help of ChatGPT. Stories written with AI assistance were significantly more original, but also more semantically similar to each other (i.e., less diverse). Critically, using a novel analysis of participant-AI interactions, we found that benefits in story originality were specifically predicted by engagement in idea generation with ChatGPT (prompting the AI to provide new story concepts, plot elements, or character ideas), but not idea elaboration (asking the AI to refine, reframe, or evaluate previously generated ideas). Additionally, we found that incorporating AI-generated content into the stories directly correlated with increased originality. Our findings suggest that AI-driven benefits depend on active engagement specifically focused on idea generation, although AI collaboration can limit idea diversity.

  • Open Access

    Artificial Intelligence Enhances Human Creativity Through Real-Time Evaluative Feedback

    by Pier Luc de Chantal, Roger Beaty, Antonio Laverghetta, Jimmy Pronchick, John Patterson, Peter Organisciak, Katarzyna Potega Vel Zabik, Baptiste Barbot, Maciej Karwowski

    The integration of artificial intelligence (AI) into creative work continues to expand, yet its impact on human creativity itself—beyond simply providing ideas—remains uncertain. We reposition AI’s role from idea generator to idea evaluator, using trained models to provide real-time feedback on human-generated ideas. Across two studies—a preregistered online experiment involving individuals with varying levels of expertise (N = 554) and a large-scale naturalistic experiment during a year-long museum exhibit (N = 36,198)—participants generated solutions to real-world problems or created visual sketches. AI feedback significantly improved participant originality in both verbal and visual creative domains. Mediation analyses revealed these gains were partly driven by changes in individuals’ self-evaluation of their own originality, implying a key role for metacognition—the ability to monitor, control and regulate one’s thinking. These findings suggest that AI’s potential extends beyond generation to include idea evaluation, helping humans assess and refine their ideas through real-time feedback.