Is ChatGPT a Good Personality Recognizer? A Preliminary Study
Authors:
- Liang HeLHLiang He
Preprint
Abstract
In recent years, personality has been regarded as a valuable personal factor being incorporated into numerous tasks such as sentiment analysis and product recommendation. This has led to widespread attention to text-based personality recognition task, which aims to identify an individual's personality based on given text. Considering that ChatGPT has recently exhibited remarkable abilities on various natural language processing tasks, we provide a preliminary evaluation of ChatGPT on text-based personality recognition task for generating effective personality data. Concretely, we employ a variety of prompting strategies to explore ChatGPT's ability in recognizing personality from given text, especially the level-oriented prompting strategy we designed for guiding ChatGPT in analyzing given text at a specified level. We compare the performance of ChatGPT on two representative real-world datasets with traditional neural network, fine-tuned RoBERTa, and corresponding state-of-the-art task-specific model. The experimental results show that ChatGPT with zero-shot chain-of-thought prompting exhibits impressive personality recognition ability. Triggered by zero-shot chain-of-thought prompting, ChatGPT outperforms fine-tuned RoBERTa on the two datasets and is capable to provide natural language explanations through text-based logical reasoning. Furthermore, relative to zero-shot chain-of-thought prompting, zero-shot level-oriented chain-of-thought prompting enhances the personality prediction ability of ChatGPT and reduces the performance gap between ChatGPT and corresponding state-of-the-art task-specific model. Besides, we also conduct experiments to observe the fairness of ChatGPT when identifying personality and discover that ChatGPT shows unfairness to some sensitive demographic attributes such as gender and age.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- Automatic Personality Prediction Based on Users’ Chinese Handwriting Change2023
- Personality BERT: A Transformer-Based Model for Personality Detection from Textual Data2022
- Knowledge-Enhanced Multi-task Learning for Course Recommendation2022
- Predictive intelligence in harmful news identification by BERT-based ensemble learning model with text sentiment analysisInformation Processing & Management, 2022
- “Knowing me, knowing you”: personalized explanations for a music recommender systemUser Modeling and User-Adapted Interaction, 2022
- Automatic children’s personality assessment from emotional speechExpert Systems with Applications, 2022
- A sentiment-aware deep learning approach for personality detection from textInformation Processing & Management, 2021
- How to Fine-Tune BERT for Text Classification?2019
- Personalizing recommendation diversity based on user personalityUser Modeling and User-Adapted Interaction, 2018
- Deep learning-based personality recognition from text posts of online social networksApplied Intelligence, 2018