The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
Preprint
Abstract
A writer's style depends not just on personal traits but also on her intent and mental state. In this paper, we show how variants of the same writing task can lead to measurable differences in writing style. We present a case study based on the story cloze task (Mostafazadeh et al., 2016a), where annotators were assigned similar writing tasks with different constraints: (1) writing an entire story, (2) adding a story ending for a given story context, and (3) adding an incoherent ending to a story. We show that a simple linear classifier informed with stylistic features is able to successfully distinguish between the three cases, without even looking at the story context. In addition, our style-based classifier establishes a new state-of-the-art result on the story cloze challenge, substantially higher than previous results based on deep learning models. Our results demonstrate that different task framings can dramatically affect the way people write.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Language Style Matching in Romantic Partners’ Conflict and Support InteractionsJournal of Language and Social Psychology, 2016
- Natural Language Use as a Marker of Personality2014
- Gender identity and lexical variation in social mediaJournal of Sociolinguistics, 2014
- Language Style Matching, Engagement, and Impasse in NegotiationsNegotiation and Conflict Management Research, 2014
- Language Style Matching as a Predictor of Social Dynamics in Small GroupsCommunication Research, 2009
- Computational methods in authorship attributionJournal of the American Society for Information Science and Technology, 2008
- On Lying and Being Lied To: A Linguistic Analysis of Deception in Computer-Mediated CommunicationDiscourse Processes, 2007
- Experimental disclosure and its moderators: A meta-analysis.Psychological Bulletin, 2006
- An exploratory study on promising cues in deception detection and application of decision tree2004
- Words of wisdom: Language use over the life span.Journal of Personality and Social Psychology, 2003