Artificial Intelligence Can Recognize Whether a Job Applicant Is Selling and/or Lying According to Facial Expressions and Head Movements Much More Correctly Than Human Interviewers
- 8 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 11 (5) , 5949-5960
Abstract
Whether an interviewee’s honest and deceptive responses can be detected by the signals of facial expressions in videos has been debated and called to be researched. We developed deep learningmodels enabled by computer vision to extract the temporal patterns of job applicants’ facial expressions and head movements to identify self-reported honest and deceptive impression management (IM) tactics from video frames in real asynchronous video interviews. A 12- to 15-min video was recorded for each of the N = 121 job applicants as they answered five structured behavioral interview questions. Each applicant completed a survey to self-evaluate their trustworthiness on four IM measures. Additionally, a field experiment was conducted to compare the concurrent validity associated with self-reported IMs between our modeling and human interviewers. Human interviewers’ performance in predicting these IMmeasures from another subset of 30 videos was obtained by having N = 30 human interviewers evaluate three recordings. Our models explained 91% and 84% of the variance in honest and deceptive IMs, respectively, and showed a stronger correlation with self-reported IMscores compared to human interviewers.Keywords
All Related Versions
Funding Information
- Taiwan Comprehensive University System
- National Science and Technology Council (NSTC), Taiwan (NSTC 109-2511-H-003-046, NSTC 110-2511-H-003-044-MY2, NSTC 112-2410-H-003-102-MY2, NSTC 111-2410-H-019-006-MY3)
This publication has 70 references indexed in Scilit:
- Detecting deception through facial expressions in a dataset of videotaped interviews: A comparison between human judges and machine learning modelsComputers in Human Behavior, 2022
- Personality Traits Associated with Various Forms of LyingPsychological Studies, 2020
- Shake and Fake: the Role of Interview Anxiety in Deceptive Impression ManagementJournal of Business and Psychology, 2020
- Deep learning model for real-time image compression in Internet of Underwater Things (IoUT)Journal of Real-Time Image Processing, 2019
- Deception Detection in Online Automated Job Interviews2018
- The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science EducationResearch in Science Education, 2017
- How Fast are the Leaked Facial Expressions: The Duration of Micro-ExpressionsJournal of Nonverbal Behavior, 2013
- A Fake Smile Thwarts Cheater DetectionJournal of Nonverbal Behavior, 2012
- Detecting Deception from Emotional and Unemotional CuesJournal of Nonverbal Behavior, 2008
- Silent talker: a new computer-based system for the analysis of facial cues to deceptionApplied Cognitive Psychology, 2006