Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results
- 3 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 4 (4) , 522-532
Abstract
We present the Deception Detection and Physiological Monitoring (DDPM) dataset and initial baseline results of deception detection on this dataset. Our application context is an interview scenario in which the interviewee attempts to deceive the interviewer on selected responses. The interviewee is recorded in visible, near-infrared, and long-wave infrared light, along with cardiac pulse, blood oxygenation, and audio. After collection, data were annotated for interviewer/interviewee, curated, ground-truthed, and organized into train / test parts for a set of canonical deception detection experiments. For the collected data, we estimated subject heart rates (remotely from face videos with a mean absolute error lower than two beats per minute), pupil size, and eye gaze. Fusion experiments discovered that a combination of remote plethysmography, pupil size, and thermal data yields the best deception detection results, with an equal error rate of 0.357. The database contains almost 13 hours of recordings of 70 subjects, and over 8 million visible-light, near-infrared, and thermal video frames, along with appropriate meta, audio and pulse oximeter data. To our knowledge, this is the only collection offering recordings of five modalities in an interview scenario that can be used in both deception detection and remote photoplethysmography research.Keywords
Funding Information
- Securiport LLC
This publication has 38 references indexed in Scilit:
- Deception Detection using Real-life Trial Data2015
- Photoplethysmography Revisited: From Contact to Noncontact, From Point to ImagingIEEE Transactions on Biomedical Engineering, 2015
- Saccadic eye movement rate as a cue to deceit.Journal of Applied Research in Memory and Cognition, 2015
- Thermal Facial Analysis for Deception DetectionIEEE Transactions on Information Forensics and Security, 2014
- Robust Pulse Rate From Chrominance-Based rPPGIEEE Transactions on Biomedical Engineering, 2013
- Pinocchio's Pupil: Using Eyetracking and Pupil Dilation to Understand Truth Telling and Deception in Sender-Receiver GamesAmerican Economic Review, 2010
- Non-contact, automated cardiac pulse measurements using video imaging and blind source separationOptics Express, 2010
- Motion Profiles for Deception Detection Using Visual Cues2010
- Lie Catching and Microexpressions2009
- THE IMPACT OF DECEPTION AND SUSPICION ON DIFFERENT HAND MOVEMENTSJournal of Nonverbal Behavior, 2006