With whom do I interact? Detecting social interactions in egocentric photo-streams
Authors:
- 1 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2959-2964
Abstract
Given a user wearing a low frame rate wearable camera during a day, this work aims to automatically detect the moments when the user gets engaged into a social interaction solely by reviewing the automatically captured photos by the worn camera. The proposed method, inspired by the sociological concept of F-formation, exploits distance and orientation of the appearing individuals -with respect to the user- in the scene from a bird-view perspective. As a result, the interaction pattern over the sequence can be understood as a two-dimensional time series that corresponds to the temporal evolution of the distance and orientation features over time. A Long-Short Term Memory-based Recurrent Neural Network is then trained to classify each time series. Experimental evaluation over a dataset of 30.000 images has shown promising results on the proposed method for social interaction detection in egocentric photo-streams.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Multi-view Face Detection Using Deep Convolutional Neural Networks2015
- R-Clustering for Egocentric Video Segmentation2015
- Human behavior analysis in video surveillance: A Social Signal Processing perspectiveNeurocomputing, 2013
- Social Relationships and Health: A Flashpoint for Health PolicyJournal of Health and Social Behavior, 2010
- Action Classification in Soccer Videos with Long Short-Term Memory Recurrent Neural Networks2010
- Social signal processing: Survey of an emerging domainImage and Vision Computing, 2009
- Automatic nonverbal analysis of social interaction in small groups: A reviewImage and Vision Computing, 2009
- Framewise phoneme classification with bidirectional LSTM and other neural network architecturesNeural Networks, 2005
- Extraction of Semantic Dynamic Content from Videos with Probabilistic Motion Models2004
- Long Short-Term MemoryNeural Computation, 1997