Bio-Inspired Human Action Recognition With a Micro-Doppler Sonar System
Open Access
- 27 July 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 6, 28388-28403
Abstract
This paper explores computational methods to address the problem of doing inference from data in multiple modalities where there exists large amounts of low dimensional data complementary to a much smaller set of high dimensional data. In this instance the low dimensional time-series data are active acoustics from a bio-inspired micro-Doppler sonar sensor system that include no or very limited spatial information, and the high dimensional data are RGB-Depth data from a 3D point cloud sensor. The task is human action recognition from the active acoustic data. To accomplish this, statistical models, trained simultaneously on both the micro-Doppler modulations induced by human actions and symbolic representations of skeletal poses, derived from the 3D point cloud data, are developed. This simultaneous training enables the model to learn relations between the rich temporal structure of the micro-Doppler modulations and the high-dimensional pose sequences of human action. During runtime, the model relies purely on the active acoustic sonar data to infer the human action. Our approach is applicable to other sensing modalities such as the millimeter wave electromagnetic radar devices.Keywords
Funding Information
- EU ICT (ICT-231168-SCANDLE)
- ONR MURI (N000141010278)
- Division of Engineering Education and Centers (INSPIRE SMA 1248056)
- National Science Foundation (SCH-INT 1344772)
- Applied Physics Laboratory, Johns Hopkins University
This publication has 39 references indexed in Scilit:
- The Johns Hopkins University multimodal dataset for human action recognition2015
- Noncontact Wideband Sonar for Human Activity Detection and ClassificationIEEE Sensors Journal, 2014
- ANALYSIS OF MOVING HUMAN MICRO-DOPPLER SIGNATURE IN FOREST ENVIRONMENTSProgress In Electromagnetics Research, 2014
- MULTIMODAL INTEGRATION OF MICRO-DOPPLER SONAR AND AUDITORY SIGNALS FOR BEHAVIOR CLASSIFICATION WITH CONVOLUTIONAL NETWORKSInternational Journal of Neural Systems, 2013
- Classification of personnel targets by acoustic micro-Doppler signaturesIET Radar, Sonar & Navigation, 2011
- Human activity analysisACM Computing Surveys, 2011
- Horseshoe bats make adaptive prey-selection decisions, informed by echo cuesProceedings Of The Royal Society B-Biological Sciences, 2011
- Probing the natural scene by echolocation in batsFrontiers in Behavioral Neuroscience, 2010
- Micro-Doppler-based target detection and feature extraction in indoor and outdoor environmentsJournal of the Franklin Institute, 2008
- A survey of advances in vision-based human motion capture and analysisComputer Vision and Image Understanding, 2006