Probabilistic Program Modeling for High-Precision Anomaly Classification
Authors:
- Kui XuKXKui Xu
- Danfeng Daphne YaoDYDanfeng Daphne Yao
- Barbara G. RyderBRBarbara G. Ryder
- Ke TianKTKe Tian
- 1 July 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636900,p. 497-511
Abstract
The trend constantly being observed in the evolution of advanced modern exploits is their growing sophistication in stealthy attacks. Code-reuse attacks such as return-oriented programming allow intruders to execute mal-intended instruction sequences on a victim machine without injecting external code. We introduce a new anomaly-based detection technique that probabilistically models and learns a program's control flows for high-precision behavioral reasoning and monitoring. Our prototype in Linux is named STILO, which stands for STatically InitiaLized markOv. Experimental evaluation involves real-world code-reuse exploits and over 4,000 testcases from server and utility programs. STILO achieves up to 28-fold of improvement in detection accuracy over the state-of-the-art HMM-based anomaly detection. Our findings suggest that the probabilistic modeling of program dependences provides a significant source of behavior information for building high-precision models for real-time system monitoring.Keywords
This publication has 37 references indexed in Scilit:
- MadeCR: Correlation-based malware detection for cognitive radio2015
- Profiling user-trigger dependence for Android malware detectionComputers & Security, 2015
- Expressing and verifying probabilistic assertionsACM SIGPLAN Notices, 2014
- Probabilistic programming2014
- Static analysis for probabilistic programs2013
- A survey of techniques for incremental learning of HMM parametersInformation Sciences, 2012
- There Is Safety in Numbers: Preventing Control-Flow Hijacking by DuplicationLecture Notes in Computer Science, 2012
- Synthesizing method sequences for high-coverage testing2011
- Host-based intrusion detection using dynamic and static behavioral modelsPattern Recognition, 2003
- Evidence-based static branch prediction using machine learningACM Transactions on Programming Languages and Systems, 1997