PyCG: Practical Call Graph Generation in Python
Authors:
- 1 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1646-1657
Abstract
Call graphs play an important role in different contexts, such as profiling and vulnerability propagation analysis. Generating call graphs in an efficient manner can be a challenging task when it comes to high-level languages that are modular and incorporate dynamic features and higher-order functions. Despite the language’s popularity, there have been very few tools aiming to generate call graphs for Python programs. Worse, these tools suffer from several effectiveness issues that limit their practicality in realistic programs. We propose a pragmatic, static approach for call graph generation in Python. We compute all assignment relations between program identifiers of functions, variables, classes, and modules through an inter-procedural analysis. Based on these assignment relations, we produce the resulting call graph by resolving all calls to potentially invoked functions. Notably, the underlying analysis is designed to be efficient and scalable, handling several Python features, such as modules, generators, function closures, and multiple inheritance. We have evaluated our prototype implementation, which we call PyCG, using two benchmarks: a micro-benchmark suite containing small Python programs and a set of macro-benchmarks with several popular real-world Python packages. Our results indicate that PyCG can efficiently handle thousands of lines of code in less than a second (0.38 seconds for 1k LoC on average). Further, it outperforms the state-of-the-art for Python in both precision and recall: PyCG achieves high rates of precision ~99.2%, and adequate recall ~69.9%. Finally, we demonstrate how PyCG can aid dependency impact analysis by showcasing a potential enhancement to GitHub’s “security advisory” notification service using a real-world example.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- JSAI: a static analysis platform for JavaScript2014
- Mitigating program security vulnerabilitiesACM Computing Surveys, 2012
- Application-Only Call Graph Construction2012
- Tool-supported refactoring for JavaScript2011
- Strictly declarative specification of sophisticated points-to analysesACM SIGPLAN Notices, 2009
- Type Analysis for JavaScript2009
- Evaluating the benefits of context-sensitive points-to analysis using a BDD-based implementationACM Transactions on Software Engineering and Methodology, 2008
- Points-to analysis using BDDsACM SIGPLAN Notices, 2003
- Scaling Java Points-to Analysis Using Spark2003
- An empirical study of static call graph extractorsACM Transactions on Software Engineering and Methodology, 1998