On the Implementation of OS-Specific Tests: The CPython Case
Open Access
- 22 September 2025
- conference paper
- Published by Sociedade Brasileira de Computação
- p. 46-54
Abstract
Modern software systems are frequently developed and tested across multiple platforms (e.g., Windows, Linux, and macOS). In the software testing context, practitioners adapt the tests to run differently according to the target platform. These tests, which need to identify the platform on which they will be executed, are referred to as OS-specific tests. In this paper, we present an empirical study to evaluate how developers implement OS-specific tests in CPython, which is the reference implementation project for the Python programming language. Then, we mine this project and assess their OS-specific tests quantitatively. For this, we propose three research questions to assess the frequency, location, and issues related to OS-specific tests. Our results show that OS-specific tests are common in the CPython project, and 13% of the analyzed test files are OS-specific tests (RQ1). OS Identification APIs are used more frequently in test code (53.46%), and the test decorator @unittest.skipUnless is the most used to skip tests depending on the platform (RQ2).We also find 170 issues related to OS-specific tests in CPython, and Windows is the most targeted platform (RQ3). Lastly, we discussed practical implications for practitioners and researchers. Based on our findings, we emphasized the importance of testing across multiple platforms and examined the relationship between issues and OS-specific tests, among other insights.Keywords
This publication has 27 references indexed in Scilit:
- Decoding Dependency Risks: A Quantitative Study of Vulnerabilities in the Maven Ecosystem2025
- How and why developers implement OS-specific testsEmpirical Software Engineering, 2024
- Excluding code from test coverage: practices, motivations, and impactEmpirical Software Engineering, 2022
- An Empirical Study of Flaky Tests in JavaScript2022
- PyNose: A Test Smell Detector For Python2021
- Rotten green tests in Java, Pharo and PythonEmpirical Software Engineering, 2021
- JavaScript API Deprecation Landscape: A Survey and Mining StudyIEEE Software, 2021
- Rotten Green Tests2019
- How do developers react to API evolution? A large-scale empirical studySoftware Quality Journal, 2016
- An empirical analysis of the distribution of unit test smells and their impact on software maintenance2012