Repository-Aware Metamorphic Relation Generation for Augmented Reality Applications using Large Language Models
Open Access
- 7 October 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2613-2625
Abstract
Metamorphic Testing (MT) provides a promising approach for testing software without defined test oracles by specifying expected relations between inputs and outputs, instead of relying on exact outputs. For example, testing Augmented Reality (AR) applications is challenging due to dynamic interactions between virtual content, physical environments, and code, which make traditional test oracles difficult to define. However, formulating metamorphic relations (MRs) is time-consuming and burdensome. We introduce a context-aware pipeline that generates and refines MRs using repository-level context and reasoning orchestration, evaluated on a dataset of 142 mobile AR system repositories. Across three context configurations generating 14,916 candidate MRs, hierarchical context yielded the broadest coverage (7,004 MRs across 142 repositories and 5,167 class–method pairs) and lower redundancy. An agentic deliberation process then reconciled conflicting candidates—observed in 79.0% of cases—reducing duplication and selecting context-aware relations in 88.2% of outcomes. A manual oracle study shows refined relations (n = 141) are both logically valid and sufficiently concrete to be directly translated into test assertions, and a preliminary case study reveals converting generated MRs (n = 5) into executable tests can detect non-equivalent mutations in real-world code. Overall, our results show that combining repository-aware MR generation with reasoning-based refinement enables scalable construction of reliable, domain-relevant test oracles.Keywords
All Related Versions
This publication has 50 references indexed in Scilit:
- VRTestSniffer: Test Smell Detector for Virtual Reality (VR) Software Projects2025
- VRExplorer: A Model-based Approach for Semi-Automated Testing of Virtual Reality Scenes2025
- LLMs in Debate: Does Arguing Make Them Better at Detecting Metamorphic Relations?2025
- Identifying metamorphic relations: A data mutation directed approachSoftware: Practice and Experience, 2023
- Exploiting deep reinforcement learning and metamorphic testing to automatically test virtual reality applicationsSoftware Testing, Verification and Reliability, 2023
- CAT-LM Training Language Models on Aligned Code And Tests2023
- Metamorphic Testing For Machine Learning: Applicability, Challenges, and Research Opportunities2023
- Challenges and opportunities on AR/VR technologies for manufacturing systems in the context of industry 4.0: A state of the art reviewJournal of Manufacturing Systems, 2022
- Metamorphic relation automation: Rationale, challenges, and solution directionsJournal of Software: Evolution and Process, 2022
- Generating Structurally Complex Test Cases By Data Mutation: A Case Study Of Testing An Automated Modelling ToolThe Computer Journal, 2007