TransMap: Pinpointing Mistakes in Neural Code Translation
Open Access
- 30 November 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 999-1011
Abstract
Automated code translation between programming languages can greatly reduce the human effort needed in learning new languages or in migrating code. Recent neural machine translation models, such as Codex, have been shown to be effective on many code generation tasks including translation. However, code produced by neural translators often has semantic mistakes. These mistakes are difficult to eliminate from the neural translator itself because the translator is a black box, which is difficult to interpret or control compared to rule-based transpilers. We propose the first automated approach to pinpoint semantic mistakes in code obtained after neural code translation. Our techniques are implemented in a prototype tool called TransMap which translates Python to JavaScript, both of which are popular scripting languages. On our created micro-benchmarks of Python programs with 648 semantic mistakes in total, TransMap accurately pinpoints the correct location for a fix for 87.96%, often highlighting 1-2 lines for the user to inspect per mistake. We report on our experience in translating 5 Python libraries with up to 1k lines of code with TransMap. Our preliminary user study suggests that TransMap can reduce the time for fixing semantic mistakes by around 70% compared to using a standard IDE with debuggers.Keywords
Funding Information
- Ministry of Education - Singapore (T1 251RES2023, MOE-T2EP20220-0014)
This publication has 35 references indexed in Scilit:
- A Survey on Software Fault LocalizationIEEE Transactions on Software Engineering, 2016
- F3: fault localization for field failures2013
- SemFix: Program repair via semantic analysis2013
- DARWINACM Transactions on Software Engineering and Methodology, 2012
- Error Invariants2012
- Cause clue clausesACM SIGPLAN Notices, 2011
- Efficient program execution indexing2008
- Pex–White Box Test Generation for .NET2008
- Pruning dynamic slices with confidenceACM SIGPLAN Notices, 2006
- Microsoft Says JUMP—Java User Migration Path2002