Rakib Hossain Misu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Rakib Hossain Misu has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
- Cristina Videira Lopes
School of Information and Computer Sciences University of California Irvine
2 shared publications Adithya Bhattiprolu
1 shared publicationAleksandar Saša Janjanin
University College London, UK
1 shared publication- Chenyuan Yang
University of Illinois at Urbana-Champaign, Champaign, USA
1 shared publication - Chris Hawblitzel
Microsoft Research, Redmond, WA, USA
1 shared publication - Eduardo Santana de Almeida
Federal University of Bahia, Salvador, Brazil
1 shared publication
Latest Publications
- 13 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (4) , 1426-1445
Toward Automated Validation of Language Model Synthesized Test Cases Using Semantic Entropy
Modern Large Language Model (LLM)-based programming agents often rely on test execution feedback to refine their generated code. These tests are synthetically generated by LLMs. However, LLMs may produce invalid or hallucinated test cases, which can mislead feedback loops and degrade the performance of agents in refining and improving code. This paper introduces VALTEST, a novel framework that leverages semantic entropy to automatically validate test cases generated by LLMs. By analyzing the semantic structure of test cases and computing entropy-based uncertainty measures, VALTEST trains a machine learning model to classify test cases as valid or invalid and filters out invalid test cases. Experiments on multiple benchmark datasets and various LLMs show that VALTEST not only boosts test validity by up to 29% but also improves code generation performance, as evidenced by significant increases in pass@1 scores. Our extensive experiments also reveal that semantic entropy is a reliable indicator to distinguish between valid and invalid test cases and provides a robust solution for improving the correctness of LLM-generated test cases used in software testing and code generation.
- 13 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (4) , 1426-1445
Toward Automated Validation of Language Model Synthesized Test Cases Using Semantic Entropy
Modern Large Language Model (LLM)-based programming agents often rely on test execution feedback to refine their generated code. These tests are synthetically generated by LLMs. However, LLMs may produce invalid or hallucinated test cases, which can mislead feedback loops and degrade the performance of agents in refining and improving code. This paper introduces VALTEST, a novel framework that leverages semantic entropy to automatically validate test cases generated by LLMs. By analyzing the semantic structure of test cases and computing entropy-based uncertainty measures, VALTEST trains a machine learning model to classify test cases as valid or invalid and filters out invalid test cases. Experiments on multiple benchmark datasets and various LLMs show that VALTEST not only boosts test validity by up to 29% but also improves code generation performance, as evidenced by significant increases in pass@1 scores. Our extensive experiments also reveal that semantic entropy is a reliable indicator to distinguish between valid and invalid test cases and provides a robust solution for improving the correctness of LLM-generated test cases used in software testing and code generation.
- Open Access
- 9 October 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Programming Languages
- Vol. 9 (OOPSLA2) , 3454-3482
AutoVerus: Automated Proof Generation for Rust Code
Generative AI has shown its value for many software engineering tasks. Still in its infancy, large language model (LLM)-based proof generation lags behind LLM-based code generation. In this paper, we present AutoVerus. AutoVerus uses LLMs to automatically generate correctness proof for Rust code. AutoVerus is designed to match the unique features of Verus, a verification tool that can prove the correctness of Rust code using proofs and specifications also written in Rust. AutoVerus consists of a network of agents that are crafted and orchestrated to mimic human experts' three phases of proof construction: preliminary proof generation, proof refinement guided by generic tips, and proof debugging guided by verification errors. To thoroughly evaluate AutoVerus and help foster future research in this direction, we have built a benchmark suite of 150 non-trivial proof tasks, based on existing code-generation benchmarks and verification benchmarks. Our evaluation shows that AutoVerus can automatically generate correct proof for more than 90% of them, with more than half of them tackled in less than 30 seconds or 3 LLM calls.
- 9 October 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Programming Languages
- Vol. 9 (OOPSLA2) , 3454-3482
Open AccessAutoVerus: Automated Proof Generation for Rust Code
Generative AI has shown its value for many software engineering tasks. Still in its infancy, large language model (LLM)-based proof generation lags behind LLM-based code generation. In this paper, we present AutoVerus. AutoVerus uses LLMs to automatically generate correctness proof for Rust code. AutoVerus is designed to match the unique features of Verus, a verification tool that can prove the correctness of Rust code using proofs and specifications also written in Rust. AutoVerus consists of a network of agents that are crafted and orchestrated to mimic human experts' three phases of proof construction: preliminary proof generation, proof refinement guided by generic tips, and proof debugging guided by verification errors. To thoroughly evaluate AutoVerus and help foster future research in this direction, we have built a benchmark suite of 150 non-trivial proof tasks, based on existing code-generation benchmarks and verification benchmarks. Our evaluation shows that AutoVerus can automatically generate correct proof for more than 90% of them, with more than half of them tackled in less than 30 seconds or 3 LLM calls.
- Open Access
- 3 February 2025
- Published by Elsevier in Information and Software Technology
- Vol. 181
Test smell: A parasitic energy consumer in software testing
- 3 February 2025
- Published by Elsevier in Information and Software Technology
- Vol. 181
Open AccessTest smell: A parasitic energy consumer in software testing
- Top Cited PapersOpen Access
- 12 July 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 1 (FSE) , 812-835
Towards AI-Assisted Synthesis of Verified Dafny Methods
Large language models show great promise in many domains, including programming. A promise is easy to make but hard to keep, and language models often fail to keep their promises, generating erroneous code. A promising avenue to keep models honest is to incorporate formal verification: generating programs’ specifications as well as code, so that the code can be proved correct with respect to the specifications. Unfortunately, existing large language models show a severe lack of proficiency in verified programming. In this paper, we demonstrate how to improve two pretrained models’ proficiency in the Dafny verification-aware language. Using 178 problems from the MBPP dataset, we prompt two contemporary models (GPT-4 and PaLM-2) to synthesize Dafny methods. We use three different types of prompts: a direct Contextless prompt; a Signature prompt that includes a method signature and test cases, and a Chain of Thought (CoT) prompt that decomposes the problem into steps and includes retrieval augmentation generated example problems and solutions. Our results show that GPT-4 performs better than PaLM-2 on these tasks, and that both models perform best with the retrieval augmentation generated CoT prompt. GPT-4 was able to generate verified, human-evaluated, Dafny methods for 58% of the problems, however GPT-4 managed only 19% of the problems with the Contextless prompt, and even fewer (10%) for the Signature prompt. We are thus able to contribute 153 verified Dafny solutions to MBPP problems, 50 that we wrote manually and 103 synthesized by GPT-4. Our results demonstrate that the benefits of formal program verification are now within reach of codegenerating large language models. Likewise, program verification systems can benefit from large language models, whether to synthesize code wholesale, to generate specifications, or to act as a “programmer’s verification apprentice”, to construct annotations such as loop invariants which are hard for programmers to write or verification tools to find. Finally, we expect that the approach we have pioneered here — generating candidate solutions that are subsequently formally checked for correctness — should transfer to other domains (e.g., legal arguments, transport signaling, structural engineering) where solutions must be correct, where that correctness must be demonstrated, explained and understood by designers and end-users.
- 12 July 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 1 (FSE) , 812-835
Top Cited PapersOpen AccessTowards AI-Assisted Synthesis of Verified Dafny Methods
Large language models show great promise in many domains, including programming. A promise is easy to make but hard to keep, and language models often fail to keep their promises, generating erroneous code. A promising avenue to keep models honest is to incorporate formal verification: generating programs’ specifications as well as code, so that the code can be proved correct with respect to the specifications. Unfortunately, existing large language models show a severe lack of proficiency in verified programming. In this paper, we demonstrate how to improve two pretrained models’ proficiency in the Dafny verification-aware language. Using 178 problems from the MBPP dataset, we prompt two contemporary models (GPT-4 and PaLM-2) to synthesize Dafny methods. We use three different types of prompts: a direct Contextless prompt; a Signature prompt that includes a method signature and test cases, and a Chain of Thought (CoT) prompt that decomposes the problem into steps and includes retrieval augmentation generated example problems and solutions. Our results show that GPT-4 performs better than PaLM-2 on these tasks, and that both models perform best with the retrieval augmentation generated CoT prompt. GPT-4 was able to generate verified, human-evaluated, Dafny methods for 58% of the problems, however GPT-4 managed only 19% of the problems with the Contextless prompt, and even fewer (10%) for the Signature prompt. We are thus able to contribute 153 verified Dafny solutions to MBPP problems, 50 that we wrote manually and 103 synthesized by GPT-4. Our results demonstrate that the benefits of formal program verification are now within reach of codegenerating large language models. Likewise, program verification systems can benefit from large language models, whether to synthesize code wholesale, to generate specifications, or to act as a “programmer’s verification apprentice”, to construct annotations such as loop invariants which are hard for programmers to write or verification tools to find. Finally, we expect that the approach we have pioneered here — generating candidate solutions that are subsequently formally checked for correctness — should transfer to other domains (e.g., legal arguments, transport signaling, structural engineering) where solutions must be correct, where that correctness must be demonstrated, explained and understood by designers and end-users.
- Open Access
- 15 March 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (3) , 1-35
SourcererJBF: A Java Build Framework For Large-Scale Compilation
Researchers and tool developers working on dynamic analysis, software testing, automated program repair, verification, and validation, need large compiled, compilable, and executable code corpora to test their ideas. The publicly available corpora are relatively small, and/or non-compilable, and/or non-executable. Developing a compiled code corpus is a laborious activity demanding significant manual effort and human intervention. To facilitate large-scale program analysis research, we develop SourcererJBF , a J ava B uild F ramework that can automatically build a large Java code corpus without project-specific instructions and human intervention. To generate a compiled code corpus, SourcererJBF creates an offline knowledge base by collecting external dependencies from the project directories and existing build scripts (if available). It constructs indices of those collected external dependencies that enable a fast search for resolving dependencies during the project compilation. As the output of the large-scale compilation, it produces JAigantic, a compilable Java corpus containing compiled projects, their bytecode, dependencies, normalized build script, and build command. We evaluated SourcererJBF’s effectiveness, correctness, performance, and scalability in a large collection of Java projects. Our experimental results demonstrate that SourcererJBF is significantly effective and scalable in building large Java code corpus. Besides, it substantiates reasonable performance and correctness similar to projects’ existing build systems.
- 15 March 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (3) , 1-35
Open AccessSourcererJBF: A Java Build Framework For Large-Scale Compilation
Researchers and tool developers working on dynamic analysis, software testing, automated program repair, verification, and validation, need large compiled, compilable, and executable code corpora to test their ideas. The publicly available corpora are relatively small, and/or non-compilable, and/or non-executable. Developing a compiled code corpus is a laborious activity demanding significant manual effort and human intervention. To facilitate large-scale program analysis research, we develop SourcererJBF , a J ava B uild F ramework that can automatically build a large Java code corpus without project-specific instructions and human intervention. To generate a compiled code corpus, SourcererJBF creates an offline knowledge base by collecting external dependencies from the project directories and existing build scripts (if available). It constructs indices of those collected external dependencies that enable a fast search for resolving dependencies during the project compilation. As the output of the large-scale compilation, it produces JAigantic, a compilable Java corpus containing compiled projects, their bytecode, dependencies, normalized build script, and build command. We evaluated SourcererJBF’s effectiveness, correctness, performance, and scalability in a large collection of Java projects. Our experimental results demonstrate that SourcererJBF is significantly effective and scalable in building large Java code corpus. Besides, it substantiates reasonable performance and correctness similar to projects’ existing build systems.
- Open Access
- 14 June 2022
- Published by Association for Computing Machinery (ACM)
- Vol. 24, 30-35
ADA: a tool for visualizing the architectural overview of open-source repositories
Writing highly maintainable and efficient software code is becoming increasingly difficult, especially while following the rapid, agile development process and working in a distributed team. One of the key indicators of that inefficient software design is a high degree of code coupling, which leads to unwanted side-effects during refactoring and acts as a burden during future development. To alleviate these problems, we developed a visualization tool, ADA, that statically analyzes an open-source repository and seeks to address the issue of code coupling by providing developers with a powerful graphic representation. ADA showcases the relationships and the degree of inter-connectivity between the classes. ADA will ultimately guide developers to instantly locate the coupled area and assist them in decoupling it.
- 14 June 2022
- Published by Association for Computing Machinery (ACM)
- Vol. 24, 30-35
Open AccessADA: a tool for visualizing the architectural overview of open-source repositories
Writing highly maintainable and efficient software code is becoming increasingly difficult, especially while following the rapid, agile development process and working in a distributed team. One of the key indicators of that inefficient software design is a high degree of code coupling, which leads to unwanted side-effects during refactoring and acts as a burden during future development. To alleviate these problems, we developed a visualization tool, ADA, that statically analyzes an open-source repository and seeks to address the issue of code coupling by providing developers with a powerful graphic representation. ADA showcases the relationships and the degree of inter-connectivity between the classes. ADA will ultimately guide developers to instantly locate the coupled area and assist them in decoupling it.