Mingzhi Zhu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mingzhi Zhu has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Boris Sobolev
Cisco, Woodbridge, NJ, USA
1 shared publication - Ling Yue
Rensselaer Polytechnic Institute, USA
1 shared publication - Michele Merler
IBM Research, Yorktown Heights, NY, USA
1 shared publication - Payel Das
IBM Research Yorktown Heights NY United States
1 shared publication - Rahul Krishna
IBM Research, Yorktown Heights, NY, USA
1 shared publication - Raju Pavuluri
IBM T.J. Watson Research Center, Yorktown Heights, USA
1 shared publication
Latest Publications
- 12 May 2026
- Published by American Association for the Advancement of Science (AAAS) in Health Data Science
- Vol. 6, 0454
BioMamba: Domain-Adaptive Biomedical Language Models
Background: Biomedical language models should improve performance on biomedical text while retaining general-language-modeling fluency. For Mamba-based models, this trade-off has not been systematically studied across biomedical literature and clinical text. Methods: We developed BioMamba, a family of biomedical Mamba2 models at 5 scales (130M, 370M, 780M, 1.3B, and 2.7B) obtained by continued pretraining of released public Mamba2 checkpoints on a balanced 80%/10%/10% mixture of PubMed abstracts, the Colossal Clean Crawled Corpus (C4), and Wikipedia. The architecture is unchanged from Mamba2; the contribution is the adaptation recipe and the accompanying open-weight checkpoints. We evaluated internal language modeling on fixed held-out sets, out-of-domain multiple-choice benchmarks, and 3 downstream tasks across multiple model scales: clinical note completion and discharge summary generation on MIMIC-IV-Note, and biomedical yes/no question answering on Biomedical Semantic Indexing and Question Answering (BioASQ) and PubMed Question Answering (PubMedQA). Results: Across 5 scales, BioMamba consistently lowered PubMed perplexity, improved Wikipedia-style held-out perplexity by 1.46 to 4.72 PPL, and left C4 perplexity essentially unchanged ( ∣ Δ P P L ∣ ≤ 0 .17 ). On 6 out-of-domain multiple-choice benchmarks, BioMamba stayed within ±3 pp of Mamba2 with no systematic regression. After supervised fine-tuning, BioMamba+SFT matched or exceeded Mamba2+SFT on MIMIC-IV note completion and discharge summary generation at every evaluated scale (paired-bootstrap P Δ > 0 ≥ 0 .997 at 2.7B) and improved PubMedQA at every scale. The strongest model (BioMamba-2.7B) reached a PubMed perplexity of 5.28 and accuracies of 90.24% and 73.00% on BioASQ and PubMedQA, respectively. Conclusions: A balanced domain-adaptive continued pretraining recipe strengthens Mamba2 language models on biomedical literature and clinical text while preserving general-language-modeling fluency.
- 12 May 2026
- Published by American Association for the Advancement of Science (AAAS) in Health Data Science
- Vol. 6, 0454
BioMamba: Domain-Adaptive Biomedical Language Models
Background: Biomedical language models should improve performance on biomedical text while retaining general-language-modeling fluency. For Mamba-based models, this trade-off has not been systematically studied across biomedical literature and clinical text. Methods: We developed BioMamba, a family of biomedical Mamba2 models at 5 scales (130M, 370M, 780M, 1.3B, and 2.7B) obtained by continued pretraining of released public Mamba2 checkpoints on a balanced 80%/10%/10% mixture of PubMed abstracts, the Colossal Clean Crawled Corpus (C4), and Wikipedia. The architecture is unchanged from Mamba2; the contribution is the adaptation recipe and the accompanying open-weight checkpoints. We evaluated internal language modeling on fixed held-out sets, out-of-domain multiple-choice benchmarks, and 3 downstream tasks across multiple model scales: clinical note completion and discharge summary generation on MIMIC-IV-Note, and biomedical yes/no question answering on Biomedical Semantic Indexing and Question Answering (BioASQ) and PubMed Question Answering (PubMedQA). Results: Across 5 scales, BioMamba consistently lowered PubMed perplexity, improved Wikipedia-style held-out perplexity by 1.46 to 4.72 PPL, and left C4 perplexity essentially unchanged ( ∣ Δ P P L ∣ ≤ 0 .17 ). On 6 out-of-domain multiple-choice benchmarks, BioMamba stayed within ±3 pp of Mamba2 with no systematic regression. After supervised fine-tuning, BioMamba+SFT matched or exceeded Mamba2+SFT on MIMIC-IV note completion and discharge summary generation at every evaluated scale (paired-bootstrap P Δ > 0 ≥ 0 .997 at 2.7B) and improved PubMedQA at every scale. The strongest model (BioMamba-2.7B) reached a PubMed perplexity of 5.28 and accuracies of 90.24% and 73.00% on BioASQ and PubMedQA, respectively. Conclusions: A balanced domain-adaptive continued pretraining recipe strengthens Mamba2 language models on biomedical literature and clinical text while preserving general-language-modeling fluency.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 29-36
Multi-task Code LLMs: Data Mix or Model Merge?
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 29-36
Open AccessMulti-task Code LLMs: Data Mix or Model Merge?