Kaixin Lan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Kaixin Lan has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Jing Zhang
Universidade de Macau
3 shared publications - Mu You
University of Macau
3 shared publications - Derek Fai Wong
University of Macau
2 shared publications - Jiaxu Zuo
University of Macau, China
2 shared publications - Lidia Sam Chao
University of Macau
2 shared publications - Tao Fang
Guangdong Institute of Intelligence Science and Technology, Zhuhai, Guangdong, China
2 shared publications
Latest Publications
- 29 July 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 333
Masked cloze scoring (MCS): Zero-shot automated essay scoring via masked predictions from large language models
- 29 July 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 333
Masked cloze scoring (MCS): Zero-shot automated essay scoring via masked predictions from large language models
- Open Access
- 14 February 2026
- Published by Elsevier in SSRN Electronic Journal
Masked Cloze Scoring (MCS): Zero-Shot Automated Essay Scoring via Masked Predictions from Large Language Models
Traditional Automated Essay Scoring (AES) methods typically require large amounts of labeled data and extensive model training. Recent advances in Large Language Models (LLMs) have enabled zero-shot AES methods that bypass these requirements. In this paper, we propose Masked Cloze Scoring (MCS), a novel zero-shot method that leverages LLMs’ masked language modeling capacity as its core mechanism for evaluating how well an essay aligns with model predictions. Unlike prior methods that prompt LLMs to assign scores directly, MCS integrates two complementary signals: in-context word prediction for micro-level lexical appropriateness, and relative essay length as a proxy for macro-level writing quality. Experimental results demonstrate that MCS consistently outperforms existing training-free zero-shot baselines on two widely used benchmark datasets, ASAP and TOEFL11. Compared with LCES, the state-of-the-art zero-shot method that requires additional training, MCS underperforms on TOEFL11 but achieves comparable performance on ASAP. Moreover, MCS attains performance comparable to that of three strong supervised cross-prompt baselines on ASAP. Beyond holistic scoring, MCS produces word-level diagnostic feedback, offering added pedagogical benefits. Overall, MCS advances the state of training-free zero-shot AES, delivering competitive performance without requiring labeled data or additional training.
- 14 February 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessMasked Cloze Scoring (MCS): Zero-Shot Automated Essay Scoring via Masked Predictions from Large Language Models
Traditional Automated Essay Scoring (AES) methods typically require large amounts of labeled data and extensive model training. Recent advances in Large Language Models (LLMs) have enabled zero-shot AES methods that bypass these requirements. In this paper, we propose Masked Cloze Scoring (MCS), a novel zero-shot method that leverages LLMs’ masked language modeling capacity as its core mechanism for evaluating how well an essay aligns with model predictions. Unlike prior methods that prompt LLMs to assign scores directly, MCS integrates two complementary signals: in-context word prediction for micro-level lexical appropriateness, and relative essay length as a proxy for macro-level writing quality. Experimental results demonstrate that MCS consistently outperforms existing training-free zero-shot baselines on two widely used benchmark datasets, ASAP and TOEFL11. Compared with LCES, the state-of-the-art zero-shot method that requires additional training, MCS underperforms on TOEFL11 but achieves comparable performance on ASAP. Moreover, MCS attains performance comparable to that of three strong supervised cross-prompt baselines on ASAP. Beyond holistic scoring, MCS produces word-level diagnostic feedback, offering added pedagogical benefits. Overall, MCS advances the state of training-free zero-shot AES, delivering competitive performance without requiring labeled data or additional training.
- Open Access
- 30 September 2025
- Published by Universidade Federal de Santa Catarina (UFSC) in Cadernos de Tradução
- Vol. 45 (esp. 3) , 1-24
How well can state-of-the-art machine translation systems render a 16th-century Chinese novel?
This study evaluates the performance of state-of-the-art machine translation systems in rendering Journey to the West, a culturally rich 16th-century Chinese novel, into Portuguese. Employing a mixed-methods approach, we compare translations produced by DeepSeek-V3, GPT-4o, DeepL Pro, and NovelTrans-J against a published human translation. Quantitative assessments conducted by an expert evaluator examine accuracy, fluency, stylistic elegance, cultural appropriateness, and overall translation quality at both the sentence and chunk levels. The results reveal that three MT systems (DeepSeek-V3, GPT-4o, and NovelTrans-J) produce translations of comparable or superior quality to the human translation. Among them, NovelTrans-J consistently outperforms all other participants, particularly in terms of cultural appropriateness. In contrast, DeepL Pro demonstrates significantly weaker performance across all evaluated dimensions. To complement the quantitative analysis, a qualitative investigation focuses on the rendering of culture-specific items (CSIs). NovelTrans-J exhibits outstanding performance, producing the fewest mistranslations and uniquely providing explanatory notes that facilitate reader comprehension. DeepSeek-V3 and GPT-4o also handle CSIs competently, though with less consistency, while DeepL Pro struggles considerably, showing a high rate of CSI mistranslations and generally low quality. Interestingly, the human translation also contains notable CSI-related errors, particularly in cases involving semantically opaque expressions, an area in which all participants encounter significant difficulty. These findings underscore the growing potential of MT systems to handle complex, culturally rich literary texts, although certain challenges, such as the translation of semantically opaque expressions, remain significant obstacles. We hope this study provides an updated perspective on the current capabilities of MT and offers practical insights to guide the development of future systems that can more accurately capture and transmit the distinctive cultural nuances embedded in literary works.
- 30 September 2025
- Published by Universidade Federal de Santa Catarina (UFSC) in Cadernos de Tradução
- Vol. 45 (esp. 3) , 1-24
Open AccessHow well can state-of-the-art machine translation systems render a 16th-century Chinese novel?
This study evaluates the performance of state-of-the-art machine translation systems in rendering Journey to the West, a culturally rich 16th-century Chinese novel, into Portuguese. Employing a mixed-methods approach, we compare translations produced by DeepSeek-V3, GPT-4o, DeepL Pro, and NovelTrans-J against a published human translation. Quantitative assessments conducted by an expert evaluator examine accuracy, fluency, stylistic elegance, cultural appropriateness, and overall translation quality at both the sentence and chunk levels. The results reveal that three MT systems (DeepSeek-V3, GPT-4o, and NovelTrans-J) produce translations of comparable or superior quality to the human translation. Among them, NovelTrans-J consistently outperforms all other participants, particularly in terms of cultural appropriateness. In contrast, DeepL Pro demonstrates significantly weaker performance across all evaluated dimensions. To complement the quantitative analysis, a qualitative investigation focuses on the rendering of culture-specific items (CSIs). NovelTrans-J exhibits outstanding performance, producing the fewest mistranslations and uniquely providing explanatory notes that facilitate reader comprehension. DeepSeek-V3 and GPT-4o also handle CSIs competently, though with less consistency, while DeepL Pro struggles considerably, showing a high rate of CSI mistranslations and generally low quality. Interestingly, the human translation also contains notable CSI-related errors, particularly in cases involving semantically opaque expressions, an area in which all participants encounter significant difficulty. These findings underscore the growing potential of MT systems to handle complex, culturally rich literary texts, although certain challenges, such as the translation of semantically opaque expressions, remain significant obstacles. We hope this study provides an updated perspective on the current capabilities of MT and offers practical insights to guide the development of future systems that can more accurately capture and transmit the distinctive cultural nuances embedded in literary works.