Abstract
A recent study investigating the effectiveness of a large language model (LLM) tutor in supporting non-native English-speaking students (NNES) in introductory computer science courses showed promising results. It demonstrated that LLMs can be leveraged to better support NNES by facilitating learning in their native languages. In this study, we repeat the previous experiment to confirm the results while addressing some key validity threats. We used a more advanced LLM, introduced the LLM Tutor from the beginning of the course, and enabled multilingual responses instead of English-only ones. For LLM Tutor usage, we found that (a) NNES showed a marginally higher enrollment rate compared to NES; (b) both groups demonstrated similar overall levels of CodeHelp usage; (c) NES and NNES exhibited parallel temporal usage patterns; (d) NNES made substantially greater use of non-English and multilingual queries, whereas NES queries were almost exclusively in English; and (e) multilingual queries often combined English programming keywords with other natural languages, reflecting code-meshed language use. Overall, students viewed the LLM Tutor as an immediate and valuable aid for independent learning, particularly valuing its guided support rather than direct answers. However, some expressed concerns regarding its accuracy, usability, and potential to foster over-reliance or compromise academic integrity. Some other students preferred other available resources or were unaware of the tool. These findings suggest that while LLM-based tools hold strong potential to enhance self-directed learning, their design and implementation should balance accessibility and guidance with safeguards that promote responsible and critical use.