Arbi Haza Nasution

Multilingual pretrained language models have enabled effective cross-lingual transfer for many natural language processing tasks; however, their performance remains limited for low-resource languages, even within typologically related language families. A major cause is severe resource imbalance, where high-resource languages dominate training and induce negative transfer on closely related but under-resourced languages, particularly for structure-sensitive tasks such as Named Entity Recognition (NER). This paper proposes Curriculum-Guided Contrastive Learning (CGCL), a unified training framework for low-resource cross-lingual NER that jointly addresses training dynamics and representation alignment. CGCL integrates multi-view contrastive learning with adaptive curriculum scheduling that explicitly controls both language-level exposure and instance-level difficulty during training. While the framework is language-family agnostic, it allows difficulty modeling to adapt to family-specific structural properties, including agglutinative morphology in Turkic languages and lexical variability in the Austronesian language family. We conduct a comprehensive evaluation on a Wikipedia-based NER benchmark across two typologically distinct but structurally comparable language families: Turkic languages and Austronesian languages. Using a unified experimental protocol, CGCL consistently outperforms multilingual fine-tuning, instruction-tuned baselines, and approaches that employ curriculum learning or contrastive learning in isolation. The largest improvements are observed in the lowest-resource languages and in structurally challenging instances, demonstrating CGCL’s effectiveness in mitigating negative transfer. These results indicate that how multilingual models are trained is at least as important as the data they are trained on, and highlight curriculum-guided contrastive learning as a scalable and robust solution for low-resource cross-lingual NER.

Children increasingly consume online video content, creating a growing need for scalable approaches to support content moderation workflows. However, directly identifying harmful or policy-violating content, such as violence, sexual content, or self-harm, remains a complex task that typically requires specialized classifiers and domain-specific annotations. In this context, sentiment analysis can provide complementary information by capturing affective signals expressed through language and visual cues. This study does not treat sentiment polarity as a direct indicator of unsafe or policy-violating content. Instead, it explores multimodal sentiment analysis as an auxiliary triage signal that may help prioritize content for human review or identify segments requiring further inspection. This paper investigates the feasibility of using large vision–language models (LVLMs) for zero-shot multimodal sentiment analysis on utterance-aligned video segments. We evaluate two LVLMs, LLaVA-OneVision-7B and Qwen2.5-VL-7B, under three input settings: text-only, vision-only, and multimodal, using a conversational TV-series dataset consisting of short utterance-level video segments and transcripts. The results show that multimodal sentiment inference can provide useful screening signals without task-specific fine-tuning, although the benefits are model-dependent. LLaVA-OneVision-7B consistently outperforms Qwen2.5-VL-7B and benefits more clearly from combining textual and visual inputs, whereas Qwen2.5-VL-7B shows limited improvement across modality settings. We also analyze the trade-off between frame sampling and image resolution. Finally, we discuss limitations related to dataset scope, annotation subjectivity, class imbalance, and the need for broader validation before real-world deployment.

Instruction tuning has emerged as an effective mechanism for adapting large language models (LLMs) to biomedical natural language processing (BioNLP), enabling improved task generalization with reduced reliance on extensive domain-specific pretraining. However, empirical evidence shows that its effectiveness varies substantially across biomedical tasks, and the factors governing these variations remain insufficiently understood. In particular, prior work often treats instruction tuning as a uniform adaptation strategy, without systematically examining the roles of task structure, instruction design, and robustness. In this work, we present a comprehensive, structure-aware and task-aware evaluation of instruction tuning for biomedical NLP. We organize biomedical tasks according to output granularity and rigidity, and analyze how instruction structures interact with token-, sentence-, and document-level tasks across multiple model families. To move beyond performance-centric evaluation, we introduce diagnostic measures that capture instruction sensitivity, format violations, and safety-related failure modes. Our results reveal a consistent relationship between output rigidity and instruction tuning gain: tasks with flexible output spaces benefit more from instruction tuning than schema-rigid tasks. We further show that instruction sensitivity varies with task granularity and instruction structure, while robustness and safety considerations expose trade-offs, and that robustness and safety considerations expose trade-offs that are not reflected in standard accuracy metrics alone. Based on these findings, we derive practical design guidelines for instruction construction, model selection, and evaluation in biomedical settings. This study reframes instruction tuning as a structured design problem rather than a one-size-fits-all approach.

Large Language Models such as ChatGPT, Gemini, and recent open-source systems have shown strong performance in natural language tasks. However, their reliability and pedagogical suitability in low-resource educational settings—especially in high-stakes multiple-choice exams that require explicit reasoning—remain underexplored. This paper benchmarks six LLMs—ChatGPT (GPT-4), Gemini 1.5 Pro, Microsoft Copilot, DeepSeek-V2 (16B), Qwen 3 (14B), and Phi-2 (2.7B)—on Turkish and Indonesian exam questions. The evaluation uses 2,000 items in total: 1,000 Turkish graduate admission questions and 1,000 Indonesian vocational questions. We assess (i) answer accuracy, (ii) justification quality (coherence, option comparison, reasoning depth, and Bloom-level alignment), and (iii) temporal consistency under repeated prompting. Expert-written rationales are used as a reference for human alignment. Quantitative analysis combines multilingual semantic similarity, lexical overlap, and coherence deviation measures, supported by SHAP-based feature attribution and UMAP visualization to examine explanation patterns across models. The results show clear variation in both correctness and justification behavior. GPT-4 produces the most consistently high-quality and pedagogically aligned explanations, while Gemini and Qwen 3 achieve competitive but less stable performance across cognitive categories. DeepSeek-V2 performs strongly on several reasoning metrics but shows lower temporal stability, whereas Copilot and Phi-2 are weaker on higher-order reasoning. These findings have direct implications for educational equity, student achievement, and quality education in low-resource contexts, particularly where AI-assisted assessment tools may influence access to fair and reliable academic evaluation. The results also highlight risks such as fluent but incorrect explanations and instability across runs, which should be carefully considered when deploying LLMs in formal assessment environments.

This paper investigates the use of large language models (LLMs) as evaluators in multidimensional machine translation (MT) assessment, focusing on the English–Indonesian language pair. Building on established evaluation frameworks, we adopt an MQM-aligned rubric that assesses translation quality along morphosyntactic, semantic, and pragmatic dimensions. Three LLM-based translation systems (Qwen 3 (0.6B), LLaMA 3.2 (3B), and Gemma 3 (1B)) are evaluated using both expert human judgments and an LLM-based evaluator (GPT–5), allowing for a detailed comparison of alignment, bias, and consistency between human and AI-based assessments. In addition, a classroom calibration study is conducted to examine how rubric-guided evaluation supports alignment among novice evaluators. The results indicate that GPT–5 exhibits strong agreement with human evaluators in terms of relative quality ranking, while systematic differences in absolute scoring highlight calibration challenges. Overall, this study provides insights into the role of LLMs as reference-free evaluators for MT and illustrates how multidimensional rubrics can support both research-oriented evaluation and pedagogical applications in a mid-resource language setting.

This paper introduces the Mixture-of-Agents (MoA) framework, a structured approach for improving the reliability of large language model (LLM)-based text annotation in low-resource NLP contexts. MoA employs coordinated agent interactions to enhance agreement, interpretability, and robustness without manual supervision. Evaluations on Turkish classification benchmarks demonstrate that MoA achieves up to 10-point improvements in macro-F1 over single-model baselines and significantly increases inter-agent consistency. Additionally, three novel reliability metrics—Conflict Rate (CR), Ambiguity Resolution Success Rate (ARSR), and Refinement Correction Rate (RCR)—are proposed to quantify annotation stability and correction dynamics. The results indicate that multi-agent coordination can substantially improve label quality, offering a scalable pathway toward trustworthy annotation in low-resource and cross-domain applications. The framework is language-agnostic and adaptable to other low-resource contexts beyond Turkish, including morphologically rich or typologically diverse languages such as Indonesian, Urdu, and Swahili. These findings highlight the scalability of MoA as a generalizable solution for multilingual and cross-domain annotation.

Recent progress in large language models (LLMs) has rekindled the promise of high-quality machine translation (MT), yet evaluation remains a bottleneck. Traditional automatic metrics (e.g., BLEU) are fast but fail to capture semantic and pragmatic nuances reflected in human judgments. We present a multidimensional framework—inspired by MQM—that augments standard metrics (Adequacy, Fluency) with three linguistic dimensions: Morphosyntactic, Semantic, and Pragmatic. We compare three Small Language Models for English→Indonesian: Qwen 3 (0.6B), LLaMA 3.2 (3B), and Gemma 3 (1B). Two controlled experiments are conducted: (i) Preliminary (1,000 translations, GPT-5-only scoring of Adequacy/Fluency + BLEU), and (ii) Final (100 translations, three human experts + GPT-5) on all five metrics. We compute inter-annotator reliability (Krippendorff’s α, weighted κ) and annotator competence (MACE). Results show consistent model ranking (Gemma 3 (1B) > LLaMA 3.2 (3B) > Qwen 3 (0.6B)) and strong GPT-5–human correlation (r = 0.822). To validate practical applicability, a classroom study with 26 translation students tested the metrics in real learning settings. Using the same multidimensional rubric, students rated MT outputs across pre-, post-, and final-test phases. Their mean absolute error (MAE) decreased from 0.97 to 0.83, while Exact Match Rate increased from 0.30 to 0.50 after rubric calibration, demonstrating that the proposed framework and GPT-5 evaluation can be effectively transferred to educational contexts for evaluator training and feedback alignment.