Winda Monika

Children increasingly consume online video content, creating a growing need for scalable approaches to support content moderation workflows. However, directly identifying harmful or policy-violating content, such as violence, sexual content, or self-harm, remains a complex task that typically requires specialized classifiers and domain-specific annotations. In this context, sentiment analysis can provide complementary information by capturing affective signals expressed through language and visual cues. This study does not treat sentiment polarity as a direct indicator of unsafe or policy-violating content. Instead, it explores multimodal sentiment analysis as an auxiliary triage signal that may help prioritize content for human review or identify segments requiring further inspection. This paper investigates the feasibility of using large vision–language models (LVLMs) for zero-shot multimodal sentiment analysis on utterance-aligned video segments. We evaluate two LVLMs, LLaVA-OneVision-7B and Qwen2.5-VL-7B, under three input settings: text-only, vision-only, and multimodal, using a conversational TV-series dataset consisting of short utterance-level video segments and transcripts. The results show that multimodal sentiment inference can provide useful screening signals without task-specific fine-tuning, although the benefits are model-dependent. LLaVA-OneVision-7B consistently outperforms Qwen2.5-VL-7B and benefits more clearly from combining textual and visual inputs, whereas Qwen2.5-VL-7B shows limited improvement across modality settings. We also analyze the trade-off between frame sampling and image resolution. Finally, we discuss limitations related to dataset scope, annotation subjectivity, class imbalance, and the need for broader validation before real-world deployment.

This paper investigates the use of large language models (LLMs) as evaluators in multidimensional machine translation (MT) assessment, focusing on the English–Indonesian language pair. Building on established evaluation frameworks, we adopt an MQM-aligned rubric that assesses translation quality along morphosyntactic, semantic, and pragmatic dimensions. Three LLM-based translation systems (Qwen 3 (0.6B), LLaMA 3.2 (3B), and Gemma 3 (1B)) are evaluated using both expert human judgments and an LLM-based evaluator (GPT–5), allowing for a detailed comparison of alignment, bias, and consistency between human and AI-based assessments. In addition, a classroom calibration study is conducted to examine how rubric-guided evaluation supports alignment among novice evaluators. The results indicate that GPT–5 exhibits strong agreement with human evaluators in terms of relative quality ranking, while systematic differences in absolute scoring highlight calibration challenges. Overall, this study provides insights into the role of LLMs as reference-free evaluators for MT and illustrates how multidimensional rubrics can support both research-oriented evaluation and pedagogical applications in a mid-resource language setting.

Recent progress in large language models (LLMs) has rekindled the promise of high-quality machine translation (MT), yet evaluation remains a bottleneck. Traditional automatic metrics (e.g., BLEU) are fast but fail to capture semantic and pragmatic nuances reflected in human judgments. We present a multidimensional framework—inspired by MQM—that augments standard metrics (Adequacy, Fluency) with three linguistic dimensions: Morphosyntactic, Semantic, and Pragmatic. We compare three Small Language Models for English→Indonesian: Qwen 3 (0.6B), LLaMA 3.2 (3B), and Gemma 3 (1B). Two controlled experiments are conducted: (i) Preliminary (1,000 translations, GPT-5-only scoring of Adequacy/Fluency + BLEU), and (ii) Final (100 translations, three human experts + GPT-5) on all five metrics. We compute inter-annotator reliability (Krippendorff’s α, weighted κ) and annotator competence (MACE). Results show consistent model ranking (Gemma 3 (1B) > LLaMA 3.2 (3B) > Qwen 3 (0.6B)) and strong GPT-5–human correlation (r = 0.822). To validate practical applicability, a classroom study with 26 translation students tested the metrics in real learning settings. Using the same multidimensional rubric, students rated MT outputs across pre-, post-, and final-test phases. Their mean absolute error (MAE) decreased from 0.97 to 0.83, while Exact Match Rate increased from 0.30 to 0.50 after rubric calibration, demonstrating that the proposed framework and GPT-5 evaluation can be effectively transferred to educational contexts for evaluator training and feedback alignment.

Background: This research, titled "Understanding of Social Media Literacy Regarding Malay Cultural Teachings: A Case Study of SMAIT Al-Ittihad Pekanbaru Students," aims to investigate the level of social media literacy among students regarding Malay cultural teachings. Social media literacy is crucial in preserving Malay culture in the digital era. An individual's ability to search for, evaluate, and utilize relevant information on social media is essential to understanding and appreciating their own culture.Objective: This study aims to determine the understanding of social media literacy regarding Malay cultural teachings among SMAIT Al-Ittihad Pekanbaru students by using the Big Six model of information literacy, which integrates information search and use skills with the use of technology in a systematic process to locate, use, evaluate, and communicate information for specific purposes and tasks.Methods: This research employed a quantitative descriptive design. Data was collected using questionnaires and observations. The population of this study was 12th-grade students of SMAIT Al-Ittihad Pekanbaru, totaling 104 students. Purposive sampling was used to select a sample of 103 students. Data analysis was conducted using a calculation based on relative frequency distribution. The sampling technique used in this study was purposive sampling. This technique involves selecting research samples based on certain considerations, aiming for the data obtained to be more representative. The sample for this study consisted of 103 12th-grade students from SMAIT Al-Ittihad Pekanbaru who met specific criteria. The criteria for selecting the research sample were: (1) Students of SMAIT Al-Ittihad Pekanbaru who had already taken the Riau Malay Culture subject, specifically the material on Tunjung Ajar Melayu: "Bercapak bersetinah, berunding berestabik; Be open-minded, open-handed; Know how to listen, be good at interpreting; Win in losing; Know how to live, know how to die inherit" (Speaking with wisdom, discussing with respect; Open-minded, generous; Able to listen, good at concluding; Winning in defeat; Knowing how to leave behind a legacy in life, knowing how to die in death). (2) Students who have an Instagram account.Results: Based on the research findings and data analysis, the understanding of social media literacy regarding Malay cultural teachings in a case study of SMAIT Al-Ittihad Pekanbaru students was 68.34% in the high category, with the following breakdown: (1) Problem formulation was 82.4%, falling into the very high category. (2) Information search strategy was 74.67%, categorized as high. (3) Location and access was 74.16%, categorized as high. (4) Using information was 56.65%, categorized as moderate. (5) Synthesis was 49.95%, categorized as moderate. (6) Evaluation was 77.25%, categorized as high.Conclusion: This study concludes that the overall understanding of social media literacy regarding Malay cultural teachings among SMAIT Al-Ittihad Pekanbaru students is in the high category, with a score of 68.34%. The highest competence lies in the "problem formulation" aspect (82.4%), while the lowest is found in the "synthesis" aspect (49.95%). These findings indicate that while students are proficient in identifying and searching for information, they need further development in processing and re-presenting the information, particularly in synthesizing cultural values from social media content. This underscores the importance of targeted educational interventions to strengthen students' abilities to interpret and communicate cultural knowledge effectively in the digital age.

We benchmark 22 open-source large language models (LLMs) against ChatGPT-4 and human annotators on two NLP tasks—sentiment analysis and emotion classification—for Indonesian tweets. This study contributes to NLP in a relatively low-resource language (Bahasa Indonesia) by evaluating zero-shot classification performance on a labeled tweet corpus. The dataset includes sentiment labels (Positive, Negative, Neutral) and emotion labels (Love, Happiness, Sadness, Anger, Fear).We compare model predictions to human annotations and report precision, recall, and F1-score, along with inference time analysis. ChatGPT-4 achieves the highest macro F1-score (0.84) on both tasks, slightly outperforming human annotators. The best-performing open-source models—such as LLaMA3.1_70B and Gemma2_27B—achieve over 90% of ChatGPT-4’s performance, while smaller models lag behind. Notably, some mid-sized models (e.g., Phi-4 at 14B parameters) perform comparably to much larger models on select categories. However, certain classes—particularly Neutral sentiment and Fear emotion—remain challenging, with lower agreement even among human annotators. Inference time varies significantly: optimized models complete predictions in under an hour, while some large models require several days. Our findings show that state-of-the-art open models can approach closed-source LLMs like ChatGPT-4 on Indonesian classification tasks, though efficiency and consistency in edge cases remain open challenges. Future work should explore fine-tuning multilingual LLMs on Indonesian data and practical deployment strategies in real-world applications.

Phishing URL detection is critical due to the severe cybersecurity threats posed by phishing attacks. While traditional methods rely heavily on handcrafted features and supervised machine learning, recent advances in large language models (LLMs) provide promising alternatives. This paper presents a comprehensive benchmarking study of 21 state-of-the-art open-source LLMs—including Llama3, Gemma, Qwen, Phi, DeepSeek, and Mistral—for phishing URL detection. We evaluate four key prompt engineering techniques—zero-shot, role-playing, chain-of-thought, and few-shot prompting—using a balanced, publicly available phishing URL dataset, with no fine-tuning or additional training of the models conducted, reinforcing the zero-shot, prompt-based nature as a distinctive aspect of our study. The results demonstrate that large open-source LLMs (≥27B parameters) achieve performance exceeding 90% F1-score without fine-tuning, closely matching proprietary models. Among the prompt strategies, few-shot prompting consistently delivers the highest accuracy (91.24% F1 with Llama3.3_70b), whereas chain-of-thought significantly lowers accuracy and increases inference time. Additionally, our analysis highlights smaller models (7B–27B parameters) offering strong performance with substantially reduced computational costs. This study underscores the practical potential of open-source LLMs for phishing detection and provides insights for effective prompt engineering in cybersecurity applications.