A Source-Record Audit of DeepSeek-R1 Responses to Romanized Sindhi Prompts
Authors:
Open Access
Abstract
Sindhi language models struggle with Romanized Sindhi as there is variation in the spellings of everyday words, and short prompts do not explicitly state the task or target language. In this study, the remaining source record of a small DeepSeek-R1 probe, which was based on seven canonical prompts and fourteen alleged A/B records, was audited. The screenshot captions were compared with the visible inputs and outputs, and duplication of images was checked. It was only relabeled when there was a single unambiguous canonical prompt that matched the visible input to a record. Task choice and exact target-string occurrence were then coded separately. Six records were auditable, three of which were recovered by relabeling; eight records were not auditable due to being duplicated, contaminated with a supplied answer, associated with a different phrase, or missing the user input or lost after label repair. Four of the six audited records adhered to the experimenter’s task, one provided a reasonably good English translation when the experimenter did not include the target language, and one resulted in an incompatible cross-linguistic reading. Five had the target (English/Sindhi) string and one did not. The screenshots do not demonstrate clear “standard” and “chain of thought” conditions: all audited B-labeled inputs are in the form of a simple prompt and there are no paired wrapper texts remaining. These are descriptive counts, not accuracy estimates and do not provide proof of causal benefit of prompt explicitness. The audit demonstrates the necessity of full prompt–response documentation, independently verified linguistic references, reruns, information-matched prompts, and preregistered evaluation plans in studies with low resources.Keywords
This publication has 18 references indexed in Scilit:
- An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's GuidearXiv, 2024
- Prompting Strategies for Citation Classification2023
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsarXiv, 2023
- Faithful Chain-of-Thought Reasoning2023
- Romanized Sindhi Rules for Text CommunicationMehran University Research Journal of Engineering and Technology, 2021
- Study of automatic text summarization approaches in different languagesArtificial Intelligence Review, 2021
- Arabic sentiment analysis: studies, resources, and toolsSocial Network Analysis and Mining, 2019
- Identification of Issues and Challenges in Romanized Sindhi TextInternational Journal of Advanced Computer Science and Applications, 2019
- An empirical study on POS tagging for Vietnamese social media textComputer Speech & Language, 2018
- Printed Text Image Database for Sindhi OCRACM Transactions on Asian and Low-Resource Language Information Processing, 2016