Absorbed or Anchored? How Large Language Models Represent Embedded Words in Modern Standard Arabic–Egyptian Arabic Code-Switching
Open Access
- 5 November 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (21) , 5141
Abstract
We study how linear probes read Modern Standard Arabic (MSA)–Egyptian Arabic code-switching in Linguistic Code-switching Evaluation (LinCE) tweets using Qwen3-8B-Base, ALLaM-7B-Instruct-preview, Gemma-2-9B and Fanar-1-9B. Probes trained on monolingual text classify 54–56% of MSA words in Egyptian contexts as Egyptian, but only 4–8% of Egyptian words in MSA contexts as MSA. Lexical and contextual controls reduce the gap, leaving 12–20 percentage points after regression and 19–28 after matching. Probes trained on balanced code-switched data recover the inserted word’s variety with 96–98% accuracy, so the asymmetry does not imply loss of variety information. Probe readings correlate with next-token variety preference. Replacing the outputs of five patching-selected attention heads with reference means changes both measures relative to layer-matched random heads, but reduces MSA-to-Egyptian misclassification by only 1–14 percentage points. Effects change direction with a different replacement mean and also occur in monolingual text. External news comments reproduce the directional readout gap (38–69 points), although the difference between lexically matched directional shifts remains unresolved. Register remains a possible contributor, and head selection and interventions use one tweet corpus. Probe-based analyses should therefore report switching directions separately and distinguish monolingual readouts from recoverable word variety.Keywords
This publication has 55 references indexed in Scilit:
- The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation AnalysisComputational Linguistics, 2026
- A panoramic survey of natural language processing in the Arab worldCommunications of the ACM, 2021
- Arabic Dialect IdentificationComputational Linguistics, 2014
- Against markedness (and what to replace it with)Journal of Linguistics, 2006
- Revisiting Arabic diglossic switching in light of the MLF model and its sub-models: the 4-M model and the Abstract Level modelBilingualism: Language and Cognition, 2003
- Duelling Languages1993
- Social Motivations For Codeswitching1993
- An Introduction to the Bootstrap1993
- A comparison of the Two One-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailabilityJournal of Pharmacokinetics and Biopharmaceutics, 1987
- DiglossiaWORD, 1959