Michele Carlo

This study tests whether equivalent instructions alter the linguistic features of writing generated at Common European Framework of Reference for Languages (CEFR) levels. Four instruction-tuned language models each generated responses at four CEFR levels across ten writing tasks, five equivalent instruction wordings, and five repetitions per condition (N = 4,000 texts). Twenty prespecified linguistic measures described vocabulary, grammar, discourse, and stance in each generated text. Requested level accounted for 27.8% of variation in these measures, whereas wording accounted for 0.1%; dispersion did not differ significantly across wordings. Classifier accuracy changed by at most 0.6 percentage points for an unseen wording, compared with an 11–16-point loss for an unseen task. Task effects remained large after adjustment for response length. Within these five paraphrases, task choice mattered more than instruction wording, supporting wording robustness under the tested conditions without establishing that generated texts validly represent human proficiency.

This study compared the linguistic resources writers use to express attitudes, signal certainty, and interact with readers, that is, stance and engagement patterns. We examined these patterns in large language model (LLM)‐generated essays and human learner essays across Common European Framework of Reference for Languages (CEFR) levels A2 through C1. We analyzed 592 human essays alongside 4724 LLM responses to identical writing tasks produced by six models. Task‐controlled mixed‐effects models revealed systematic divergences that exceed what raw frequencies suggest, indicating that structural proficiency can mask functional misalignments. At A2, LLMs overused attitude markers (71% higher than humans) while underusing engagement markers (22% lower). At upper levels, gaps widened further: self‐mention collapsed to 41% of human rates at B2 and 27% at C1, engagement markers fell to 27% at both B2 and C1, and hedges and boosters dropped to approximately 61% of human rates at C1. These patterns held across all six models, indicating that CEFR‐conditioned generation produces a functional profile that is consistent across models but mismatched to human writing: it associates lower proficiency with evaluative excess and higher proficiency with suppressed interactional marking. When LLM texts serve as models or feedback sources in language teaching, practitioners can expect stance and engagement profiles to diverge from human learner writing in level‐dependent ways.

Large language models (LLMs) can generate text that resembles L2 learner writing at requested proficiency levels, but whether such texts adapt to communicative contexts as human learners do remains unclear. This study compared LLM‐generated and human second‐language writing across three task‐defined categories (informal–personal, semi‐formal–expository, formal–argumentative) and four CEFR levels (A2–C1) using Biber's multi‐dimensional analysis (MDA). Six instruction‐tuned models responded to Write & Improve prompts ( n = 592 human; n = 4724 LLM), with EFCAMDAT as a human baseline. Under CEFR‐only prompting, LLM outputs shifted on MDA dimensions tied to informational density and explicitness, but showed weaker, differently directed, or less variable differentiation on dimensions tied to persuasion and elaboration, especially in formal–argumentative tasks. The largest observed gap was at C1, where CEFR descriptors most emphasize register flexibility. LLM outputs were more uniform than human writing and clustered near the center of the human distributional space, showing less variation than authentic learner texts. The broad pattern recurred across all six model families. CEFR‐conditioned generation can reproduce some structural signals of proficiency while showing limited human‐like register differentiation in MDA space, a pattern we interpret as form without function.

This study investigates how Large Language Models (LLMs) encode second language (L2) writing proficiency distinctions compared to human learners, focusing on the structural alignment between synthetic outputs and human developmental patterns. We analyzed CEFR-graded Write & Improve 2024 learner essays and matched LLM generations (A2–C1) using statistical models controlling for prompt and text length. With a CALF feature set (Complexity, Accuracy, Lexical Complexity, Fluency), level explained 2.9 % of variance in human texts ( R ² = 0.029) but 10.6 % in pooled LLM outputs ( R ² = 0.106), indicating clearer A2–C1 features in LLM-generated texts overall. LLM level effects were robust ( R ² range = 0.038–0.454), with sharper differences between neighboring CEFR bands than in learner essays (e.g., A2–B1 distance, D ² = 4.81 versus 1.07). Prompt-wise analyses found effects for all 10 prompts in the LLM set but only 2/10 in the learner set, with much greater separation than in the learner cohort. These results support CALF and CEFR construct validity: with prompt and length controlled, A2–C1 texts show ordered CALF gradients, especially for LLM texts across prompts. The sharper LLM separation likely reflects idealized, low-variance production; therefore, CALF captures core structural proficiency (syntax, lexis, accuracy, fluency) but not discourse-pragmatic qualities or human variability. The analyses provide level-calibrated evidence that LLM texts show clearer distinctions than learners’, under identical prompts, whose development is gradual and overlapping. This positions LLMs as both tool (exemplars, rubric calibration) and challenge (assessment validity and fairness), while offering SLA researchers insight into how proficiency constructs are encoded in human versus model-based writing.

This study presents a comprehensive evaluation of multilingual sentiment categorization performance using locally deployed large language models (LLMs) on consumer-grade hardware, focusing on GDPR-compliant implementation scenarios. Through extensive Monte Carlo validation involving 947,700 classifications over 702 iterations, we demonstrate significant performance capabilities across English, Italian, and Japanese languages while operating within consumer hardware constraints. Using lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half on a Python-based llama-cpp framework on consumer NVIDIA GPU hardware, English achieved 96.3% accuracy (95% CI: 0.963–0.964), with Italian and Japanese showing strong performance at 92.2% (95% CI: 0.921–0.922) and 90.7% (95% CI: 0.906–0.908) respectively. Notably, our analysis demonstrates that plurality voting can achieve extremely high confidence levels across all languages, suggesting an efficient approach to improving classification reliability without requiring extensive computational resources.