Valence-Aware Gradient Memory and Fractional Kernel Comparisons for LoRA-Based Language Model Unlearning

Selective machine unlearning aims to suppress targeted information while preserving retained and general knowledge. We investigate valence-aware gradient memory within low-rank adaptation (LoRA), separating retention-supporting and forgetting-related directions. Earlier experiments used Gamma-scaled exponential moving averages rather than genuine fractional dynamics; we preserve that distinction and introduce a normalized finite-history Grünwald–Letnikov (GL) fractional-integral accumulator with algebraically decaying coefficients. The study progresses from synthetic transformer optimization and pretrained text classification to causal language model experiments, including a new controlled evaluation on TOFU Forget10 using Qwen2.5-0.5B-Instruct. A SciQ- and WikiText-regularized teacher reduces reference-answer NLL on original and paraphrased TOFU questions while exhibiting a small World Facts penalty in a separate audit. From this fixed teacher, four unlearning methods were compared for 120 updates over three random seeds. Dual GL yields the largest paraphrased forget-set NLL increase ($2.169\pm 0.331$), exceeding dual EMA ($1.964\pm 0.250$), but incurs greater retain-set ($1.043\pm 0.207$ versus $0.914\pm 0.147$) and World Facts ($1.333\pm 0.645$ versus $1.153\pm 0.567$) degradation. Exact paired sign-flip tests do not establish statistical significance ($p=0.25$ for GL versus EMA). The stronger suppression achieved with fractional memory is accompanied by greater disruption of retained and general factual knowledge, suggesting a shift in model behavior rather than more selective forgetting. Whether the targeted information becomes genuinely inaccessible rather than merely less likely to be reproduced remains unresolved.