Abstract
Bias handling in expert-augmented LLMs is not merely a matter of routing tasks to experts, it requires coordinating how biased cues are detected, how their social meanings are attributed, and how interventions are generated. However, existing parameter-efficient fine-tuning and mixture-of-experts approaches typically organize specialization around task labels, domains, or input-level routing signals, rather than explicitly modeling how evidence of bias should inform its interpretation and subsequent rewriting. This paper proposes CoPAMoE, a cognitive process-aware PEFT-MoE framework for bias understanding and mitigation. CoPAMoE organizes bias-related subtasks into three functional stages, Cue Perception, Attribution Reasoning, and Intervention Control. It combines stage-partitioned LoRA experts, shared experts, a task--stage routing prior, and an Adjacent Stage Gated Coordination (ASGC) module. The resulting architecture explicitly separates stage-specific adaptation from task-general bias signals and supports controlled information transfer between adjacent stages. Experiments on a unified benchmark built from CrowS-Pairs, StereoSet, and WinoBias show that CoPAMoE improves average performance over representative PEFT and MoE baselines on two backbone LLMs. Ablation, routing, representation, oracle, and efficiency analyses further suggest that the gains are associated with stage-aware specialization and cross-stage dependency transfer.

This publication has 18 references indexed in Scilit: