Letting Tutor Personas "Speak Up" for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization
Preprint
Abstract
With the emergence of large language models (LLMs) as a powerful class of generative artificial intelligence (AI), their use in tutoring has become increasingly prominent. Prior works on LLM-based tutoring typically learn a single tutor policy and do not capture the diversity of tutoring styles. In real-world tutor-student interactions, pedagogical intent is realized through adaptive instructional strategies, with tutors varying the level of scaffolding, instructional directiveness, feedback, and affective support in response to learners' needs. These differences can all impact dialogue dynamics and student engagement. In this paper, we explore how tutor personas embedded in human tutor-student dialogues can be used to guide LLM behavior without relying on explicitly prompted instructions. We modify Bidirectional Preference Optimization (BiPO) to learn a steering vector, an activation-space direction that steers model responses towards certain tutor personas. We find that this steering vector captures tutor-specific variation across dialogue contexts, improving semantic alignment with ground-truth tutor utterances and increasing preference-based evaluations, while largely preserving lexical similarity. Analysis of the learned directional coefficients further reveals interpretable structure across tutors, corresponding to consistent differences in tutoring behavior. These results demonstrate that activation steering offers an effective and interpretable way for controlling tutor-specific variation in LLMs using signals derived directly from human dialogue data.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization2024
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model2023
- Training Millions of Personalized Dialogue Agents2018
- The Use of Worked Examples as a Substitute for Problem Solving in Learning AlgebraCognition and Instruction, 1985
- THE ROLE OF TUTORING IN PROBLEM SOLVING *Journal of Child Psychology and Psychiatry, 1976