SignLLM: Sign Language Production Large Language Models
Authors:
Top Cited Papers
- 19 October 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6681-6693
Abstract
In this paper, we propose SignLLM, a multilingual Sign Language Production (SLP) large language model, which includes two novel multilingual SLP modes MLSF and Prompt2LangGloss that allow sign language gestures generation from query texts input and question-style prompts input respectively. Both modes can use a new RL loss based on reinforcement learning and a new RL module named Priority Learning Channel. These RL components can accelerate the training by enhancing the model's capability to sample high-quality data. To train SignLLM, we introduce Prompt2sign, a comprehensive multilingual sign language dataset, which builds from public data, including American Sign Language (ASL) and seven others. This dataset standardizes information by extracting pose information from sign language videos into a unified compressed format. We extensively evaluate SignLLM, demonstrating that our model achieves state-of-the-art performance on SLP tasks across eight sign languages.Keywords
All Related Versions
This publication has 68 references indexed in Scilit:
- Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density NetworksInternational Journal of Computer Vision, 2021
- Progressive Transformers for End-to-End Sign Language Production2020
- CONFIG: Controllable Neural Face Image Generation2020
- XingGAN for Person Image Generation2020
- World-Consistent Video-to-Video Synthesis2020
- Neural Voice Puppetry: Audio-Driven Facial Reenactment2020
- Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial NetworksInternational Journal of Computer Vision, 2020
- The Korean Sign Language Dataset for Action Recognition2019
- Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signersComputer Vision and Image Understanding, 2015
- Large Lexicon Detection of Sign Language2007