Towards Unified Multimodal Search and Generation System: An AI-Driven Text–Image Framework for Arabic
Open Access
- 1 January 2026
- journal article
- Published by Walter de Gruyter GmbH in Open Information Science
- Vol. 10 (1)
Abstract
Recent advances in multimodal learning have significantly enhanced natural language understanding (NLU) and information retrieval across various languages; however, Arabic remains substantially underrepresented in this domain. In this work, we propose an innovative deep learning framework that draws inspiration from the Unified Multimodal Search and Generation System paradigm, combined with OpenAI’s CLIP and DALL⋅E architectures, to enhance Arabic-language image retrieval and generation. Given the highly inflected nature of Arabic and the limited application of modern deep learning methods in Arabic NLP, we fine-tune these models on an Arabic dataset to enable effective text–image alignment as well as the generation of semantically coherent images from Arabic queries. Our findings demonstrate the potential of the proposed framework to advance Arabic multimodal processing and contribute to narrowing the gap between Arabic NLP and recent developments in multimodal learning.Keywords
This publication has 18 references indexed in Scilit:
- The Landscape of Arabic Large Language ModelsCommunications of the ACM, 2025
- Towards Multi-modal Transformers in Federated Learning2024
- A transformer-based unified multimodal framework for Alzheimer's disease assessmentComputers in Biology and Medicine, 2024
- Multitask Vision-Language Prompt Tuning2024
- UniT: Multimodal Multitask Learning with a Unified Transformer2021
- WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning2021
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training2021
- M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training2021
- Microsoft COCO: Common Objects in Context2014
- Framing Image Description as a Ranking Task: Data, Models and Evaluation MetricsJournal of Artificial Intelligence Research, 2013