Barbara Leporini

As multimodal interaction becomes central to digital learning, technical interfaces, such as eBooks and conversational agents, present unique accessibility challenges for users relying on the auditory modality. While modern Text-to-Speech (TTS) systems excel at rendering natural language, they often fail to convey the complex, structural syntax of source code, creating a significant barrier for blind and low-vision programmers. We frame the vocal rendering of source code as a cross-modal translation task, in which spatial and symbolic information visually encoded in code must be mapped into a linear auditory signal. This study explores "Code-to-Speech" (CTS) as a critical component of multimodal technical communication. We present a comparative evaluation of five leading AI-driven TTS engines (including ChatGPT, Gemini, and ElevenLabs) to determine their efficacy in rendering mixed-content (text and code) materials. Adopting a mixed-methods approach, we combine automated linguistic metrics (WER, BLEU, METEOR) with a human-centered qualitative assessment. Our findings reveal significant differences: ChatGPT and Gemini consistently achieved lower error rates, while Copilot omitted critical tokens. However, all systems struggle with essential structural cues such as indentation, pauses, and code comments. We also show that designed prompts improve intelligibility, particularly for novices. Our work establishes a benchmark for CTS and underscores the need for multimodal AI architectures that support inclusive technical interaction.

The global prevalence of visual disabilities is increasing, significantly impacting the quality of life and social inclusion of those affected. Existing technologies, while helpful, continue to face challenges in terms of cost, accessibility, and real-time effectiveness. Individuals with visual impairments face significant barriers in their daily interactions with the environment, limiting their independence and social participation. The lack of adaptable and personalized solutions that address these specific needs remains a significant barrier. This study presents an extension of the Modeling Scenarios of the Internet of Things (MoSIoT) framework, a platform designed to improve the quality of life of people with disabilities by enhancing their interactions with IoT devices. The enhanced MoSIoT framework uses artificial intelligence and augmented reality technologies to significantly improve navigation and environmental interaction, enhancing the well-being of individuals with visual impairments. The proposed enhancements include advanced object detection, scene recognition, and augmented reality geolocation, all tailored to provide a richer and more autonomous user experience. A case study conducted as part of this research illustrates the practical application of these technologies in a real-world scenario, demonstrating how they can be used to support individuals with visual impairments in various everyday contexts. Practical tests conducted using tools such as YOLO for object recognition and Azure IoT for device integration have significantly improved user autonomy and quality of life. The results show that the enhanced framework can facilitate more effective environmental interaction and better social inclusion, highlighting its potential for implementation across different contexts and devices.

1. In the domains of Science, Technology, Engineering, and Mathematics (STEM), visual representations are not merely illustrative aids; they are the primary medium for representing complex data, ab...