Pedro Albuquerque Santos

Communication barriers between deaf and hearing individuals present significant challenges to social inclusion, highlighting the need for effective technological aids. This study aimed to bridge this gap by developing a mobile system for the real-time translation of Portuguese Sign Language (LGP) alphabet gestures into text, addressing a specific technological void for LGP. The core of the solution is a mobile application integrating two distinct machine learning approaches trained on a custom LGP dataset: firstly, a Convolutional Neural Network (CNN) optimized with TensorFlow Lite for efficient, on-device image classification, enabling offline use; secondly, a method utilizing MediaPipe for hand landmark extraction from the camera feed, with classification performed by a server-side Multilayer Perceptron (MLP). Evaluation tests confirmed that both approaches could recognize LGP alphabet gestures with good accuracy (F1-scores of approximately 76% for the CNN and 77% for the MediaPipe+MLP) and processing speed (1 to 2 s per gesture on high-end devices using the CNN and 3 to 5 s under typical network conditions using MediaPipe+MLP), facilitating efficient real-time translation, though performance trade-offs regarding speed versus accuracy under different conditions were observed. The implementation of this dual-method system provides crucial flexibility, adapting to varying network conditions and device capabilities, and offers a scalable foundation for future expansion to include more complex gestures. This work delivers a practical tool that may contribute to improve communication accessibility and the societal integration of the deaf community in Portugal.

Parkinson’s disease (PD) is a neurodegenerative disorder characterized by motor and non-motor symptoms. The MoveONParkinson project aims to enhance exercise engagement among people with Parkinson’s Disease (PwPD) in the Portuguese context through the ONParkinson digital platform, which provides mobile and web interfaces. While the broader MoveONParkinson project has been previously described from a health-focused perspective, this study specifically focuses on the development and integration of an AI-driven conversational agent (CA) for the Portuguese language, called PANDORA, within the mobile interface of the solution to assist and motivate PwPD in their exercise routines. PANDORA (Parkinson Assistant in Natural Dialogue and Oriented by Rules and Assessments), designed based on Self-Determination Theory (SDT), addresses the psychological needs of autonomy, competence, and relatedness. A preliminary study involving 20 PwPD, 10 caregivers, and 5 healthcare professionals informed the design requirements for PANDORA. The development process involved four main phases: (1) Design of the Chatbot’s Motivation Model, (2) Design and implementation of the conversational agent, (3) Technical Performance Evaluation, and (4) User Experience Evaluation. Technical Performance Evaluation, conducted with three physiotherapists, assessed domain coverage, coherence response capacity, and dialog management capacity, achieving 100% accuracy in domain coverage and coherence response capacity and 89% in dialog management capacity. The User Experience Study involved eight PwPD users recruited from Portuguese healthcare units performing predefined tasks, with user satisfaction scores ranging from 4.2 to 4.9 on a five-point Likert scale. The findings indicate that integrating a conversational agent with motivational cues tends to increase patient engagement. However, further studies are required to determine PANDORA’s impact on exercise engagement in PwPD.

Given the pivotal role of agriculture in ensuring food security, fostering economic stability, and addressing environmental sustainability, the sector has increasingly embraced smart farming solutions to respond to recent climate and societal challenges, such as rising water and food demands. These solutions provide actionable insights crucial for decision-making, enabling farm stakeholders to optimize resources, improve yields, and mitigate risks. However, the complexity of the predictive models often associated with this type of solutions results in a lack of transparency, hindering trust and adoption. To respond to such challenges, this paper explores the application of explainable AI (XAI) techniques to agriculture tabular data. Specifically, we focus on two case studies: wheat yield prediction and grapes produced for wine purposes yield prediction. Through these case studies, we propose initial contributions on how XAI techniques can be applied in the context of agriculture and how generated explanations can be adapted to the users’ level of expertise. Finally, as part of ongoing and future research directions, we introduce AgriUXE (Agricultural eXperience Enhanced through eXplainability), a novel user-centered digital platform designed to augment the explainability of multimodal data and machine learning model predictions for sustainable smart farming solutions. By providing transparent, data-driven decisions and generating user-adaptive explanations, AgriUXE aims to support the optimization of the user experience within these solutions.

This paper investigates the creation of multimodal web interfaces to foster inclusive services, aiming to empower individuals with disabilities. This paper details the development of SEU (Services to Empower yoU), an online platform designed to improve access to offline services for persons with disabilities through personalised multimodal web interfaces. Developed using a participatory design approach, SEU involved collaboration with Human- Computer Interaction experts, occupational therapists, accessibility experts, and persons with disabilities. The platform supports multiple user needs across cognitive, visual, motor, and hearing impairments. SEU was evaluated in a two-phase usability study, which included expert assessment and testing by persons with disabilities, supplemented with interviews, observations, and questionnaires. Feedback highlighted SEU’s effective design and usability, emphasising its utility in enhancing service accessibility. The study indicates that both experts and persons with disabilities consider the platform an added value. Although the platform was well-received, suggestions for better cognitive support and the addition of a mobile application were noted, acknowledging the regular smartphone usage among the target users.

Objective: In this paper, we introduce a study focused on the creation of multimodal inclusive web interfaces to foster inclusive services, aiming to empower individuals with disabilities. The primary objective of this endeavour was to develop an online platform known as SEU, which facilitates access to offline services for persons with disabilities. SEU accomplishes this by offering user interfaces that are both accessible and adaptable, catering to the diverse needs of various user profiles. Methods: A participatory design methodology was followed. Human-Computer Interaction Experts, Occupational Therapists, Accessibility Experts, and Persons with Disabilities participated actively during the creation process. A first usability study assessed the web platform regarding accessibility issues. This user study comprises two phases. In phase A, the online platform was assessed by the experts (Occupational Therapists and Accessibility Experts), and persons with disabilities tested the platform in phase B. The user study design included interviews, user observation and usability questionnaires posed after completing tasks in a controlled environment for one day. Results: The platform SEU was developed to present multimodal inclusive web interfaces, which change according to the user’s disability. Four different accessibility profiles were considered: cognitive, visual, motor, and hearing impairment. In the user study, four experts and eight end-users with distinct disabilities participated. Experts consider the platform presents important features and is very well designed, providing meaningful feedback. The results obtained with the end-users were equally positive regarding the available features and the usability of the web interfaces. Conclusions: Designing a web interface that is accessible and adaptable for users with distinct disabilities is viable and promising. The preliminary study with experts and PwD indicates they consider it an added value. However, the adaptations for users with cognitive impairments need a 1 deeper study to achieve more efficient solutions. The development of a mobile application interface was suggested by the end users, as most of them use smartphones in everyday life.

Parkinson’s Disease is the second most common neurodegenerative disorder and physical exercise is essential in its treatment process. The use of new technologies to persuade, motivate and activate individuals for health behaviour change has been a quickly expanding field of research. This paper focuses on studying how pervasive serious games can be designed using low-cost off-the-shelf controllers and interaction devices usually used with commercial consoles for gaming. The research work has the ultimate goal of establishing how serious games based on Virtual Reality should be designed according to the impact of the technology on people with Parkinson’s Disease.