Rui Jesus

The degradation of works of art constitutes a significant problem for the preservation of cultural heritage. In the case of paintings, the observed alterations can be physical, chemical, or visual, affecting both the integrity and appearance of the artworks. Degradation compromises the authenticity, aesthetic legibility, and historical value of paintings, making the early monitoring of such issues, as well as the development of appropriate conservation and restoration strategies, essential. For an effective approach, the characterisation of the materials and techniques used by the artist, as well as the degradation processes inherent in the materials used, proves to be crucial. In this context, the application of artificial intelligence (AI) emerges as a non-invasive solution capable of detecting and predicting degradation in works of art. This bibliographic review aims to explore existing studies in this field in depth, with special attention to contemporary paintings considered as case studies. The methodology involved a systematic review of peer-reviewed studies, theses, and interdisciplinary databases, using keywords related to the topic under investigation (e.g., “degradation detection,” “artificial intelligence,” “craquelure segmentation”). The results indicate that artificial intelligence enables the early detection of degradations that may not yet be visible to the naked eye while also improving objectivity and consistency in the analysis of complex and irregular patterns typical of paintings. It became evident that there is a significant gap in the literature, regarding studies addressing the potential of AI for degradation detection specific to contemporary paintings. However, these could be a valuable case study given their potential material and technical heterogeneity, as well as their differences from traditional easel paintings.

Effective water quality monitoring is very important for detecting pollution and protecting public health. However, traditional methods are slow, relying on costly equipment, central laboratories, and expert staffing, which delays real-time measurements. At the same time, significant advancements have been made in the field of plasmonic sensing technologies, making them ideal for environmental monitoring. However, their reliance on large, expensive spectrometers limits accessibility. This work aims to bridge the gap between advanced plasmonic sensing and practical water monitoring needs, by integrating plasmonic sensors with mobile technology. We present BioColor, a mobile platform that consists of a plasmonic sensor setup, mobile application, and cloud services. The platform processes captured colorimetric sensor images in real-time using optimized image processing algorithms, including region-of-interest segmentation, color extraction (mean and dominant), and comparison via the CIEDE2000 metric. The results are visualized within the mobile app, providing instant and automated access to the sensing outcome. In our validation experiments, the system consistently measured color differences in various sensor images captured under media with different refractive indices. A user experience test with 12 participants demonstrated excellent usability, resulting in a System Usability Scale (SUS) score of 93. The BioColor platform brings advanced sensing capabilities from hardware into software, making environmental monitoring more accessible, efficient, and continuous.

Communication barriers between deaf and hearing individuals present significant challenges to social inclusion, highlighting the need for effective technological aids. This study aimed to bridge this gap by developing a mobile system for the real-time translation of Portuguese Sign Language (LGP) alphabet gestures into text, addressing a specific technological void for LGP. The core of the solution is a mobile application integrating two distinct machine learning approaches trained on a custom LGP dataset: firstly, a Convolutional Neural Network (CNN) optimized with TensorFlow Lite for efficient, on-device image classification, enabling offline use; secondly, a method utilizing MediaPipe for hand landmark extraction from the camera feed, with classification performed by a server-side Multilayer Perceptron (MLP). Evaluation tests confirmed that both approaches could recognize LGP alphabet gestures with good accuracy (F1-scores of approximately 76% for the CNN and 77% for the MediaPipe+MLP) and processing speed (1 to 2 s per gesture on high-end devices using the CNN and 3 to 5 s under typical network conditions using MediaPipe+MLP), facilitating efficient real-time translation, though performance trade-offs regarding speed versus accuracy under different conditions were observed. The implementation of this dual-method system provides crucial flexibility, adapting to varying network conditions and device capabilities, and offers a scalable foundation for future expansion to include more complex gestures. This work delivers a practical tool that may contribute to improve communication accessibility and the societal integration of the deaf community in Portugal.

In recent years, there has been growing interest in taking advantage of the technological progress in information technology and computer science to enhance the synergy between multidisciplinary organisations with a mutual objective of improving scientific knowledge and engaging society in cultural activities. Such an example of collaboration networks includes those where governmental, scientific and cultural institutions work in unison to provide services that support research through the use of technology while disseminating information and promoting cultural heritage. Here, we present a case study implementing the results of the work between multidisciplinary departments of the NOVA University Lisbon and third-party cultural heritage organisations. In particular, a mobile and desktop PC application uses augmented reality to showcase results obtained from analysis of artwork by Amadeo de Souza-Cardoso using artificial intelligence. The mobile application is intended to be used to enhance museum visitors’ experience and strengthen the link between scientific, governmental, and heritage organisations.

Motion capture is a fundamental technique in the development of video games and in film production to animate a virtual character based on the movements of an actor, creating more realistic animations in a short amount of time. One of the ways to obtain this movement from an actor is to capture the motion of the player through an optical sensor to interact with the virtual world. However, during movement some parts of the human body can be occluded by others and there can be noise caused by difficulties in sensor capture, reducing the user experience. This work presents a solution to correct the motion capture errors from the Microsoft Kinect sensor or similar through a deep neural network (DNN) trained with a pre-processed dataset of poses offered by Carnegie Mellon University (CMU) Graphics Lab. A temporal filter is implemented to smooth the movement, given by a set of poses returned by the deep neural network. This system is implemented in Python with the TensorFlow application programming interface (API), which supports the machine learning techniques and the Unity game engine to visualize and interact with the obtained skeletons. The results are evaluated using the mean absolute error (MAE) metric where ground truth is available and with the feedback of 12 participants through a questionnaire for the Kinect data.

Tourism on the island of Santa Maria, Azores, has been increasing due to its characteristics in terms of biodiversity and geodiversity. This island has several hiking trails; the available information can be consulted in pamphlets and physical placards, whose maintenance and updating is difficult and expensive. Thus, the need to improve the visitors’ experience arises, in this case, by using the technological means currently available to everyone: a smartphone. This paper describes the development and evaluation of the user experience of a mobile application for guiding visitors on said hiking trails, as well as the design principles and main issues observed during this process. The application is based on an augmented reality interaction model providing visitors with an interactive and recreational experience through Augmented Reality in outdoor environments (without additional marks in the physical space and using georeferenced information), helping in navigation during the route and providing updated information with easy maintenance. For the design and evaluation of the application, two studies were carried out with users on-site (Santa Maria, Azores). The first had 77 participants, to analyze users and define the application’s characteristics, and the second had 10 participants to evaluate the user experience. The feedback from participants was obtained through questionnaires. In these questionnaires, an average SUS (System Usability Scale) score of 83 (excellent) and positive results in the UEQ (User Experience Questionnaire) were obtained.