João Sena Esteves
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
João Sena Esteves has a total of 48 co-authors. Here are the top co-authors by number of shared publications:
- Filomena Soares
University of Minho
24 shared publications - Vinícius Silva
University of Minho
15 shared publications - Ana Paula da Silva Pereira
Universidade do Minho
8 shared publications - Celina Pinto Leao
University of Minho
3 shared publications - Cristina Peixoto Santos
Center for MicroElectromechanical Systems (CMEMS), University of Minho, 4800-058 Guimarães, Portugal
3 shared publications - Gil Lopes
LIACC—Laboratory of Artificial Intelligence and Computer Science, Polytechnic Institute of Porto, 4200-072 Porto, Portugal
3 shared publications
Latest Publications
- 10 October 2023
- Published by Springer Nature
- p. 268-280
An Augmented System Based on Machine Learning for Boccia Assisted Gameplay
- 10 October 2023
- Published by Springer Nature
- p. 268-280
An Augmented System Based on Machine Learning for Boccia Assisted Gameplay
- 10 October 2023
- Published by Springer Nature
- p. 174-187
Web Application for the Learning of Emotions in Children with Autism Spectrum Disorder
- 10 October 2023
- Published by Springer Nature
- p. 174-187
Web Application for the Learning of Emotions in Children with Autism Spectrum Disorder
- Open Access
- 22 September 2021
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 5 (10) , 57
Fostering Emotion Recognition in Children with Autism Spectrum Disorder
Facial expressions are of utmost importance in social interactions, allowing communicative prompts for a speaking turn and feedback. Nevertheless, not all have the ability to express themselves socially and emotionally in verbal and non-verbal communication. In particular, individuals with Autism Spectrum Disorder (ASD) are characterized by impairments in social communication, repetitive patterns of behaviour, and restricted activities or interests. In the literature, the use of robotic tools is reported to promote social interaction with children with ASD. The main goal of this work is to develop a system capable of automatic detecting emotions through facial expressions and interfacing them with a robotic platform (Zeno R50 Robokind® robotic platform, named ZECA) in order to allow social interaction with children with ASD. ZECA was used as a mediator in social communication activities. The experimental setup and methodology for a real-time facial expression (happiness, sadness, anger, surprise, fear, and neutral) recognition system was based on the Intel® RealSense™ 3D sensor and on facial features extraction and multiclass Support Vector Machine classifier. The results obtained allowed to infer that the proposed system is adequate in support sessions with children with ASD, giving a strong indication that it may be used in fostering emotion recognition and imitation skills.
- 22 September 2021
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 5 (10) , 57
Open AccessFostering Emotion Recognition in Children with Autism Spectrum Disorder
Facial expressions are of utmost importance in social interactions, allowing communicative prompts for a speaking turn and feedback. Nevertheless, not all have the ability to express themselves socially and emotionally in verbal and non-verbal communication. In particular, individuals with Autism Spectrum Disorder (ASD) are characterized by impairments in social communication, repetitive patterns of behaviour, and restricted activities or interests. In the literature, the use of robotic tools is reported to promote social interaction with children with ASD. The main goal of this work is to develop a system capable of automatic detecting emotions through facial expressions and interfacing them with a robotic platform (Zeno R50 Robokind® robotic platform, named ZECA) in order to allow social interaction with children with ASD. ZECA was used as a mediator in social communication activities. The experimental setup and methodology for a real-time facial expression (happiness, sadness, anger, surprise, fear, and neutral) recognition system was based on the Intel® RealSense™ 3D sensor and on facial features extraction and multiclass Support Vector Machine classifier. The results obtained allowed to infer that the proposed system is adequate in support sessions with children with ASD, giving a strong indication that it may be used in fostering emotion recognition and imitation skills.
- Open Access
Skeleton Driven Action Recognition Using an Image-Based Spatial-Temporal Representation and Convolution Neural Network
Individuals with Autism Spectrum Disorder (ASD) typically present difficulties in engaging and interacting with their peers. Thus, researchers have been developing different technological solutions as support tools for children with ASD. Social robots, one example of these technological solutions, are often unaware of their game partners, preventing the automatic adaptation of their behavior to the user. Information that can be used to enrich this interaction and, consequently, adapt the system behavior is the recognition of different actions of the user by using RGB cameras or/and depth sensors. The present work proposes a method to automatically detect in real-time typical and stereotypical actions of children with ASD by using the Intel RealSense and the Nuitrack SDK to detect and extract the user joint coordinates. The pipeline starts by mapping the temporal and spatial joints dynamics onto a color image-based representation. Usually, the position of the joints in the final image is clustered into groups. In order to verify if the sequence of the joints in the final image representation can influence the model’s performance, two main experiments were conducted where in the first, the order of the grouped joints in the sequence was changed, and in the second, the joints were randomly ordered. In each experiment, statistical methods were used in the analysis. Based on the experiments conducted, it was found statistically significant differences concerning the joints sequence in the image, indicating that the order of the joints might impact the model’s performance. The final model, a Convolutional Neural Network (CNN), trained on the different actions (typical and stereotypical), was used to classify the different patterns of behavior, achieving a mean accuracy of 92.4% ± 0.0% on the test data. The entire pipeline ran on average at 31 FPS.
Open AccessSkeleton Driven Action Recognition Using an Image-Based Spatial-Temporal Representation and Convolution Neural Network
Individuals with Autism Spectrum Disorder (ASD) typically present difficulties in engaging and interacting with their peers. Thus, researchers have been developing different technological solutions as support tools for children with ASD. Social robots, one example of these technological solutions, are often unaware of their game partners, preventing the automatic adaptation of their behavior to the user. Information that can be used to enrich this interaction and, consequently, adapt the system behavior is the recognition of different actions of the user by using RGB cameras or/and depth sensors. The present work proposes a method to automatically detect in real-time typical and stereotypical actions of children with ASD by using the Intel RealSense and the Nuitrack SDK to detect and extract the user joint coordinates. The pipeline starts by mapping the temporal and spatial joints dynamics onto a color image-based representation. Usually, the position of the joints in the final image is clustered into groups. In order to verify if the sequence of the joints in the final image representation can influence the model’s performance, two main experiments were conducted where in the first, the order of the grouped joints in the sequence was changed, and in the second, the joints were randomly ordered. In each experiment, statistical methods were used in the analysis. Based on the experiments conducted, it was found statistically significant differences concerning the joints sequence in the image, indicating that the order of the joints might impact the model’s performance. The final model, a Convolutional Neural Network (CNN), trained on the different actions (typical and stereotypical), was used to classify the different patterns of behavior, achieving a mean accuracy of 92.4% ± 0.0% on the test data. The entire pipeline ran on average at 31 FPS.
- 16 June 2021
- Published by Springer Nature
- p. 449-459
Environmental Parameters Monitoring System with an Application Interface for Smartphone
- 16 June 2021
- Published by Springer Nature
- p. 449-459
Environmental Parameters Monitoring System with an Application Interface for Smartphone
- 1 April 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 824-828
Stability Control of a Quadcopter Fixed to a Base
The present paper proposes a system based on the usage of Proportional, Integral and Derivative (PID) Control for stabilizing a quadcopter attached to a metal base. A smartphone Graphic User Interface (GUI) was created so the user can adjust manually, or automatically, the three gains that are characteristic of this type of control (proportional, integral and derivative gains). This system is intended to be seen as an educational concept for exploring PID Control. The user can change each of the three PID gain values and observe what are the consequences reflected on the behavior of the system. To test the latter, an attempt of tuning the three gains in order to stabilize the quadcopter was performed. The system reacted as expected, taking into account the theoretical knowledge about each of the PID gains and their influence on the system response. Overall, the final outcome of this work is an educational tool, designed and implemented by students in a compulsory curricular unit, which allows the user to understand how a PID controller works in an intuitive and practical way.
- 1 April 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 824-828
Stability Control of a Quadcopter Fixed to a Base
The present paper proposes a system based on the usage of Proportional, Integral and Derivative (PID) Control for stabilizing a quadcopter attached to a metal base. A smartphone Graphic User Interface (GUI) was created so the user can adjust manually, or automatically, the three gains that are characteristic of this type of control (proportional, integral and derivative gains). This system is intended to be seen as an educational concept for exploring PID Control. The user can change each of the three PID gain values and observe what are the consequences reflected on the behavior of the system. To test the latter, an attempt of tuning the three gains in order to stabilize the quadcopter was performed. The system reacted as expected, taking into account the theoretical knowledge about each of the PID gains and their influence on the system response. Overall, the final outcome of this work is an educational tool, designed and implemented by students in a compulsory curricular unit, which allows the user to understand how a PID controller works in an intuitive and practical way.
- Open Access
- 1 January 2019
- Published by INSTICC
- p. 497-503
A Supervised Autonomous Approach for Robot Intervention with Children with Autism Spectrum Disorder
- 1 January 2019
- Published by INSTICC
- p. 497-503
Open AccessA Supervised Autonomous Approach for Robot Intervention with Children with Autism Spectrum Disorder