Francisco Soares Neves
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Francisco Soares Neves has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Andry Maykol Pinto
Faculdade de Engenharia, INESC TEC, Universidade do Porto, Porto, Portugal
5 shared publications - Rafael Marques Claro
Instituto de Engenharia de Sistemas e Computadores Tecnologia e Ciencia
5 shared publications A. Pedro Aguiar
Department of Electrical and Computer Engineering, Universidade do Porto, Porto, Portugal
1 shared publicationAndry M. Pinto
Universidade do Porto Faculdade de Engenharia
1 shared publication- António Mateus-Pinheiro
Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal
1 shared publication - Bruna Araújo1 shared publication
Latest Publications
- Open Access
- 3 October 2025
- Published by Wiley
A Multimodal Agentic AI for the Autonomous Precise Landing of UAVs
Autonomous landing for Unmanned Aerial Vehicles (UAVs) requires both precision and resilience against environmental uncertainties, capabilities that current approaches struggle to deliver. This paper presents a novel learning-based solution that combines an advanced multimodal transformer-based detector with a reinforcement learning formulation to achieve reliable autonomous landing behavior across varying scenario uncertainties. Beyond the integration of multimodality for robust target detection, this research incorporates a comprehensive analysis of the impact of state representation on decision-making performance. The proposed methodology is validated through extensive simulation studies and real-world field experiments conducted on physical UAV platforms under natural wind disturbances, demonstrating reliable transfer from simulated training environments to controlled outdoor conditions. Field experiments across varying initial conditions and wind stress confirm the system’s robustness, achieving landing precision of 0.10 ± 0.08 meters in outdoor trials, demonstrating centimeter-level accuracy that surpasses the meter-level precision of global positioning systems.
- 3 October 2025
- Published by Wiley
Open AccessA Multimodal Agentic AI for the Autonomous Precise Landing of UAVs
Autonomous landing for Unmanned Aerial Vehicles (UAVs) requires both precision and resilience against environmental uncertainties, capabilities that current approaches struggle to deliver. This paper presents a novel learning-based solution that combines an advanced multimodal transformer-based detector with a reinforcement learning formulation to achieve reliable autonomous landing behavior across varying scenario uncertainties. Beyond the integration of multimodality for robust target detection, this research incorporates a comprehensive analysis of the impact of state representation on decision-making performance. The proposed methodology is validated through extensive simulation studies and real-world field experiments conducted on physical UAV platforms under natural wind disturbances, demonstrating reliable transfer from simulated training environments to controlled outdoor conditions. Field experiments across varying initial conditions and wind stress confirm the system’s robustness, achieving landing precision of 0.10 ± 0.08 meters in outdoor trials, demonstrating centimeter-level accuracy that surpasses the meter-level precision of global positioning systems.
- 19 January 2025
- Published by Wiley in Journal of Field Robotics
- Vol. 42 (5) , 2151-2172
A Multimodal Perception System for Precise Landing of UAVs in Offshore Environments
The integration of precise landing capabilities into unmanned aerial vehicles (UAVs) is crucial for enabling autonomous operations, particularly in challenging environments such as the offshore scenarios. This work proposes a heterogeneous perception system that incorporates a multimodal fiducial marker, designed to improve the accuracy and robustness of autonomous landing of UAVs in both daytime and nighttime operations. This work presents ViTAL‐TAPE, a visual transformer‐based model, that enhance the detection reliability of the landing target and overcomes the changes in the illumination conditions and viewpoint positions, where traditional methods fail. VITAL‐TAPE is an end‐to‐end model that combines multimodal perceptual information, including photometric and radiometric data, to detect landing targets defined by a fiducial marker with 6 degrees‐of‐freedom. Extensive experiments have proved the ability of VITAL‐TAPE to detect fiducial markers with an error of 0.01 m. Moreover, experiments using the RAVEN UAV, designed to endure the challenging weather conditions of offshore scenarios, demonstrated that the autonomous landing technology proposed in this work achieved an accuracy up to 0.1 m. This research also presents the first successful autonomous operation of a UAV in a commercial offshore wind farm with floating foundations installed in the Atlantic Ocean. These experiments showcased the system's accuracy, resilience and robustness, resulting in a precise landing technology that extends mission capabilities of UAVs, enabling autonomous and Beyond Visual Line of Sight offshore operations.
- 19 January 2025
- Published by Wiley in Journal of Field Robotics
- Vol. 42 (5) , 2151-2172
A Multimodal Perception System for Precise Landing of UAVs in Offshore Environments
The integration of precise landing capabilities into unmanned aerial vehicles (UAVs) is crucial for enabling autonomous operations, particularly in challenging environments such as the offshore scenarios. This work proposes a heterogeneous perception system that incorporates a multimodal fiducial marker, designed to improve the accuracy and robustness of autonomous landing of UAVs in both daytime and nighttime operations. This work presents ViTAL‐TAPE, a visual transformer‐based model, that enhance the detection reliability of the landing target and overcomes the changes in the illumination conditions and viewpoint positions, where traditional methods fail. VITAL‐TAPE is an end‐to‐end model that combines multimodal perceptual information, including photometric and radiometric data, to detect landing targets defined by a fiducial marker with 6 degrees‐of‐freedom. Extensive experiments have proved the ability of VITAL‐TAPE to detect fiducial markers with an error of 0.01 m. Moreover, experiments using the RAVEN UAV, designed to endure the challenging weather conditions of offshore scenarios, demonstrated that the autonomous landing technology proposed in this work achieved an accuracy up to 0.1 m. This research also presents the first successful autonomous operation of a UAV in a commercial offshore wind farm with floating foundations installed in the Atlantic Ocean. These experiments showcased the system's accuracy, resilience and robustness, resulting in a precise landing technology that extends mission capabilities of UAVs, enabling autonomous and Beyond Visual Line of Sight offshore operations.
- Open Access
- 25 June 2024
- Published by Wiley
A Multimodal Perception System for Precise Landing of UAVs in Offshore Environments
The integration of precise landing capabilities into UAVs is crucial for enabling autonomous operations, particularly in challenging environments such as the offshore scenarios. This work proposes a heterogeneous perception system that incorporates a multimodal fiducial marker, designed to improve the accuracy and robustness of autonomous landing of UAVs in both daytime and nighttime operations. This work presents ViTAL-TAPE, a visual transformer-based model, that enhance the detection reliability of the landing target and overcomes the changes in the illumination conditions and viewpoint positions, where traditional methods fail. VITAL-TAPE is an end-to-end model that combines multimodal perceptual information, including photometric and radiometric data, to detect landing targets defined by a fiducial marker with 6 degrees-of-freedom. Extensive experiments have proved the ability of VITAL-TAPE to detect fiducial markers with an error of 0.01 m. Moreover, experiments using the RAVEN UAV, designed to endure the challenging weather conditions of offshore scenarios, demonstrated that the autonomous landing technology proposed in this work achieved an accuracy up to 0.1 m. This research also presents the first successful autonomous operation of a UAV in a commercial offshore wind farm with floating foundations installed in the Atlantic Ocean. These experiments showcased the system's accuracy, resilience and robustness, resulting in a precise landing technology that extends mission capabilities of UAVs, enabling autonomous and Beyond Visual Line of Sight offshore operations.
- 25 June 2024
- Published by Wiley
Open AccessA Multimodal Perception System for Precise Landing of UAVs in Offshore Environments
The integration of precise landing capabilities into UAVs is crucial for enabling autonomous operations, particularly in challenging environments such as the offshore scenarios. This work proposes a heterogeneous perception system that incorporates a multimodal fiducial marker, designed to improve the accuracy and robustness of autonomous landing of UAVs in both daytime and nighttime operations. This work presents ViTAL-TAPE, a visual transformer-based model, that enhance the detection reliability of the landing target and overcomes the changes in the illumination conditions and viewpoint positions, where traditional methods fail. VITAL-TAPE is an end-to-end model that combines multimodal perceptual information, including photometric and radiometric data, to detect landing targets defined by a fiducial marker with 6 degrees-of-freedom. Extensive experiments have proved the ability of VITAL-TAPE to detect fiducial markers with an error of 0.01 m. Moreover, experiments using the RAVEN UAV, designed to endure the challenging weather conditions of offshore scenarios, demonstrated that the autonomous landing technology proposed in this work achieved an accuracy up to 0.1 m. This research also presents the first successful autonomous operation of a UAV in a commercial offshore wind farm with floating foundations installed in the Atlantic Ocean. These experiments showcased the system's accuracy, resilience and robustness, resulting in a precise landing technology that extends mission capabilities of UAVs, enabling autonomous and Beyond Visual Line of Sight offshore operations.
- Open Access
- 5 November 2023
- Published by Elsevier in IBRO Neuroscience Reports
- Vol. 15, S529-S530
HILUS, A NEW CYTOGENIC HIPPOCAMPAL NICHE MODULATED BY STRESS EXPOSURE AND TREATMENT WITH ANTIDEPRESSANTS
- 5 November 2023
- Published by Elsevier in IBRO Neuroscience Reports
- Vol. 15, S529-S530
Open AccessHILUS, A NEW CYTOGENIC HIPPOCAMPAL NICHE MODULATED BY STRESS EXPOSURE AND TREATMENT WITH ANTIDEPRESSANTS
- Open Access
- 24 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. PP, 1
Decoding Reinforcement Learning for newcomers
The Reinforcement Learning (RL) paradigm is showing promising results as a generic purpose framework for solving decision-making problems (e.g., robotics, games, finance). The aim of this work is to reduce the learning barriers and inspire young students, researchers and educators to use RL as an obvious tool to solve robotics problems. This paper provides an intelligible step-by-step RL problem formulation and the availability of an easy-to-use interactive simulator for students at various levels (e.g., undergraduate, bachelor, master, doctorate), researchers and educators. The interactive tool facilitates the familiarization with the key concepts of RL, its problem formulation and implementation. In this work, RL is used for solving a robotics 2D navigational problem where the robot needs to avoid collisions with obstacles while aiming to reach a goal point. A navigational problem is simple and convenient for educational purposes, since the outcome is unambiguous (e.g., the goal is reached or not, a collision happened or not). Due to a lack of open-source graphical interactive simulators concerning the field of RL, this paper combines theoretical exposition with an accessible practical tool to facilitate the apprehension. The results demonstrated are produced by a Python script that is released as open-source to reduce the learning barriers in such innovative research topic in robotics.
- 24 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. PP, 1
Open AccessDecoding Reinforcement Learning for newcomers
The Reinforcement Learning (RL) paradigm is showing promising results as a generic purpose framework for solving decision-making problems (e.g., robotics, games, finance). The aim of this work is to reduce the learning barriers and inspire young students, researchers and educators to use RL as an obvious tool to solve robotics problems. This paper provides an intelligible step-by-step RL problem formulation and the availability of an easy-to-use interactive simulator for students at various levels (e.g., undergraduate, bachelor, master, doctorate), researchers and educators. The interactive tool facilitates the familiarization with the key concepts of RL, its problem formulation and implementation. In this work, RL is used for solving a robotics 2D navigational problem where the robot needs to avoid collisions with obstacles while aiming to reach a goal point. A navigational problem is simple and convenient for educational purposes, since the outcome is unambiguous (e.g., the goal is reached or not, a collision happened or not). Due to a lack of open-source graphical interactive simulators concerning the field of RL, this paper combines theoretical exposition with an accessible practical tool to facilitate the apprehension. The results demonstrated are produced by a Python script that is released as open-source to reduce the learning barriers in such innovative research topic in robotics.
- Open Access
- 29 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 32152-32166
Energy Efficient Path Planning for 3D Aerial Inspections
The use of Unmanned Aerial Vehicles (UAVs) in different inspection tasks is increasing. This technology reduces inspection costs and collects high quality data of distinct structures, including areas that are not easily accessible by human operators. However, the reduced energy available on the UAVs limits their flight endurance. To increase the autonomy of a single flight, it is important to optimize the path to be performed by the UAV, in terms of energy loss. Therefore, this work presents a novel formulation of the Travelling Salesman Problem (TSP) and a path planning algorithm that uses a UAV energy model to solve this optimization problem. The novel TSP formulation is defined as Asymmetric Travelling Salesman Problem with Precedence Loss (ATSP-PL), where the cost of moving the UAV depends on the previous position. The energy model relates each UAV movement with its energy consumption, while the path planning algorithm is focused on minimizing the energy loss of the UAV, ensuring that the structure is fully covered. The developed algorithm was tested in both simulated and real scenarios. The simulated experiments were performed with realistic models of wind turbines and a UAV, whereas the real experiments were performed with a real UAV and an illumination tower. The inspection paths generated presented improvements over 24% and 8%, when compared with other methods, for the simulated and real experiments, respectively, optimizing the energy consumption of the UAV.
- 29 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 32152-32166
Open AccessEnergy Efficient Path Planning for 3D Aerial Inspections
The use of Unmanned Aerial Vehicles (UAVs) in different inspection tasks is increasing. This technology reduces inspection costs and collects high quality data of distinct structures, including areas that are not easily accessible by human operators. However, the reduced energy available on the UAVs limits their flight endurance. To increase the autonomy of a single flight, it is important to optimize the path to be performed by the UAV, in terms of energy loss. Therefore, this work presents a novel formulation of the Travelling Salesman Problem (TSP) and a path planning algorithm that uses a UAV energy model to solve this optimization problem. The novel TSP formulation is defined as Asymmetric Travelling Salesman Problem with Precedence Loss (ATSP-PL), where the cost of moving the UAV depends on the previous position. The energy model relates each UAV movement with its energy consumption, while the path planning algorithm is focused on minimizing the energy loss of the UAV, ensuring that the structure is fully covered. The developed algorithm was tested in both simulated and real scenarios. The simulated experiments were performed with realistic models of wind turbines and a UAV, whereas the real experiments were performed with a real UAV and an illumination tower. The inspection paths generated presented improvements over 24% and 8%, when compared with other methods, for the simulated and real experiments, respectively, optimizing the energy consumption of the UAV.
- Open Access
End-to-End Detection of a Landing Platform for Offshore UAVs Based on a Multimodal Early Fusion Approach
A perception module is a vital component of a modern robotic system. Vision, radar, thermal, and LiDAR are the most common choices of sensors for environmental awareness. Relying on singular sources of information is prone to be affected by specific environmental conditions (e.g., visual cameras are affected by glary or dark environments). Thus, relying on different sensors is an essential step to introduce robustness against various environmental conditions. Hence, a perception system with sensor fusion capabilities produces the desired redundant and reliable awareness critical for real-world systems. This paper proposes a novel early fusion module that is reliable against individual cases of sensor failure when detecting an offshore maritime platform for UAV landing. The model explores the early fusion of a still unexplored combination of visual, infrared, and LiDAR modalities. The contribution is described by suggesting a simple methodology that intends to facilitate the training and inference of a lightweight state-of-the-art object detector. The early fusion based detector achieves solid detection recalls up to 99% for all cases of sensor failure and extreme weather conditions such as glary, dark, and foggy scenarios in fair real-time inference duration below 6 ms.
Open AccessEnd-to-End Detection of a Landing Platform for Offshore UAVs Based on a Multimodal Early Fusion Approach
A perception module is a vital component of a modern robotic system. Vision, radar, thermal, and LiDAR are the most common choices of sensors for environmental awareness. Relying on singular sources of information is prone to be affected by specific environmental conditions (e.g., visual cameras are affected by glary or dark environments). Thus, relying on different sensors is an essential step to introduce robustness against various environmental conditions. Hence, a perception system with sensor fusion capabilities produces the desired redundant and reliable awareness critical for real-world systems. This paper proposes a novel early fusion module that is reliable against individual cases of sensor failure when detecting an offshore maritime platform for UAV landing. The model explores the early fusion of a still unexplored combination of visual, infrared, and LiDAR modalities. The contribution is described by suggesting a simple methodology that intends to facilitate the training and inference of a lightweight state-of-the-art object detector. The early fusion based detector achieves solid detection recalls up to 99% for all cases of sensor failure and extreme weather conditions such as glary, dark, and foggy scenarios in fair real-time inference duration below 6 ms.