Alexandre Lombard

Improving energy efficiency and reducing environmental impact are among the most significant global challenges today. In particular, building energy systems (BESs) are the main consumers of urban energy. Reducing energy consumption and optimizing the operation of BESs can help conserve energy resources. Traditional rule-based controllers (RBCs) are limited in their ability to optimize building energy consumption while maintaining occupant comfort. Data-driven model predictive control (MPC), which uses building thermal dynamic models and historical data for training, may struggle to handle uncertain cases. Reinforcement learning (RL) allows the control systems to make decisions in real time. However, when the system becomes scalable, training a single agent becomes complicated and more time-consuming. Therefore, multi-agent reinforcement learning (MARL) methods are being investigated and applied to building-related energy systems. This paper provides a comprehensive literature review of recent advancements in MARL control methods for building-related energy systems and evaluates their adaptability, complexity, and realization. This review paper identifies current challenges and future directions in this field.

L'augmentation continue du nombre de véhicules a sans cesse conduit à innover pour mieux réguler le trafic routier. Aujourd'hui, l'automatisation des véhicules et la communication inter-véhiculaire rend possible une nouvelle approche : l'intersection autonome et coopérative (IAC) dans laquelle les véhicules autonomes communiquent pour négocier leur droit de passage à l'intersection. Théoriquement plus efficace, plus sûre et plus économique que les approches traditionnelles, son implémentation et son déploiement constituent néanmoins un défi scientifique et technologique. Dans ce mémoire, nous proposons plusieurs contributions soutenant ce développement : une stratégie de contrôle longitudinal et latéral du véhicule conçues pour fonctionner avec des informations discrètes et présentant une latence (provenant de la communication sans-fil ou du positionnement satellite), une solution de prévention des risques d'inter-blocage spécifique à l'IAC, et enfin, l'implémentation et l'expérimentation d'un prototype réel d'IAC avec trois véhicules autonomes. Un travail exploratoire est ensuite fourni pour généraliser les principes de l'IAC, afin d'automatiser et optimiser les opérations de changement de voie et d'insertion.

This paper presents an experimental study of a path-tracking framework for autonomous vehicles in which the lateral control command is applied to a dynamic control point along the wheelbase. Instead of enforcing a fixed reference at either the front or rear axle, the proposed method continuously interpolates between both, enabling smooth adaptation across driving contexts, including low-speed maneuvers and reverse motion. The lateral steering command is obtained by barycentric blending of two complementary controllers: a front-axle Stanley formulation and a rear-axle curvature-based geometric controller, yielding continuous transitions in steering behavior and improved tracking stability. In addition, we introduce a curvature-aware longitudinal control strategy based on virtual track borders and ray-tracing, which converts upcoming geometric constraints into a virtual obstacle distance and regulates speed accordingly. The complete approach is implemented in a unified control stack and validated in simulation and on a real autonomous vehicle equipped with GPS-RTK, radar, odometry, and IMU. The results in closed-loop tracking and backward maneuvers show improved trajectory accuracy, smoother steering profiles, and increased adaptability compared to fixed control-point baselines.

The integration of autonomous vehicles (AVs) into traffic road requires careful consideration of their ability to adapt to other road users. Among road users, pedestrians are particularly vulnerable and are likely to interact with AVs. While most studies have focused on pedestrian behavior in interactions with human-driven vehicles, the effects of AV presence and Vehicle-to-Pedestrian (V2P) communication on pedestrian behavior remain largely unexplored. This article explores the impact of these factors, as well as the influence of the learning process induced by the introduction of AVs into traffic, on pedestrian behavior. To this end, we analyze data from our experimental campaign involving 100 participants in a virtual reality (VR) environment designed to address this issue. This study provides a comprehensive analysis of pedestrian behavior, constituting a significant advancement in the understanding of pedestrian-autonomous vehicle interactions and contributing to the integration of AV.

As vehicle systems become increasingly connected and intelligent, insurance providers are turning to machine learning techniques to personalize billing based on individual driving behavior. This shift raises important questions about how to balance predictive performance with user privacy. In this paper, we present PrivFedProfiling, a decentralized privacy-preserving learning framework designed for use-based insurance (UBI) systems. Our method leverages Federated Learning (FL) to collaboratively train behavior models across distributed driver devices without transferring raw data. To further strengthen privacy, we integrate Differential Privacy (DP) and Homomorphic Encryption (HE) within the training process, protecting sensitive patterns in shared model updates. The proposed approach uses a Multilayer Perceptron (MLP) architecture and is validated using synthetic driving behavior data generated from the SUMO simulator. It offers a realistic yet controllable environment for testing. Results indicate that our method maintains high model accuracy while ensuring strong privacy guarantees, making it suitable for real-world deployment.

The field of Autonomous Vehicles (AV) has seen significant advancements in recent years, making the prospect of fully autonomous driving more realistic than ever, largely due to breakthroughs in Machine Learning. A crucial aspect of autonomous driving is the ability to accurately interpret the surrounding environment, as misinterpretations can lead to dangerous situations for other road users. Among them, pedestrians are particularly vulnerable, requiring AVs to develop a precise understanding of their behavior to ensure safety. However, pedestrian behavior is unpredictable and can place them at risk, making AV-pedestrian interactions particularly challenging. To reduce uncertainty in predicting pedestrian actions, AVs can enhance safety by communicating their intended path. Yet, the impact of such communication on pedestrian behavior remains underexplored in current research. Understanding this influence is key to improving pedestrian behavior models, ultimately leading to more effective simulations for training AV control systems. This paper presents a novel experimental study design for examining the effects of different AV-to-pedestrian communication methods on pedestrian behavior.

Robot social navigation needs to adapt to different human factors and environmental contexts. However, since these factors and contexts are difficult to predict and cannot be exhaustively enumerated, traditional learning-based methods have difficulty in ensuring the social attributes of robots in long-term and cross-environment deployments. This letter introduces an online context learning method that aims to empower robots to adapt to new social environments online. The proposed method adopts a two-layer structure. The bottom layer is built using a deep reinforcement learning-based method to ensure the output of basic robot navigation commands. The upper layer is implemented using an online robot learning-based method to socialize the control commands suggested by the bottom layer. Experiments using a community-wide simulator show that our method outperforms the state-of-the-art ones. Experimental results in the most challenging scenarios show that our method improves the performance of the state-of-the-art by 8%. The source code of the proposed method, the data used, and the tools for the per-training step are publicly available at https://github.com/Nedzhaken/SOCSARL-OL.

Robot social navigation needs to adapt to different human factors and environmental contexts. However, since these factors and contexts are difficult to predict and cannot be exhaustively enumerated, traditional learning-based methods have difficulty in ensuring the social attributes of robots in long-term and cross-environment deployments. This letter introduces an online context learning method that aims to empower robots to adapt to new social environments online. The proposed method adopts a two-layer structure. The bottom layer is built using a deep reinforcement learning-based method to ensure the output of basic robot navigation commands. The upper layer is implemented using an online robot learning-based method to socialize the control commands suggested by the bottom layer. Experiments using a community-wide simulator show that our method outperforms the state-of-the-art ones. Experimental results in the most challenging scenarios show that our method improves the performance of the state-of-the-art by 8%. The source code of the proposed method, the data used, and the tools for the per-training step are publicly available at https://github.com/Nedzhaken/SOCSARL-OL.

Integrating robots into real-world applications requires effective consideration of various agents, including other robots. Multi-agent reinforcement learning (MARL) is an established field that addresses multi-agent systems problems by leveraging reinforcement learning techniques. Despite its potential, the study of multi-robot systems, particularly in vision-based robotics, remains in its early stages. In this context, this article tackles the PointGoal navigation problem for multi-robot systems, extending the traditional single agent focus to a multi-agent context. To this end, we introduce a training environment designed to address vision-based multi-robot challenges. In addition, we propose a method based on the centralized training-decentralized execution paradigm within MARL to explore three PointGoal navigation scenarios: the SpecificGoal scenario, where each agent has a distinct target; the CommonGoal scenario, where all agents share the same target; and the Ad-hoCoop scenario, which requires agents to adapt to varying team sizes. Our results contribute to lay the groundwork for adopting MARL approaches to address vision-based tasks for multi-robot systems. The code for the approach and the training environment are available at: https://github.com/TakieddineSOUALHI/MARLPointNAV