Abstract
White storks (Ciconia ciconia), along with other migratory birds, return to the same wintering sites with remarkable consistency. Understanding whether an individual bird will return to a previously used site, and what drives that decision, has major implications for understanding how biological agents navigate complex environments under uncertainty. Despite the growing availability of animal tracking data and satellite-based Earth observation imagery, previous studies have largely analyzed these two data streams in isolation. This study develops a multimodal deep learning framework that fuses movement trajectory data with remote sensing imagery to predict individual-level migratory site fidelity, a behavioral classification problem that deep learning data fusion methods have under-explored. Our approach uses a dual-encoder neural architecture in PyTorch that jointly learns from both data modalities. One encoder uses a Recurrent Neural Network (RNN) to process spatiotemporal GPS telemetry, capturing the timing, speed, directionality, and stopover behavior of individual migration trajectories. A second encoder uses a Convolutional Neural Network (CNN) to process multi-spectral Sentinel-2 satellite imagery of candidate wintering sites via Google Earth Engine, capturing habitat characteristics at those locations. The learned representations are concatenated and passed through fully connected layers to produce a binary site fidelity prediction. The central scientific contribution is an ablation study comparing three configurations: trajectory data alone, imagery alone, and the fully fused model. This empirically quantifies each modality’s contribution to prediction. This study delivers a geospatial AI pipeline integrating animal tracking archives with Earth Observation (EO) data, with methods transferable to other domains where predicting agent behavior across geographic space matters.