Jan Skaloud

Hyperspectral cameras have recently been miniaturized for operation on lightweight airborne platforms such as UAV or small aircraft. Unlike frame cameras (RGB or Multispectral), many hyperspectral sensors use a linear array or ’push-broom’ scanning design. This design makes image rectification and the calibration of intrinsic and extrinsic camera parameters significantly more challenging. Standard methods employed to address such tasks rely on precise GPS/INS estimates of the airborne platform trajectory and a detailed terrain model. However, inaccuracies in the trajectory or surface model can introduce systematic errors and complicate geometric modeling, which ultimately degrade the quality of the image rectification. To overcome these challenges, we propose a method for tie point extraction and camera calibration for ’push-broom’, hyperspectral sensors using only raw spectral imagery and a raw, possibly low quality, GPS/INS trajectory. We demonstrate that our approach allows for the automatic calibration of airborne systems with push-broom, hyperspectral cameras, outperforms other state-of-the-art automatic rectification methods and reaches an accuracy on par with manual calibration methods.

Tree instance segmentation of airborne laser scanning (ALS) data is of utmost importance for forest monitoring, but remains challenging due to variations in the data caused by factors such as sensor resolution, vegetation state at acquisition time, terrain characteristics, etc. Moreover, obtaining a sufficient amount of precisely labeled data to train fully supervised instance segmentation methods is expensive. To address these challenges, we propose a weakly supervised approach where labels of an initial segmentation result obtained either by a non-finetuned model or a closed form algorithm are provided as a quality rating by a human operator. The labels produced during the quality assessment are then used to train a rating model, whose task is to classify a segmentation output into the same classes as specified by the human operator. Finally, the segmentation model is finetuned using feedback from the rating model. This in turn improves the original segmentation model by 34\% in terms of correctly identified tree instances while considerably reducing the number of non-tree instances predicted. Challenges still remain in data over sparsely forested regions characterized by small trees (less than two meters in height) or within complex surroundings containing shrubs, boulders, etc. which can be confused as trees where the performance of the proposed method is reduced.

The AVIRIS-4 is a state of the art imaging spectrometer developed by NASA-JPL for the consortium of Swiss universities ARES. Measurement campaigns using this instrument started in 2024. According to the agreement with NASA-JPL, all imagery produced by the AVIRIS-4 should be made available as open data. To support the efforts of the ARES consortium in improving the quality and interoperability of the produced research data, in line with the FAIR framework for open data, a new tool is proposed to label tie points on hyperspectral imagery and use them as quality control indicators of the georeferencing process. A case study on the first flight of the system acquired for the geometric calibration of AVIRIS-4 is presented. Based on the proposed quality indicators, errors in the measured trajectory data collected in flight were identified, indicating the need for a new calibration flight, further refinement of the proposed trajectory optimization framework and guiding the design of future missions.

The Airborne Visible/Infrared Imaging Spectrometer-4 (AVIRIS-4) represents the next generation in the series of airborne imaging spectrometers built by National Aeronautics and Space Administration (NASA) Jet Propulsion Laboratory (JPL). Commissioned by the Swiss Airborne Research Facility for the Earth System (ARES) research consortium, AVIRIS-4 is geared toward delivering cutting-edge imaging spectroscopy data for scientific and practical applications as a replacement for its predecessor Airborne Prism EXperiment (APEX). AVIRIS-4 is based on a Dyson-type imaging spectrometer design, also employed by NASA-operated AVIRIS-3 and Earth Surface Mineral Dust Source Investigation (EMIT), and integrates a scaled two-mirror telescope housed in a compact vacuum vessel. This enables airborne measurements in unpressurized aircraft at altitudes ranging from 500 to 7620 m, achieving image resolutions between 0.3 and 4.5 m with a field of view of 40.2° in 1241 spatial pixels. AVIRIS-4 surpasses previous state-of-the-art sensor heads in signal-to-noise ratio (SNR) performance and features a spectral range of 375–2504 nm and 7.4-nm spectral sampling. The operation, data capture, and mission control hardware as well as the calibration and data processing software are developed by UZH, EPFL, and ZHAW. This letter outlines the design and calibration strategies implemented in AVIRIS-4’s development and highlights its performance during its first year of operation in 2024.

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a non-trivial problem due to challenges associated with the different influences of errors affecting their mapping accuracy including those coming from navigation sensors. Stemming from a special form of graph-based optimization, the dynamic networks allow rigorous modeling of spatio-temporal constraints and thus provide the common framework for optimizing original observations from inertial systems with those coming from optical sensors. In this work, we propose a cross-domain observation model that leverages pixel-to-point correspondences as links between imagery and lidar returns. First, we describe how the existence of such correspondences can be introduced into optimizations. Second, we employ a reference dataset to emulate a set of precise pixel-to-point correspondences to assess its prospective impact on the common (rather than cascade) optimization. We report the improvement in the estimated trajectory attitude error with lower quality IMU and thus the point-cloud registration. Finally, we study whether such correspondences could be contained from freely available deep learning networks with the desired accuracy and quality. We conclude that although the introduction of such camera-to-lidar constraints has significant potential, none of the studied machine learning networks can fulfill the requirement on correspondence detection in terms of quality.

LiDAR-based semantic segmentation is critical for autonomous trains, requiring accurate predictions across varying distances. This paper introduces two targeted data augmentation methods designed to improve segmentation performance on the railway-specific OSDaR23 dataset. The person instance pasting method enhances segmentation of pedestrians at distant ranges by injecting realistic variations into the dataset. The track sparsification method redistributes point density in LiDAR scans, improving track segmentation at far distances with minimal impact on close-range accuracy. Both methods are evaluated using a state-of-the-art 3D semantic segmentation network, demonstrating significant improvements in distant-range performance while maintaining robustness in close-range predictions. We establish the first 3D semantic segmentation benchmark for OSDaR23, demonstrating the potential of data-centric approaches to address railway-specific challenges in autonomous train perception.

Wind estimation is crucial for studying the atmospheric boundary layer. Traditional methods such as weather balloons offer limited in situ capabilities; besides an Air Data System (ADS) combined with inertial measurements and satellite positioning is required to estimate the wind on fixed-wing drones. As pressure probes are an important constituent of an ADS, they are susceptible to malfunctioning or failure due to blockages, thus affecting the capability of wind sensing and possibly the safety of the drone. This paper presents a novel approach, using low-fidelity aerodynamic models of drones to estimate wind synthetically. In our work, the aerodynamic model parameters are derived from post-processed flight data, in contrast to existing approaches that use expensive wind tunnel calibration for identifying the same. In sum, our method integrates aerodynamic force and moment models into a Vehicle Dynamic Model (VDM)-based navigation filter to yield a synthetic wind estimate without relying on an airspeed sensor. We validate our approach using two geometrically distinct drones, each characterized by a unique aerodynamic model and different quality of inertial sensors, altogether tested across several flights. Experimental results demonstrate that the proposed cross-platform method provides a synthetic wind velocity estimate, thus offering a practical backup to traditional techniques.

Hyperspectral cameras have recently been miniaturized for operation on lightweight airborne platforms such as UAV or small aircraft. Unlike frame cameras (RGB or Multispectral), many hyperspectral sensors use a linear array or 'push-broom' scanning design. This design presents significant challenges for image rectification and the calibration of the intrinsic and extrinsic camera parameters. Typically, methods employed to address such tasks rely on a precise GPS/INS estimate of the airborne platform trajectory and a detailed terrain model. However, inaccuracies in the trajectory or surface model information can introduce systematic errors and complicate geometric modeling which ultimately degrade the quality of the rectification. To overcome these challenges, we propose a method for tie point extraction and camera calibration for 'push-broom' hyperspectral sensors using only the raw spectral imagery and raw, possibly low quality, GPS/INS trajectory. We demonstrate that our approach allows for the automatic calibration of airborne systems with hyperspectral cameras, outperforms other state-of-the-art automatic rectification methods and reaches an accuracy on par with manual calibration methods.

This paper presents a comparative analysis of two methodologies for estimating unknown parameters in a Vehicle Dynamic Model (VDM)-based sensor fusion framework for small drones. Focusing on a delta-wing drone, we conduct open-air wind tunnel experiments to determine a functional aerodynamic model. Subsequently, we compare two methodologies for unknown model parameters identification, one based on linear regression on wind tunnel experimental data, and the other employing partial-update-based estimators on recorded flight data. The performance of both parameter estimation approaches is then evaluated in a VDM-based framework through three independent test flights. Our results highlight the necessity of wind tunnel experiments for aerodynamic model formulation, while the data-driven method proves useful to identify the parameters at a low cost. Furthermore, we demonstrate that both (flight) data-driven and wind-tunnel experiment-based identified aerodynamics significantly enhance positioning accuracy, particularly in the absence of satellite signals, when integrated with low-cost consumer-grade MEMS inertial sensors.