Stephan Nebiker

Street-level visualisations play an important role in urban street redesign by helping stakeholders and non-specialist audiences understand proposed spatial transformations. However, conventional image-editing workflows are often time-consuming, while purely image-based generative AI approaches provide limited control over object position, scale, and spatial relationships. This study presents a novel, highly automated workflow combining georeferenced street-level smartphone imagery, procedural 3D modelling, semantic image segmentation, and controlled generative AI to produce realistic visualisations of planned street transformations. Two-dimensional CAD drawings of planned changes are procedurally converted into simplified 3D models. Semantic masks are rendered from georeferenced camera viewpoints in the 3D model and combined with masks extracted from the street-level smartphone images. The resulting semantic and inpainting masks condition a Stable Diffusion XL-based inpainting model through segmentation and Canny-edge ControlNets. Text prompts control the overall visual appearance, while an IP-Adapter enables selected elements, such as vegetation and tree species, to be refined using reference images. The workflow was applied to two street redesign scenarios. The results show that planned elements can be generated at positions and with dimensions closely corresponding to the planning data, while different visual and seasonal variants can be explored without changing the spatial configuration. The method therefore provides a scalable approach for generating street-level visualisations while preserving geometric control.

Tree growth determines how much CO2 is sequestered from the atmosphere and temporarily stored in woody biomass. At the same time tree growth is affected by increasing temperatures, more frequent drought periods, late frosts and other extreme events associated with climate change. While continuous measurements of radial (secondary) tree growth using dendrometers are well established, monitoring of shoot elongation (primary growth) has largely been neglected because suitable measurement techniques are lacking. As a result, the effects of climate change on primary tree growth remain insufficiently understood. This work aims at reconstructing native deciduous trees in 3D as a basis for measuring and monitoring shoot elongation over entire tree canopies. Here we explored the use of low-cost UAV photogrammetry and of a multi-camera CraneCam system under real-world conditions. Data were collected in two study areas over an entire growing season. We present sensor evaluations, photogrammetric data acquisition and processing strategies. A special focus is placed on the analysis of the resulting photogrammetric 3D point clouds in terms of accuracy, resolution and completeness. Results demonstrate 3D point accuracies of 5-6 mm for entire trees using consumer-grade UAVs weighing less than 250 g and a 3D reconstruction completeness between 92% and 98% depending on the UAV type. The paper introduces a novel 3Dßprinted ground-truth branch to evaluate the capability to reconstructing fine-detail structures such as thin tree shoots. Finally, we discuss operational challenges and initial experiments towards a skeletonization of entire trees based on photogrammetric point clouds.

The effective management of municipal road infrastructure requires up-to-date, standardized and reliable condition information to support sustainable maintenance. While visual road-condition assessment methods based on established standards are widely applied to municipal roads, they remain largely manual, time-consuming, costly and subjective. This study presents an end-to-end workflow for the automated visual inspection and condition assessment of municipal road infrastructure using high-resolution, 3D street-level imagery acquired by professional mobile mapping systems. The proposed approach integrates an efficient preprocessing pipeline for precise road-surface extraction with deep learning models trained for the specific task and an advanced postprocessing method for robust results aggregation. For this purpose, a large dataset covering approximately 352 km of municipal roads across eight municipalities was created by combining street-level imagery with expert-annotated road-condition index (RCI) values. Two neural network variants were implemented: a regression model predicting standardized RCI values and a binary classifier distinguishing between roads requiring maintenance and those in good condition. To ensure decision-oriented outputs at the infrastructure-asset level, frame-based predictions are aggregated into homogeneous road segments using outlier detection and change-point analysis along the road axis. The regression model achieved a mean absolute error of 0.48 RCI values at frame level and 0.40 RCI values at road-segment level, outperforming conventional inter-expert variability, while the binary classification model reached an F1-score of 0.85. These findings demonstrate that AI-based visual road-condition assessment using professional mobile mapping data can provide accurate, standardized and scalable condition information for municipal road infrastructure. The proposed workflow supports maintenance prioritization and infrastructure management decisions without requiring explicit detection of individual pavement defects, offering a practical pathway toward automated, cost-effective road-condition monitoring.

Atolls face risks from rising sea levels and reduced sediment supply from coral reefs, leading to accelerated erosion. This study examines Aldabra, a raised atoll and UNESCO World Heritage Site in the Indian Ocean, tracking shoreline changes against a regional sea level rise of 2–3 mm per year. Using aerial and satellite images from 1960 and 2011, 85% of Aldabra’s shoreline was analyzed. Over 51 years, 61% of the shoreline remained unchanged, while 24% changed at a low average rate of 0.25 ± 0.36 m per year. Erosion and accretion rates were nearly balanced, with significant localized changes, such as a lagoon shoreline transforming into a mangrove habitat. Critical erosion occurred at turtle nesting sites and the research station. The lagoon shoreline experienced more rapid changes than the ocean-facing side. Despite these dynamics, Aldabra's overall shoreline and land area remained stable, similar to other Indo-Pacific atolls, highlighting its adaptive capacity. This research underscores the need to minimize local impacts on sediment availability to preserve the natural dynamics and adaptation potential of reef islands. Continuous shoreline monitoring is essential for developing timely adaptation strategies to conserve Aldabra's unique ecosystem.

Buildings are major contributors to global energy consumption, with the thermal performance of their envelopes playing a crucial role. Detecting thermal bridges, which compromise insulation, is essential for energy efficiency. To efficiently detect thermal bridges, thermal infrared (TIR) imagery is widely used through visual inspections, more recently by exploiting sensors mounted on unmanned aerial vehicles (UAVs). While RGB images have been extensively used in Structure-from-Motion and Multi-View Stereo processes, applying these techniques to TIR images presents challenges due to lower resolution and inconsistent colour spaces. To overcome the challenges posed by TIR imagery, approaches from different fields investigated the integration of TIR images with other data to support the alignment. Our approach improves upon these methods by using a DJI Mavic 3 Enterprise Thermal UAV to collect RGB and TIR datasets simultaneously. Our guided image alignment and camera rig estimation approach accounts for unknown camera calibration, misalignment, and lever arm parameters, ensuring robust alignment of TIR images with a total error of 5 pixels. With this approach, the geometric accuracy of the resulting point cloud reached an RMSE of 0.13 m. Finally, thermal calibration values collected on site were applied to correct the thermal images, improving temperature value accuracy for 3D model texturing with a temperature deviation of 2.8 °C. The developed method requires no prior camera calibration, TIR image pre-processing, or ground control points, permitting a complete automation of the process.

Accurately georeferenced data acquired using mobile mapping systems is of great importance for many geospatial applications. The accuracy of direct georeferencing – the standard procedure in the field of outdoor mobile mapping – strongly relies on GNSS reception and therefore varies greatly depending on the environment. By incorporating control point observations, integrated georeferencing enables homogenous accuracy and reliability over the whole mapping perimeter. Unfortunately, exact measurement and documentation of control points is needed, which often must be done manually. To automate this process, approaches from the field of autonomous vehicles use pole-like objects to support localization in complex urban environments, with the disadvantage of requiring a prior mapping campaign. However, various classes of pole-like objects have been recorded with accurate location and entered in public cadastres, so they could serve as 2D control points. In this paper, we present an approach for improving the trajectory accuracy in challenging urban environments by means of integrated georeferencing. It uses range image observations to pole-like objects from publicly available cadastres. Our approach achieves sub-metre accuracy, with a maximal cross-track difference of 98 cm, using real-world data acquired with our low-cost mobile mapping system. We further demonstrate that it significantly improves discontinuities and inaccuracy peaks in direct georeferencing and that the limiting factors are errors in the depth estimation of available range imaging sensors.

Modelling environments for urban projects often involve virtual reconstructions of existing urban areas where generalised city models are not sufficient. The use of photogrammetry with aerial imagery from UAVs on the one hand, and imagery from street-level mobile mapping systems on the other, has advanced significantly in recent years. However, there are limitations when these two imagery types are used separately.The main contribution of this paper is an end-to-end solution for creating large-scale and complex reality-based mesh models of urban environments. We outline a novel image-based 3D reality capturing process by combining street-level imagery from a backpack multi-camera mobile mapping system and nadir and oblique UAV imagery. This reconstruction is based on dense multi-view image matching method. A use case is presented with a goal to reconstruct an area for an urban project which aims to introduce an elevated cycle highway to facilitate traffic. The investigated area has a size of 0.64 square kilometres and is in the city of Basel in Switzerland. The MMS and UAV images were all oriented in a single photogrammetric project. Several reconstructions with different configurations were investigated and the results of the successful reconstruction with a geometric accuracy of a few cm are presented.

In this paper we present an edge-based hardware and software framework for the 3D detection and mapping of parked vehicles on a mobile mapping platform for the use case of on-street parking statistics. First, we investigate different point cloud-based 3D object detection methods on our extremely dense and noisy depth maps obtained from low-cost RGB-D sensors to find a suitable object detector and determine the optimal preparation of our data. We then retrain the chosen object detector to detect all types of vehicles, rather than standard cars only. Finally, we design and develop a software framework integrating the newly trained object detector. By repeating the parking statistics of our previous work (Nebiker et al., 2021), our software is tested regarding the detection accuracy. With our edge-based framework, we achieve a precision and recall of 100% and 98% respectively on any parking configuration and vehicle type, outperforming all other known work on on-street parking statistics. Furthermore, our software is evaluated in terms of processing speed and volume of generated data. While the processing speed reaches only 1.9 frames per second due to limited computing resources, the amount of data generated is just 0.25 KB per frame.

A successful application of low-cost 3D cameras in combination with artificial intelligence (AI)-based 3D object detection algorithms to outdoor mobile mapping would offer great potential for numerous mapping, asset inventory, and change detection tasks in the context of smart cities. This paper presents a mobile mapping system mounted on an electric tricycle and a procedure for creating on-street parking statistics, which allow government agencies and policy makers to verify and adjust parking policies in different city districts. Our method combines georeferenced red-green-blue-depth (RGB-D) imagery from two low-cost 3D cameras with state-of-the-art 3D object detection algorithms for extracting and mapping parked vehicles. Our investigations demonstrate the suitability of the latest generation of low-cost 3D cameras for real-world outdoor applications with respect to supported ranges, depth measurement accuracy, and robustness under varying lighting conditions. In an evaluation of suitable algorithms for detecting vehicles in the noisy and often incomplete 3D point clouds from RGB-D cameras, the 3D object detection network PointRCNN, which extends region-based convolutional neural networks (R-CNNs) to 3D point clouds, clearly outperformed all other candidates. The results of a mapping mission with 313 parking spaces show that our method is capable of reliably detecting parked cars with a precision of 100% and a recall of 97%. It can be applied to unslotted and slotted parking and different parking types including parallel, perpendicular, and angle parking.

Estimating the pose of a mobile robotic platform is a challenging task, especially when the pose needs to be estimated in a global or local reference frame and when the estimation has to be performed while the platform is moving. While the position of a platform can be measured directly via modern tachymetry or with the help of a global positioning service GNSS, the absolute platform orientation is harder to derive. Most often, only the relative orientation is estimated with the help of a sensor mounted on the robotic platform such as an IMU, with one or multiple cameras, with a laser scanner or with a combination of any of those. Then, a sensor fusion of the relative orientation and the absolute position is performed. In this work, an additional approach is presented: first, an image-based relative pose estimation with frames from a panoramic camera using a state-of-the-art visual odometry implementation is performed. Secondly, the position of the platform in a reference system is estimated using motorized tachymetry. Lastly, the absolute orientation is calculated using a visual marker, which is placed in the space, where the robotic platform is moving. The marker can be detected in the camera frame and since the position of this marker is known in the reference system, the absolute pose can be estimated. To improve the absolute pose estimation, a sensor fusion is conducted. Results with a Lego model train as a mobile platform show, that the trajectory of the absolute pose calculated independently with four different markers have a deviation < 0.66 degrees 50% of the time and that the average difference is < 1.17 degrees. The implementation is based on the popular Robotic Operating System ROS.