Thomas Längle

Conventional sensor-based sorting systems often rely on handcrafted image-processing features, which are highly task-specific, require expert knowledge, and are difficult to transfer to new sorting tasks. In contrast, deep learning enables automated feature learning and has previously been shown to outperform color-based sorting in several application scenarios. In this study, we investigate the suitability of two deep-learning-based semantic segmentation architectures, U-Net and SegFormer, for real-time sensor-based sorting. Their performance is evaluated in both offline experiments and practical sorting trials for the applications bulky waste sorting and hibiscus tea sorting. Both models proved suitable for the investigated tasks. U-Net showed slight advantages in inference speed and purity, reaching purities of 95.9 wt.% for bulky waste and 100.0 wt.% for hibiscus tea. SegFormer achieved almost equal results for sorting hibiscus tea and four percentage points lower purity for sorting bulky waste. Real-time applicability was demonstrated for SegFormer variants B0 to B3, which met the required throughput of 62 megapixels per second. Overall, the results show that transformer-based models are a viable and promising alternative to CNNs for real-time sensor-based sorting.

Pneumatic ejection has dominated sensor-based sorting for decades, yet diffuse air jets cause bycatch at high mass throughput, and compressed-air generation accounts for a large share of operating cost. We propose (i) a grid-based sorting architecture replacing the conventional single nozzle bar with a two-dimensional grid, and (ii) impact-based mechanical actuators within the grid. Together, these eliminate compressed air entirely while increasing spatio-temporal resolution by leveraging redundant particle–actuator assignments. To coordinate online ejection—deciding which actuators to trigger and when—we formulate sorting as a finite-horizon soc problem and introduce mtcr. mtcr integrates olf soc, mixed discrete–continuous-time modeling, situation-aware action-space reduction, and a greedy rollout heuristic, supported by analytically tractable evaluation of expected sorting costs. In high-mass-flow simulations with equal accept/reject ratios, mtcr maintains both tnr and tpr above 90 , demonstrating effective actuator coordination and bycatch reduction. Laboratory experiments on a 56-solenoid prototype show that the grid significantly exceeds a state-of-the-art pneumatic chute sorter in rejection performance at matched bycatch level, with actuation energy being at least 79 times lower. Grid-based mechanical sorting thus offers a promising route toward precise, energy-efficient separation for cost-sensitive industries such as recycling.

We present an event-based vibro-mechanical sensing approach for classifying objects enclosed inside an opaque capsule under externally applied vibration. Hidden objects are probed with a sequence of controlled excitation modes, and the resulting capsule micro-motions are recorded by an event camera at high temporal resolution and low data rate. From the event stream, we extract frequency-domain features and classify the resulting samples with a linear support vector machine. On an 8-class classification task, the proposed method achieves a mean accuracy of 82.2% in five-fold cross-validation. Compared with a conventional high-speed frame-based setup at similar temporal resolution, the event-based representation reduces the data volume by approximately two orders of magnitude. These results demonstrate the potential of event-based vision for low-bandwidth vibro-mechanical inspection.

Multimodal semantic segmentation with heterogeneous sensors must reconcile complementary information across modalities that differ in spatial resolution and channel dimensionality. In particular, high-resolution RGB imaging provides detailed spatial structure but often fails to distinguish visually similar materials, whereas hyperspectral imaging (HSI) provides discriminative spectral signatures but at lower spatial resolution. We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. Although our evaluation targets RGB-HSI fusion,BCAF is modality-agnostic and applies to co-registered RGB with lower-resolution, high-channel auxiliary sensors. On the benchmark SpectralWaste dataset, BCAF delivers strong performance, achieving 75.4% at 55 images/s. We further evaluate a novel industrial dataset: K3I-Cycling (first RGB subset already released on Fordatis). On this dataset, BCAF reaches 62.3% mIoU for material segmentation (paper, metal, plastic, etc.) and 66.2% mIoU for plastic-type segmentation (PET, PP, HDPE, LDPE, PS, etc.). These results show that preserving native-grid spatial detail and spectral structure improves multimodal segmentation under real-time constraints. Code and model checkpoints are publicly available at https://github.com/jonasvilhofunk/BCAF_2026.

Growing waste streams and the transition to a circular economy require efficient automated waste sorting. In industrial settings, materials move on fast conveyor belts, where reliable identification and ejection demand pixel-accurate segmentation. RGB imaging delivers high-resolution spatial detail, which is essential for accurate segmentation, but it confuses materials that look similar in the visible spectrum. Hyperspectral imaging (HSI) provides spectral signatures that separate such materials, yet its lower spatial resolution limits detail. Effective waste sorting therefore needs methods that fuse both modalities to exploit their complementary strengths. We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. We also analyze trade-offs between RGB input resolution and the number of HSI spectral slices. Although our evaluation targets RGB-HSI fusion, BCAF is modality-agnostic and applies to co-registered RGB with lower-resolution, high-channel auxiliary sensors. On the benchmark SpectralWaste dataset, BCAF achieves state-of-the-art performance of 76.4\% mIoU at 31 images/s and 75.4\% mIoU at 55 images/s. We further evaluate a novel industrial dataset: K3I-Cycling (first RGB subset already released on Fordatis). On this dataset, BCAF reaches 62.3\% mIoU for material segmentation (paper, metal, plastic, etc.) and 66.2\% mIoU for plastic-type segmentation (PET, PP, HDPE, LDPE, PS, etc.).

The optical bulk material sorting is a key technology on our way toward a circular economy and efficient recycling. However, controlling the sorting accuracy has so far been severely limited, as the achievable accuracy of conventional sorters is strongly determined by the mass flow and the mixing ratio of the incoming particle stream. To enable closed-loop control, in the previous work, we introduced a modification to the sorter design, in which controlled fractions of the already sorted mass flows are returned to the inlet of the sorter. In this article, we now propose two open-loop and two closed-loop feedback (CLF) stochastic model predictive controllers (MPCs) for the control of sorting systems with recirculation operating under dynamically changing conditions. In addition, we propose to integrate a desired minimum accuracy as a chance constraint into our controllers’ stochastic formulation. Our evaluations using a coupled discrete element–computational fluid dynamics (DEM–CFD) simulation show that our controllers considerably improve on the system without recirculation and outperform the previously known controllers. Furthermore, we found that they are able to maintain a predefined minimum quality even in highly dynamic scenarios, making the approach highly valuable for tasks where achieving a certain quality at any point in time is crucial.

Sensor-based sorting describes a family of systems that enable the removal of individual objects from a material stream. The technology is widely used in various industries such as agriculture, food, mining, and recycling. Examples of sorting tasks include the removal of fungus-infested grains, the enrichment of copper content in copper mining or the sorting of plastic waste according to the type of plastic. Sorting decisions are made based on information acquired by one or more sensors. A particular strength of the technology is the flexibility in sorting decisions, which is achieved by using various sensors and programming the data analysis. However, a comprehensive understanding of the process is necessary for the development of new sorting systems that can address previously unresolved tasks. This survey is aimed at innovative researchers and practitioners who are unfamiliar with sensor-based sorting or have only encountered certain aspects of the overall process. The references provided serve as starting points for further exploration of specific topics.

Article Image Processing Forum – Forum Bildverarbeitung 2022 was published on June 7, 2023 in the journal tm - Technisches Messen (volume 0, issue 0).