Matthieu Puigt

Insects, particularly butterflies, play a crucial role in maintaining ecosystems. Therefore, better management is needed, which involves identifying the different species. This work aims to classify four important butterfly species using raw images from a spatio-spectral camera. This camera generates images in 192 spectral layers, posing a significant challenge as capturing a moving butterfly across all spectral layers is impossible, often resulting in limited spectral data for the butterfly specimen. To address this issue, we assume that the spectral features of butterfly species follow Gaussian distributions. However, due to significant overlap between some Gaussian distributions caused by spectral similarities between species, traditional classification methods for Gaussian observations, such as Gaussian Naive Bayes and the Z-score, prove ineffective. To overcome these limitations, we develop robust approaches based on convex combinations of both previous methods. These new approaches improve the results on our test images, providing better robustness even in the presence of noise. Notably, we observed significantly better results when butterflies are detected in the near-infrared, with an accuracy of 88.75% for a single detection image, reaching up to 97.5% for a sequence of four images.

In this paper we propose a novel framework that successfully combines random projection or compression to weighted Nonnegative Matrix Factorization (NMF). Indeed a large body of NMF research has focused on the unweighted case—i.e., a complete data matrix to factorize—with a few extensions to handle incomplete data. Also most of these works are typically not efficient enough when the size of the data is arbitrarily large. Random projections belong to the major techniques used to process big data and although have been successfully applied to NMF, there was no investigation with weighted NMF. For this reason we propose to combine random projection with weighted NMF, where the weight models the confidence in the data (or the absence of confidence in the case of missing data). We experimentally show the proposed framework to significantly speed-up state-of-the-art NMF methods under some mild conditions when applied on various data.

Miniaturized complementary metal-oxide semiconductor (CMOS) hyperspectral cameras utilizing Fabry–Perot interferometers (FPIs) have emerged as a low-cost solution providing fast-acquisition miniaturized sensors well suited for both in-field analysis and remote sensing. However, FPIs generate harmonics around each wavelength of interest, hindering the accuracy and reliability of spectral information. This letter proposes a novel scene-dependent spectral correction and calibration method for miniaturized CMOS hyperspectral cameras using the FPI technology. Unlike the manufacturer's scene-independent spectral correction matrix, our approach utilizes deconvolution with Tikhonov regularization weighted by the entropy of the Fabry–Perot harmonics to remove the generated artifacts and restore the original spectra. It adapts to the scene's unique characteristics, reducing harmonics and improving hyperspectral data quality. The experiments on synthetic data and real images acquired by an FPI sensor demonstrate the superiority of our method in removing harmonic distortions and achieving improved accuracy in spectral calibration.

This paper presents novel unmixing and demosaicing methods for snapshot spectral imaging (SSI) systems utilizing Fabry-Perot filters. Unlike conventional approaches that perform unmixing after image restoration or demosaicing, our proposed methods leverage Fabry-Perot filter deconvolution and extend the “pure pixel” framework to the SSI sensor patch level, enabling improved unmixing accuracy and introducing the concept of localized spectral purity. Through extensive experimentation on synthetically generated data and real images captured by SSI cameras, we demonstrate the superiority of our methods over state-of-the-art techniques. Furthermore, our results showcase the effectiveness of the proposed approach over our recently proposed joint unmixing and demosaicing method based on low-rank matrix completion.

With the recent advancements in design and processing speed, a new snapshot mosaic imaging sensor architecture (SSI) has been successfully developed, holding the potential to transform the way dynamic scenes are captured using miniaturized platforms. However, SSI systems encounter a core trade-off concerning spatial and spectral resolution due to the assignment of individual spectral bands to each pixel. While the SSI camera manufacturer provides a pipeline to process such data, we propose in this paper to process the RAW SSI data directly. We show this strategy to be much more accurate than post-processing after the pipeline. In particular, in the first part of this paper, we propose a low-rank matrix factorization and completion framework which jointly tackles both the demosaicing and the unmixing steps of the SSI data. In addition to a “natural” technique, we expand the well-known pure pixel assumption to the SSI sensor level and propose two dedicated methods to extract the endmembers. The first one can be seen as a weighted Sparse Component Analysis (SCA) method, while the second one relaxes the abundance sparsity assumption of the former. The abundances are then recovered by applying the naive approach with the fixed extracted endmembers. Finally, we experimentally validate the merits of the proposed methods using synthetically generated data and real images obtained with an SSI camera.

The miniaturization of hyperspectral cameras has opened a new path to capture spectral information. One such camera, called the hybrid linescan camera, requires accurate control of its movement. Contrary to classical linescan cameras, where one line is available for every band in one shot, the latter asks for multiple shots to fill a line with multiple bands. Unfortunately, the reconstruction is corrupted by a parallax effect, which affects each band differently. In this article, we propose a two-step procedure, which first reconstructs an approximate datacube in two different ways, and second, performs a corrective warping on each band based on a multiple homography framework. The second step combines different stitching methods to perform this reconstruction. A complete synthetic and experimental comparison is performed by using geometric indicators of reference points. It appears throughout the course of our experimentation that misalignment is significantly reduced but remains non-negligible at the potato leaf scale.

Random projections recently became popular tools to process big data. When applied to Nonnegative Matrix Factorization (NMF), it was shown that, in practice, with the same compression level, structured random projections were more efficient than classical strategies based on, e.g., Gaussian compression. However, as they are data-dependent, they remain costly and might not fully benefit from recent very fast random projection techniques. In this paper, we thus investigate an alternative framework to structured random projections—named random projection streams (RPS)—which (i) are based on classical random compression strategies only—and are thus data-independent—and (ii) can benefit from the above fast techniques. We experimentally show that, under some mild conditions, RPS allow the same NMF performance as structured random projection along iterations. We also show that even a CPU implementation of Gaussian Compression Streams allows a faster convergence than structured random projections when applied to weighted NMF.