Otmar Loffeld

In this work, we propose several Time-of-Fligth (ToF) sensing schemes which tackle the challenge of covering very-wide areas and long ranges in nearly real-time, with relatively simple implementation and low associated computational load. We thoroughly describe two methodologies for the resolution of the inverse problem. Firstly, we extend a greedy algorithm, such as Orthogonal Matching Pursuit (OMP), by considering an initial refinement of the spatial domain in which the signal recovery is performed. Then, we propose various non-adaptive techniques for the construction of the sensing matrices, relying on the optimization of coherence and density. We further develop them by including an additional verification step which accounts for the non-instantaneous transitions from one element to another of the code and avoids any possible coincidence between rising and falling edges which may degrade the coherence. We also investigate the upper super-resolution limit, the over-sampling rate which yields unitary coherence, by considering the Instrument Response Function (IRF) of the ToF sensor. Secondly, we expand an adaptive sensing scheme in which the rows of the sensing matrix are generated accounting for the information from previous measurements, such as Adaptive Progressive Edge Algorithm (APEG), by considering several groups of signals during the adaptation of the sensing matrices. The signals are then individually recovered for each pixel via OMP over the identified joint signal support. We validate the proposed methodologies by numerical simulations over datasets from stereo and ToF cameras.

In recent years, convolutional neural networks (CNNs) have been successfully applied to inverse synthetic aperture radar (ISAR) sparse imaging because of their powerful ability in feature extraction. However, these CNNs only adopt single path feed-forward architectures and lack paths for directly transmitting original feature representations (OFRs) in shallow layers to reconstruction layers, which limits the complete reconstruction of target shape due to the underutilization of the OFRs that are efficient for recovering target details. Later, fully CNN (FCNN) introduces several skip connections (SKs) to establish the additional ways for directly passing the OFRs to the reconstruction layers. Nevertheless, the transmitted OFRs inevitably include the feature information of artifacts, which usually results the appearance of artifacts in final reconstructed target image. To address this issue, we introduce the gate units to FCNN, and refer to the improved FCNN as G-FCNN. Furthermore, the learnable gate units weight the OFRs transmitted by SKs and autonomously decide how many OFRs are transmitted further. To circumvent the shortage of the real data available for network training, we utilize the transfer learning strategy to guarantee a good performance of the G-FCNN. The imaging results of real data show that the G-FCNN-based imaging method is superior to the existing CNN-based imaging methods.

With the start of the Spanish PAZ system, another earth observation satellite has become available to the scientific community. Following the launch of the PAZ science phase, we performed a number of ground experiments using different modes of the satellite SAR instrument, measuring the transmitted radar signal as well as its ground reflections using the HITCHHIKER receiver. The data acquired in these experiments is analyzed and used in collaboration with the PAZ calibration team at Instituto Nacional De Técnica Aerospacial (INTA) to characterize the PAZ instrument, thus verifying the system calibration. Furthermore, we obtained bistatic radar images from the data of the HITCHHIKER ground receiver.

Time-of-Flight cameras are active sensors that are able to capture both the light intensity reflected by each observed point in the scene and the distance between these points and the camera. Enhancing intensity images with a depth modality enables capturing surfaces in 3D and boosts the applicability of these sensors. Nevertheless, high-level information still needs to be extracted from the data stream in order to accomplish high-level tasks, like recognition or classification. Ideally, the semantic gap between sensor output and high-level requirements should be as small as possible, in order to reduce both computational cost and failure probability. An additional depth modality helps in this regard, but there are further cues that can be seen by a ToF sensor that have remained underexploited so far. In this paper we take the first steps towards a trimodal ToF camera, which adds a valuable material modality to the classical intensity and depth modalities. To this end, ToF raw data is used to obtain Fourier samples of the material impulse response function (MIRF) to modulated illumination. The MIRF depends on the surface- and sub-surface-level scattering mechanisms of the material and, thus, can be used to identify materials of different nature. Consequently, distinctive feature vectors can be obtained from the Fourier measurements. Additionally, including the incidence angle in the feature vectors allows capturing the MIRF dependency on this parameter. Experimental validation confirms the feasibility of this approach. We also constructed a live demonstrator of our trimodal ToF camera.

Video synthetic aperture radar (SAR) has shown great potentials in detection and tracking of slow ground moving targets. The classical shadow-aided detection was applied in video SAR, and most recently, the deep learning approach has been developed for shadow-aided moving target detection. This article presents a joint moving target detection approach for video SAR using a dual faster region-based convolutional neural network (Faster R-CNN), which algorithmically combines the shadow detection in the SAR image and the Doppler energy detection in the range-Doppler (RD) spectrum domain, and this new approach can suppress false alarm sufficiently. Video SAR image and its corresponding low resolution RD spectrum are fed into the developed dual Faster R-CNN. A correct detection can be achieved if the shadow of a moving target and its Doppler energy are simultaneously detected by paired region proposals, which are obtained by sharing the region proposals of two independent region proposal networks (RPNs). Therefore, the performance of moving target detection can be significantly improved by using diverse features in different domains. This proposed approach has been verified by both the simulated and real video SAR data. Compared to other classical methods, our approach exhibits a great detection performance in terms of fewer false alarms and acceptable missing alarms.

Real-time spaceborne bistatic SAR imaging could significantly reduce the whole processing time and can enhance the spaceborne SAR mission availability. Onboard real-time SAR imaging relies on the Doppler parameters estimated from the real-time onboard orbit determination system (OODS) measurement, whose accuracy level is not comparable to the orbit ephemeris data in ground-based SAR processing. The investigation of the impact of error in real-time OODS measurements on bistatic SAR image quality is necessary, and it can help to clarify the key parameter limits of the real-time OODS. The monostatic analytical approximation model (MonoAAM) for spaceborne SAR reduces simulation complexity and processing time compared to the widely used numerical simulation method. However, due to the different configurations between spaceborne bistatic and monostatic SAR, simply applying the MonoAAM on spaceborne bistatic SAR cannot guarantee the desired result. A bistatic analytical approximation model (BiAAM) for Doppler rate estimation error from real-time OODS measurement in real-time spaceborne bistatic SAR imaging is proposed for characterizing the estimation error. Selecting quadratic phase error (QPE) as an evaluation variable, the proposed BiAAM model can provide QPE estimation results for each position of the satellite in its orbit and the maximum QPE estimation for the whole orbit, while revealing the different process of OODS measurement error transferring to QPE in spaceborne bistatic SAR. The correctness and reliability of BiAAM are evaluated by comparing the result with a Monte Carlo numerical simulation method. With the supporting result from BiAAM, the concept and early-stage development of a real-time onboard bistatic SAR imaging mission could be possibly benefited.

Spatial resolution is one of the fundamental bottlenecks in the area of time-resolved imaging. Since each pixel measures a scene dependent time-profile, there is a technological limit on the size of pixel arrays that can be simultaneously used to perform measurements. To overcome this barrier, in this paper, we propose a low-complexity, one-bit sensing scheme. On the data capture front, the time-resolved measurements are mapped to a sequence of +1 and -1. This leads to an extremely simple implementation and at the same time poses a new form of information loss. On the image recovery front, our one-bit time-resolved imaging scheme is complemented with a non-iterative recovery algorithm that can handle the case of single and multiple light paths. Extensive computer simulations and physical experiments benchmarked against conventional Time-of-Flight imaging data corroborate our theoretical framework. Thus, our low-complexity alternative to time-resolved imaging can indeed potentially lead to a new imaging methodology.

Video synthetic aperture radar (SAR) has been found to be very valuable for detection and tracking of slow moving targets and for observing changes over short time periods. Shadows produced by target motion in sequential radar images can be used to detect targets themselves. This article presents a framework for shadow-aided moving target detection using deep neural networks (DNNs) in video SAR. The faster region-based convolutional neural network (Faster-RCNN) is first used to detect shadows in a single frame, and then all the detections in each frame are filtered by the improved density-based clustering algorithm to mitigate false alarms. The missing alarm in detection can be suppressed by using the designed bi-directional long-short-term memory (Bi-LSTM) network. The proposed approaches have been tested by both the simulated and the real data, and a comparison to classical processing is given. This article reveals that the DNN techniques are very suitable for video SAR moving target detection.