Alex Marchioni

Extensive monitoring systems generate data that is usually compressed for network transmission. This compressed data might then be processed in the cloud for tasks such as anomaly detection. However, compression can potentially impair the detector's ability to distinguish between regular and irregular patterns due to information loss. Here we extend the information-theoretic framework introduced in [1] to simultaneously address the trade-off between the three features on which the effectiveness of the system depends: the effectiveness of compression, the amount of distortion it introduces, and the distinguishability between compressed normal signals and compressed anomalous signals. We leverage a Gaussian assumption to draw curves showing how moving on a Pareto surface helps administer such a trade-off better than simply relying on optimal rate-distortion compression and hoping that compressed signals can be distinguished from each other.

As neural networks are trained to perform tasks of increasing complexity, their size increases, which presents several challenges in their deployment on devices with limited resources. To cope with this, a recently proposed approach hinges on substituting the classical Multiply-and-ACcumulate (MAC) neurons in the hidden layers with other neurons called Multiply-And-Max/min (MAM) whose selective behavior helps identify important interconnections, thus allowing aggressive pruning of the others. Hybrid MAM&MAC structures promise a 10x or even 100x reduction in their memory footprint compared to what can be obtained by pruning MAC-only structures. However, a cornerstone of maintaining this promise is the assumption that MAC&MAM architectures have the same expressive power as MAC-only ones. To concretize such a cornerstone, we take here a step in the theoretical characterization of the capabilities of mixed MAM&MAC networks. We prove, with two theorems, that two hidden MAM layers followed by a MAC neuron with possibly a normalization stage is a universal approximator.

Anomalies are rare events, and this affects the design flow of detectors that monitor systems that behave normally most of the time but whose failure may have serious consequences. This limitation is particularly evident in the detector performance evaluation: it requires an abundance of normal and anomalous data but realistically faces a scarcity of the latter. To address this, in this article, we develop a framework comprising a set of abstract anomalies modeling the effects real-world failures and disturbances have on sensor readings. In addition, we devise synthetic generation procedures for these anomalies. Given a dataset of normal tracks from the actual application, one may apply such procedures to produce anomalous-like time series for a comprehensive detector assessment. We show that this framework can anticipate the detector performing best with real-world anomalies in the context of human and structural health monitoring, also highlighting that, in these cases, the best detector is not the most complex.

Large monitoring systems produce data that is often compressed to be transmitted over the network. For latency or security reasons, compressed data may be processed at the edge, i.e., along the path from sensors to the cloud, for some purposes such as anomaly detection. However, the performance of a detector distinguishing between normal and anomalous behavior may be affected by the loss of information due to compression. We here analyze how lossy compression affects the performance of a generic anomaly detector. This relationship is formalized in terms of information-theoretic quantities. Within such a framework we leverage a Gaussian assumption to derive analytical results regarding the importance of white noise as a representative of both the average and asymptotic anomalies. Moreover, in an anomaly-agnostic scenario, we also show the existence of a level of compression for which an anomaly is undetectable though compression is not completely destructive. Numerical evidence confirms that the proposed information-theoretic quantities anticipate the performance of practical compressors and detectors in the case of Gaussian and non-Gaussian signals allowing an assessment of the tradeoff between compression and detection.

Subspace analysis is a widely used technique for coping with high-dimensional data and is becoming a fundamental step in the early treatment of many signal processing tasks. However, traditional subspace analysis often requires a large amount of memory and computational resources, as it is equivalent to eigenspace determination. To address this issue, specialized streaming algorithms have been developed, allowing subspace analysis to be run on low-power devices such as sensors or edge devices. Here, we present a classification and a comparison of these methods by providing a consistent description and highlighting their features and similarities. We also evaluate their performance in the task of subspace identification with a focus on computational complexity and memory footprint for different signal dimensions. Additionally, we test the implementation of these algorithms on common hardware platforms typically employed for sensors and edge devices.

Analog In-Memory computing (AIMC) is a novel paradigm looking for solutions to prevent the unnecessary transfer of data by distributing computation within memory elements. One such operation is matrix-vector multiplication (MVM), a workhorse of many fields ranging from linear regression to Deep Learning. The same concept can be readily applied to the encoding stage in Compressed Sensing (CS) systems, where an MVM operation maps input signals into compressed measurements. With a focus on an encoder built on top of a Phase-Change Memory (PCM) AIMC platform, the effects of device non-idealities, namely programming spread and drift over time, are observed in terms of the reconstruction quality obtained for synthetic signals, sparse in the Discrete Cosine Transform (DCT) domain. PCM devices are simulated using statistical models summarizing the properties experimentally observed in an AIMC prototype, designed in a 90 nm STMicroelectronics technology. Different families of decoders are tested, and tradeoffs in terms of encoding energy are analyzed. Furthermore, the benefits of a hardware drift compensation strategy are also observed, highlighting its necessity to prevent the need for a complete reprogramming of the entire analog array. The results show >30 dB average reconstruction quality for mid-range conductances and a suitably selected decoder right after programming. Additionally, the hardware drift compensation strategy enables robust performance even when different drift conditions are tested.

Compressed Sensing was recently proposed to reduce the long acquisition time of Magnetic Resonance Imaging by undersampling the signal frequency content and then algorithmically reconstructing the original image. We propose a way to significantly improve the above method by exploiting a deep neural network to tackle both problems of frequency sub-sampling and image reconstruction simultaneously, thanks to the introduction of a new loss function to drive the training and the addition of a post-processing non-neural stage. Furthermore, we highlight how some of the quantities along the processing chain can be used as a proxy of the quality of the recovered image, thus allowing a self-assessment of the whole technique. All improvements hinge on the possibility of identifying constraints to which the final image must obey and suitably enforce them. The effectiveness of our approach is tested on real-world MRI acquisitions from the fastMRI public database and achieves an appreciable improvement in Peak Signal-to-Noise Ratio with respect to the original CS-based proposal with speed-up factors 4 and 8.

We analyze the effect of lossy compression in the processing of sensor signals that must be used to detect anomalous events in the system under observation. The intuitive relationship between the quality loss at higher compression and the possibility of telling anomalous behaviours from normal ones is formalized in terms of information-theoretic quantities. Some analytic derivations are made within the Gaussian framework and possibly in the asymptotic regime for what concerns the stretch of signals considered. Analytical conclusions are matched with the performance of practical detectors in a toy case allowing the assessment of different compression/detector configurations.