SLSimon Leglaive

Simon Leglaive

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Simon Leglaive has a total of 29 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Modèles de mélange pour la séparation multicanale de sources sonores en milieu réverbérant

    by Simon Leglaive

    Cette thèse porte sur la séparation sous-déterminée de sources sonores en milieu réverbérant. Nous adoptons une approche probabiliste où les signaux sources sont représentés comme des variables aléatoires latentes dans un domaine temps-fréquence. La structure spécifique des signaux musicaux dans ce domaine est exploitée par l’intermédiaire de modèles de factorisation en matrices non-négatives. Les méthodes de la littérature traitent généralement les filtres de mélange comme des paramètres déterministes estimés uniquement à partir des données observées. Ces filtres correspondent cependant à des réponses de salle, ils ont donc une structure bien particulière qu’il est possible d’exploiter afin de guider leur estimation. Dans une première partie, le processus de mélange convolutif temporel est approché dans le domaine de la transformée de Fourier à court-terme, sous une hypothèse de filtres de mélange à réponse impulsionnelle courte. Nous développons des modèles autorégressifs à moyenne ajustée ayant pour objectif de transcrire la dynamique temporelle des filtres sous forme de corrélations fréquentielles. Ces modèles sont ensuite utilisés dans une méthode de séparation de sources où les filtres sont estimés au sens du maximum a posteriori, par un algorithme espérance-maximisation. Dans une seconde partie, nous proposons une méthode d’inférence variationnelle des coefficients temps-fréquence des sources à partir des observations temporelles du mélange. Le processus de mélange convolutif est donc cette fois représenté de façon exacte. En plus de convenir à la séparation de mélanges fortement réverbérants, cette approche nous permet de développer des a priori simples sur les filtres de mélange afin de guider leur estimation. Nous proposons un modèle basé sur la distribution t de Student et exploitant la décroissance exponentielle de la réverbération dans le domaine temporel.

  • Open Access

    Objective and subjective evaluation of speech enhancement methods in the UDASE task of the 7th CHiME challenge

    by Simon Leglaive, Matthieu Fraticelli, Hend ElGhazaly, Léonie Borne, Mostafa Sadeghi, Scott Wisdom, Manuel Pariente, John R. Hershey, Daniel Pressnitzer, Jon P. Barker


  • Open Access

    A multimodal dynamical variational autoencoder for audiovisual speech representation learning

    by Samir Sadok, Simon Leglaive, Laurent Girin, Xavier Alameda-Pineda, Renaud Séguier


  • Open Access

    Motion-DVAE: Unsupervised learning for fast human motion denoising

    by Guénolé Fiche, Simon Leglaive, Xavier Alameda-Pineda, Renaud Séguier


  • Open Access

    SwimXYZ: A large-scale dataset of synthetic swimming motions and videos

    by Guénolé Fiche, Vincent Sevestre, Camila Gonzalez-Barral, Simon Leglaive, Renaud Séguier

    Technologies play an increasingly important role in sports and become a real competitive advantage for the athletes who benefit from it. Among them, the use of motion capture is developing in various sports to optimize sporting gestures. Unfortunately, traditional motion capture systems are expensive and constraining. Recently developed computer vision-based approaches also struggle in certain sports, like swimming, due to the aquatic environment. One of the reasons for the gap in performance is the lack of labeled datasets with swimming videos. In an attempt to address this issue, we introduce SwimXYZ, a synthetic dataset of swimming motions and videos. SwimXYZ contains 3.4 million frames annotated with ground truth 2D and 3D joints, as well as 240 sequences of swimming motions in the SMPL parameters format. In addition to making this dataset publicly available, we present use cases for SwimXYZ in swimming stroke clustering and 2D pose estimation.


  • Open Access

    Learning and controlling the source-filter representation of speech with a variational autoencoder

    by Samir Sadok, Simon Leglaive, Laurent Girin, Xavier Alameda-Pineda, Renaud Séguier

    Understanding and controlling latent representations in deep generative models is a challenging yet important problem for analyzing, transforming and generating various types of data. In speech processing, inspiring from the anatomical mechanisms of phonation, the source-filter model considers that speech signals are produced from a few independent and physically meaningful continuous latent factors, among which the fundamental frequency $f_0$ and the formants are of primary importance. In this work, we start from a variational autoencoder (VAE) trained in an unsupervised manner on a large dataset of unlabeled natural speech signals, and we show that the source-filter model of speech production naturally arises as orthogonal subspaces of the VAE latent space. Using only a few seconds of labeled speech signals generated with an artificial speech synthesizer, we propose a method to identify the latent subspaces encoding $f_0$ and the first three formant frequencies, we show that these subspaces are orthogonal, and based on this orthogonality, we develop a method to accurately and independently control the source-filter speech factors within the latent subspaces. Without requiring additional information such as text or human-labeled data, this results in a deep generative model of speech spectrograms that is conditioned on $f_0$ and the formant frequencies, and which is applied to the transformation speech signals. Finally, we also propose a robust $f_0$ estimation method that exploits the projection of a speech signal onto the learned latent subspace associated with $f_0$.

  • Learning and controlling the source-filter representation of speech with a variational autoencoder

    by Samir Sadok, Simon Leglaive, Laurent Girin, Xavier Alameda-Pineda, Renaud Séguier


  • Top Cited Papers

    Unsupervised Speech Enhancement Using Dynamical Variational Autoencoders

    by Xiaoyu Bie, Simon Leglaive, Xavier Alameda-Pineda, Laurent Girin

    Dynamical variational autoencoders (DVAEs) are a class of deep generative models with latent variables, dedicated to model time series of high-dimensional data. DVAEs can be considered as extensions of the variational autoencoder (VAE) that include temporal dependencies between successive observed and/or latent vectors. Previous work has shown the interest of using DVAEs over the VAE for speech spectrograms modeling. Independently, the VAE has been successfully applied to speech enhancement in noise, in an unsupervised noise-agnostic set-up that requires neither noise samples nor noisy speech samples at training time, but only requires clean speech signals. In this paper, we extend these works to DVAE-based single-channel unsupervised speech enhancement, hence exploiting both speech signals unsupervised representation learning and dynamics modeling. We propose an unsupervised speech enhancement algorithm that combines a DVAE speech prior pre-trained on clean speech signals with a noise model based on nonnegative matrix factorization, and we derive a variational expectation-maximization (VEM) algorithm to perform speech enhancement. The algorithm is presented with the most general DVAE formulation and is then applied with three specific DVAE models to illustrate the versatility of the framework. Experimental results show that the proposed DVAE-based approach outperforms its VAE-based counterpart, as well as several supervised and unsupervised noise-dependent baselines, especially when the noise type is unseen during training.


  • Top Cited Papers

    Dynamical Variational Autoencoders: A Comprehensive Review

    by Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, Xavier Alameda-Pineda

    Variational autoencoders (VAEs) are powerful deep generative models widely used to represent high-dimensional complex data through a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, the input data vectors are processed independently. Recently, a series of papers have presented different extensions of the VAE to process sequential data, which model not only the latent space but also the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks or state-space models. In this monograph, we perform a literature review of these models. We introduce and discuss a general class of models, called dynamical variational autoencoders (DVAEs), which encompasses a large subset of these temporal VAE extensions. Then, we present in detail seven recently proposed DVAE models, with an aim to homogenize the notations and presentation lines, as well as to relate these models with existing classical temporal models. We have reimplemented those seven DVAE models and present the results of an experimental benchmark conducted on the speech analysis-resynthesis task (the PyTorch code is made publicly available). The monograph concludes with a discussion on important issues concerning the DVAE class of models and future research guidelines.


  • Audio-Visual Speech Enhancement Using Conditional Variational Auto-Encoders

    by Mostafa Sadeghi, Simon Leglaive, Xavier Alameda-Pineda, Laurent Girin, Radu Horaud

    Variational auto-encoders (VAEs) are deep generative latent variable models that can be used for learning the distribution of complex data. VAEs have been successfully used to learn a probabilistic prior over speech signals, which is then used to perform speech enhancement. One advantage of this generative approach is that it does not require pairs of clean and noisy speech signals at training. In this article, we propose audio-visual variants of VAEs for single-channel and speaker-independent speech enhancement. We develop a conditional VAE (CVAE) where the audio speech generative process is conditioned on visual information of the lip region. At test time, the audio-visual speech generative model is combined with a noise model based on nonnegative matrix factorization, and speech enhancement relies on a Monte Carlo expectation-maximization algorithm. Experiments are conducted with the recently published NTCD-TIMIT dataset as well as the GRID corpus. The results confirm that the proposed audio-visual CVAE effectively fuses audio and visual information, and it improves the speech enhancement performance compared with the audio-only VAE model, especially when the speech signal is highly corrupted by noise. We also show that the proposed unsupervised audio-visual speech enhancement approach outperforms a state-of-the-art supervised deep learning method.