Ghazi Felhi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Ghazi Felhi has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
Djamé Seddah
Automatic Language Modelling and ANAlysis & Computational Humanities
1 shared publication- Djamé Seddah
INRIA Paris
1 shared publication Joseph Le Roux
1 shared publicationJoseph Roux
Laboratoire d'informatique de Paris-Nord
1 shared publication- Laurent Devoille
Laboratoire National de métrologie et d'Essais - Nanometrology, 29 avenue Hennequin, 78197 Trappes Cedex, France
1 shared publication - Loïc Coquelin
Laboratoire National de Métrologie et d’Essais, Paris, France
1 shared publication
Latest Publications
- Open Access
Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs
We propose a generative model for text generation, which exhibits disentangled latent representations of syntax and semantics. Contrary to previous work, this model does not need syntactic information such as constituency parses, or semantic information such as paraphrase pairs. Our model relies solely on the inductive bias found in attention-based architectures such as Transformers. In the attention of Transformers, keys handle information selection while values specify what information is conveyed. Our model, dubbed QKVAE, uses Attention in its decoder to read latent variables where one latent variable infers keys while another infers values. We run experiments on latent representations and experiments on syntax/semantics transfer which show that QKVAE displays clear signs of disentangled syntax and semantics. We also show that our model displays competitive syntax transfer capabilities when compared to supervised models and that comparable supervised models need a fairly large amount of data (more than 50K samples) to outperform it on both syntactic and semantic transfer. The code for our experiments is publicly available.
Open AccessExploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs
We propose a generative model for text generation, which exhibits disentangled latent representations of syntax and semantics. Contrary to previous work, this model does not need syntactic information such as constituency parses, or semantic information such as paraphrase pairs. Our model relies solely on the inductive bias found in attention-based architectures such as Transformers. In the attention of Transformers, keys handle information selection while values specify what information is conveyed. Our model, dubbed QKVAE, uses Attention in its decoder to read latent variables where one latent variable infers keys while another infers values. We run experiments on latent representations and experiments on syntax/semantics transfer which show that QKVAE displays clear signs of disentangled syntax and semantics. We also show that our model displays competitive syntax transfer capabilities when compared to supervised models and that comparable supervised models need a fairly large amount of data (more than 50K samples) to outperform it on both syntactic and semantic transfer. The code for our experiments is publicly available.
- Open Access
- 1 January 2022
- Published by Association for Computational Linguistics (ACL)
- p. 5763-5776
Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs
Ghazi Felhi, Joseph Roux, Djamé Seddah. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
- 1 January 2022
- Published by Association for Computational Linguistics (ACL)
- p. 5763-5776
Open AccessExploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs
Ghazi Felhi, Joseph Roux, Djamé Seddah. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.