NBNguyen Bach

Nguyen Bach

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Nguyen Bach has a total of 8 co-authors. Here are the top co-authors by number of shared publications:

  • Fei Huang

    DAMO Academy Alibaba Group China

    2 shared publications
  • Kewei Tu

    School of Information Science and Technology Engineering Research Center of Intelligent Vision DAMO Academy Alibaba Group ShanghaiTech University Shanghai Imaging University of Chinese Academy of Sciences

    2 shared publications
  • Tao Wang

    School of Information Science and Technology Engineering Research Center of Intelligent Vision DAMO Academy Alibaba Group ShanghaiTech University Shanghai Imaging University of Chinese Academy of Sciences

    2 shared publications
  • Xinyu Wang

    School of Information Science and Technology Engineering Research Center of Intelligent Vision DAMO Academy Alibaba Group ShanghaiTech University Shanghai Imaging University of Chinese Academy of Sciences

    2 shared publications
  • Yong Jiang

    School of Information Science and Technology Engineering Research Center of Intelligent Vision DAMO Academy Alibaba Group ShanghaiTech University Shanghai Imaging University of Chinese Academy of Sciences

    2 shared publications
  • Zhongqiang Huang

    School of Information Science and Technology Engineering Research Center of Intelligent Vision DAMO Academy Alibaba Group ShanghaiTech University Shanghai Imaging University of Chinese Academy of Sciences

    2 shared publications

Latest Publications

  • Open Access

    Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor

    by Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

    Knowledge distillation is a critical technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). The objective function of knowledge distillation is typically the cross-entropy between the teacher and the student's output distributions. However, for structured prediction problems, the output space is exponential in size; therefore, the cross-entropy objective becomes intractable to compute and optimize directly. In this paper, we derive a factorized form of the knowledge distillation objective for structured prediction, which is tractable for many typical choices of the teacher and student models. In particular, we show the tractability and empirical effectiveness of structural knowledge distillation between sequence labeling and dependency parsing models under four different scenarios: 1) the teacher and student share the same factorization form of the output structure scoring function; 2) the student factorization produces more fine-grained substructures than the teacher factorization; 3) the teacher factorization produces more fine-grained substructures than the student factorization; 4) the factorization forms from the teacher and the student are incompatible.

  • Open Access

    More Embeddings, Better Sequence Labelers?

    by Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

    Recent work proposes a family of contextual embeddings that significantly improves the accuracy of sequence labelers over non-contextual embeddings. However, there is no definite conclusion on whether we can build better sequence labelers by combining different kinds of embeddings in various settings. In this paper, we conduct extensive experiments on 3 tasks over 18 datasets and 8 languages to study the accuracy of sequence labeling with various embedding concatenations and make three observations: (1) concatenating more embedding variants leads to better accuracy in rich-resource and cross-domain settings and some conditions of low-resource settings; (2) concatenating additional contextual sub-word embeddings with contextual character embeddings hurts the accuracy in extremely low-resource settings; (3) based on the conclusion of (1), concatenating additional similar contextual embeddings cannot lead to further improvements. We hope these conclusions can help people build stronger sequence labelers in various settings.