Mingrui Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Mingrui Liu has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Abdullah Kayi
IBM Research, Yorktown Heights, NY, USA
1 shared publication - Brian Kingsbury
IBM Research, Yorktown Heights, USA
1 shared publication - Chunyan Miao
School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore
1 shared publication - David Kung
IBM T. J. Watson Research Center, Yorktown Heights, New York, United States
1 shared publication - George Saon
IBM Research AI
1 shared publication - Hanjie Xu
McMaster University
1 shared publication
Latest Publications
- Open Access
- 22 May 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on the Web
- Vol. 17 (3) , 1-27
Heterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification
The prevalent heterogeneous Graph Neural Network (GNN) models learn node and graph representations using pre-defined meta-paths or only automatically discovering meta-paths. However, the existing methods suffer from information loss due to neglecting undiscovered meta-structures with richer semantics than meta-paths in heterogeneous graphs. To take advantage of the current rich meta-structures in heterogeneous graphs, we propose a novel approach called HeGTM to automatically extract essential meta-structures (i.e., meta-paths and meta-graphs) from heterogeneous graphs. The discovered meta-structures can capture more prosperous relations between different types of nodes that can help the model to learn representations. Furthermore, we apply the proposed approach for text classification. Specifically, we first design a heterogeneous graph for the text corpus, and then apply HeGTM on the constructed text graph to learn better text representations that contain various semantic relations. In addition, our approach can also be used as a strong meta-structure extractor for other GNN models. In other words, the auto-discovered meta-structures can replace the pre-defined meta-paths. The experimental results on text classification demonstrate the effectiveness of our approach to automatically extracting informative meta-structures from heterogeneous graphs and its usefulness in acting as a meta-structure extractor for boosting other GNN models.
- 22 May 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on the Web
- Vol. 17 (3) , 1-27
Open AccessHeterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification
The prevalent heterogeneous Graph Neural Network (GNN) models learn node and graph representations using pre-defined meta-paths or only automatically discovering meta-paths. However, the existing methods suffer from information loss due to neglecting undiscovered meta-structures with richer semantics than meta-paths in heterogeneous graphs. To take advantage of the current rich meta-structures in heterogeneous graphs, we propose a novel approach called HeGTM to automatically extract essential meta-structures (i.e., meta-paths and meta-graphs) from heterogeneous graphs. The discovered meta-structures can capture more prosperous relations between different types of nodes that can help the model to learn representations. Furthermore, we apply the proposed approach for text classification. Specifically, we first design a heterogeneous graph for the text corpus, and then apply HeGTM on the constructed text graph to learn better text representations that contain various semantic relations. In addition, our approach can also be used as a strong meta-structure extractor for other GNN models. In other words, the auto-discovered meta-structures can replace the pre-defined meta-paths. The experimental results on text classification demonstrate the effectiveness of our approach to automatically extracting informative meta-structures from heterogeneous graphs and its usefulness in acting as a meta-structure extractor for boosting other GNN models.
- Open Access
On Inferring User Socioeconomic Status with Mobility Records
When users move in a physical space (e.g., an urban space), they would have some records called mobility records (e.g., trajectories) generated by devices such as mobile phones and GPS devices. Naturally, mobility records capture essential information of how users work, live and entertain in their daily lives, and therefore, they have been used in a wide range of tasks such as user profile inference, mobility prediction and traffic management. In this paper, we expand this line of research by investigating the problem of inferring user socioeconomic statuses (such as prices of users' living houses as a proxy of users' socioeconomic statuses) based on their mobility records, which can potentially be used in real-life applications such as the car loan business. For this task, we propose a socioeconomic-aware deep model called DeepSEI. The DeepSEI model incorporates two networks called deep network and recurrent network, which extract the features of the mobility records from three aspects, namely spatiality, temporality and activity, one at a coarse level and the other at a detailed level. We conduct extensive experiments on real mobility records data, POI data and house prices data. The results verify that the DeepSEI model achieves superior performance than existing studies. All datasets used in this paper will be made publicly available.
Open AccessOn Inferring User Socioeconomic Status with Mobility Records
When users move in a physical space (e.g., an urban space), they would have some records called mobility records (e.g., trajectories) generated by devices such as mobile phones and GPS devices. Naturally, mobility records capture essential information of how users work, live and entertain in their daily lives, and therefore, they have been used in a wide range of tasks such as user profile inference, mobility prediction and traffic management. In this paper, we expand this line of research by investigating the problem of inferring user socioeconomic statuses (such as prices of users' living houses as a proxy of users' socioeconomic statuses) based on their mobility records, which can potentially be used in real-life applications such as the car loan business. For this task, we propose a socioeconomic-aware deep model called DeepSEI. The DeepSEI model incorporates two networks called deep network and recurrent network, which extract the features of the mobility records from three aspects, namely spatiality, temporality and activity, one at a coarse level and the other at a detailed level. We conduct extensive experiments on real mobility records data, POI data and house prices data. The results verify that the DeepSEI model achieves superior performance than existing studies. All datasets used in this paper will be made publicly available.
- Open Access
- 26 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 3565-3576
Asynchronous Decentralized Distributed Training of Acoustic Models
Large-scale distributed training of deep acoustic models plays an important role in today's high-performance automatic speech recognition (ASR). In this paper we investigate a variety of asynchronous decentralized distributed training strategies based on data parallel stochastic gradient descent (SGD) to show their superior performance over the commonly-used synchronous distributed training via allreduce, especially when dealing with large batch sizes. Specifically, we study three variants of asynchronous decentralized parallel SGD (ADPSGD), namely, fixed and randomized communication patterns on a ring as well as a delay-by-one scheme. We introduce a mathematical model of ADPSGD, give its theoretical convergence rate, and compare the empirical convergence behavior and straggler resilience properties of the three variants. Experiments are carried out on an IBM supercomputer for training deep long short-term memory (LSTM) acoustic models on the 2000-hour Switchboard dataset. Recognition and speedup performance of the proposed strategies are evaluated under various training configurations. We show that ADPSGD with fixed and randomized communication patterns cope well with slow learners. When learners are equally fast, ADPSGD with the delay-by-one strategy has the fastest convergence with large batches. In particular, using the delay-by-one strategy, we can train the acoustic model in less than 2 hours using 128 V100 GPUs with competitive word error rates.
- 26 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 3565-3576
Open AccessAsynchronous Decentralized Distributed Training of Acoustic Models
Large-scale distributed training of deep acoustic models plays an important role in today's high-performance automatic speech recognition (ASR). In this paper we investigate a variety of asynchronous decentralized distributed training strategies based on data parallel stochastic gradient descent (SGD) to show their superior performance over the commonly-used synchronous distributed training via allreduce, especially when dealing with large batch sizes. Specifically, we study three variants of asynchronous decentralized parallel SGD (ADPSGD), namely, fixed and randomized communication patterns on a ring as well as a delay-by-one scheme. We introduce a mathematical model of ADPSGD, give its theoretical convergence rate, and compare the empirical convergence behavior and straggler resilience properties of the three variants. Experiments are carried out on an IBM supercomputer for training deep long short-term memory (LSTM) acoustic models on the 2000-hour Switchboard dataset. Recognition and speedup performance of the proposed strategies are evaluated under various training configurations. We show that ADPSGD with fixed and randomized communication patterns cope well with slow learners. When learners are equally fast, ADPSGD with the delay-by-one strategy has the fastest convergence with large batches. In particular, using the delay-by-one strategy, we can train the acoustic model in less than 2 hours using 128 V100 GPUs with competitive word error rates.
- Open Access
Improved Schemes for Episodic Memory-based Lifelong Learning
Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field called lifelong learning. Recently, episodic memory based approaches such as GEM \cite{lopez2017gradient} and A-GEM \cite{chaudhry2018efficient} have shown remarkable performance. In this paper, we provide the first unified view of episodic memory based approaches from an optimization's perspective. This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-I and MEGA-II. MEGA-I and MEGA-II modulate the balance between old tasks and the new task by integrating the current gradient with the gradient computed on the episodic memory. Notably, we show that GEM and A-GEM are degenerate cases of MEGA-I and MEGA-II which consistently put the same emphasis on the current task, regardless of how the loss changes over time. Our proposed schemes address this issue by using novel loss-balancing updating rules, which drastically improve the performance over GEM and A-GEM. Extensive experimental results show that the proposed schemes significantly advance the state-of-the-art on four commonly used lifelong learning benchmarks, reducing the error by up to 18\%.
Open AccessImproved Schemes for Episodic Memory-based Lifelong Learning
Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field called lifelong learning. Recently, episodic memory based approaches such as GEM \cite{lopez2017gradient} and A-GEM \cite{chaudhry2018efficient} have shown remarkable performance. In this paper, we provide the first unified view of episodic memory based approaches from an optimization's perspective. This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-I and MEGA-II. MEGA-I and MEGA-II modulate the balance between old tasks and the new task by integrating the current gradient with the gradient computed on the episodic memory. Notably, we show that GEM and A-GEM are degenerate cases of MEGA-I and MEGA-II which consistently put the same emphasis on the current task, regardless of how the loss changes over time. Our proposed schemes address this issue by using novel loss-balancing updating rules, which drastically improve the performance over GEM and A-GEM. Extensive experimental results show that the proposed schemes significantly advance the state-of-the-art on four commonly used lifelong learning benchmarks, reducing the error by up to 18\%.