Yujing Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yujing Wang has a total of 102 co-authors. Here are the top co-authors by number of shared publications:
- Mao Yang
Microsoft Research, Beijing, China
4 shared publications - Yunhai Tong
Peking University
3 shared publications - Ce Zhang
Institute for Computing Platforms, Federal Institute of Technology Zurich, Zürich, Switzerland
2 shared publications - Fan Yang
Microsoft Research, Beijing, China
2 shared publications - Hao Sun
Microsoft, Beijing, China
2 shared publications - Li Lyna Zhang
Microsoft Research, Beijing, China
2 shared publications
Latest Publications
- Open Access
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 6161-6168
CSRM-LLM: Embracing Multilingual LLMs for Cold-Start Relevance Matching in Emerging E-commerce Markets
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 6161-6168
Open AccessCSRM-LLM: Embracing Multilingual LLMs for Cold-Start Relevance Matching in Emerging E-commerce Markets
- Open Access
- 21 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4178-4182
You Can't Ignore Either: Unifying Structure and Feature Denoising for Robust Graph Learning
Recent research on the robustness of Graph Neural Networks (GNNs) under noises or attacks has attracted great attention due to its importance in real-world applications. Most previous methods explore a single noise source, recovering corrupt node embedding by reliable structures bias or developing structure learning with reliable node features. However, the noises and attacks may come from both structures and features in graphs, making the graph denoising a dilemma and challenging problem. In this paper, we develop a unified graph denoising (UGD) framework to unravel the deadlock between structure and feature denoising. Specifically, a high-order neighborhood proximity evaluation method is proposed to recognize noisy edges, considering features may be perturbed simultaneously. Moreover, we propose to refine noisy features with reconstruction based on a graph auto-encoder. An iterative updating algorithm is further designed to optimize the framework and acquire a clean graph, thus enabling robust graph learning for downstream tasks. Our UGD framework is self-supervised and can be easily implemented as a plug-and-play module. We carry out extensive experiments, which proves the effectiveness and advantages of our method. Code is avalaible at https://github.com/YoungTimmy/UGD.
- 21 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4178-4182
Open AccessYou Can't Ignore Either: Unifying Structure and Feature Denoising for Robust Graph Learning
Recent research on the robustness of Graph Neural Networks (GNNs) under noises or attacks has attracted great attention due to its importance in real-world applications. Most previous methods explore a single noise source, recovering corrupt node embedding by reliable structures bias or developing structure learning with reliable node features. However, the noises and attacks may come from both structures and features in graphs, making the graph denoising a dilemma and challenging problem. In this paper, we develop a unified graph denoising (UGD) framework to unravel the deadlock between structure and feature denoising. Specifically, a high-order neighborhood proximity evaluation method is proposed to recognize noisy edges, considering features may be perturbed simultaneously. Moreover, we propose to refine noisy features with reconstruction based on a graph auto-encoder. An iterative updating algorithm is further designed to optimize the framework and acquire a clean graph, thus enabling robust graph learning for downstream tasks. Our UGD framework is self-supervised and can be easily implemented as a plug-and-play module. We carry out extensive experiments, which proves the effectiveness and advantages of our method. Code is avalaible at https://github.com/YoungTimmy/UGD.
- 12 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (11) , 2576-2589
PruneAug: Bridging DNN Pruning and Inference Latency on Diverse Sparse Platforms Using Automatic Layerwise Block Pruning
Although pruning is an effective technique to reduce the number of weights in deep neural networks (DNNs), it remains challenging for the resulting sparse networks to perform low-latency inference on everyday hardware. This problem is mainly caused by the incompatibility between the unstructured sparsity adopted for accuracy preservation and the sparse platform's (the combination of sparse kernel library and the underlying hardware) expectation of regular sparse patterns. In order to resolve this conflict, we propose PruneAug, an augmentation over existing unstructured pruning methods that finds block-sparse networks with much lower latency but preserves the accuracy. The fundamental idea of PruneAug is to prune the network with a layerwise block dimension assignment in a platform-aware fashion. Subject to an accuracy-loss constraint, PruneAug minimizes the latency of the block sparse network by jointly optimizing this layerwise block dimension assignment and the network's sparsity level. Admittedly, this approach expands the solution space. To curb our search cost, we include multiple optimizations while designing PruneAug's search space and strategy. Our evaluation over diverse pruning methods, DNNs, datasets, and sparse platforms shows that PruneAug enables different pruning methods to achieve speedup (as much as$\boldsymbol{\sim}13\boldsymbol{\times}$ depending on the platform) while maintaining competitive accuracy relative to unstructured sparsity, extracting the full potential of sparse platforms.- 12 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (11) , 2576-2589
PruneAug: Bridging DNN Pruning and Inference Latency on Diverse Sparse Platforms Using Automatic Layerwise Block Pruning
Although pruning is an effective technique to reduce the number of weights in deep neural networks (DNNs), it remains challenging for the resulting sparse networks to perform low-latency inference on everyday hardware. This problem is mainly caused by the incompatibility between the unstructured sparsity adopted for accuracy preservation and the sparse platform's (the combination of sparse kernel library and the underlying hardware) expectation of regular sparse patterns. In order to resolve this conflict, we propose PruneAug, an augmentation over existing unstructured pruning methods that finds block-sparse networks with much lower latency but preserves the accuracy. The fundamental idea of PruneAug is to prune the network with a layerwise block dimension assignment in a platform-aware fashion. Subject to an accuracy-loss constraint, PruneAug minimizes the latency of the block sparse network by jointly optimizing this layerwise block dimension assignment and the network's sparsity level. Admittedly, this approach expands the solution space. To curb our search cost, we include multiple optimizations while designing PruneAug's search space and strategy. Our evaluation over diverse pruning methods, DNNs, datasets, and sparse platforms shows that PruneAug enables different pruning methods to achieve speedup (as much as$\boldsymbol{\sim}13\boldsymbol{\times}$ depending on the platform) while maintaining competitive accuracy relative to unstructured sparsity, extracting the full potential of sparse platforms.- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 292-301
MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
Recent breakthroughs in large models have highlighted the critical significance of data scale, labels and modals. In this paper, we introduce MS MARCO Web Search, the first large-scale information-rich web dataset, featuring millions of real clicked query-document labels. This dataset closely mimics real-world web document and query distribution, provides rich information for various kinds of downstream tasks and encourages research in various areas, such as generic end-to-end neural indexer models, generic embedding models, and next generation information access system with large language models. MS MARCO Web Search offers a retrieval benchmark with three web retrieval challenge tasks that demands innovations in both machine learning and information retrieval system research domains. As the first dataset that meets large, real and rich data requirements, MS MARCO Web Search paves the way for future advancements in AI and system research. MS MARCO Web Search dataset is available at: https://github.com/microsoft/MS-MARCO-Web-Search.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 292-301
Open AccessMS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
Recent breakthroughs in large models have highlighted the critical significance of data scale, labels and modals. In this paper, we introduce MS MARCO Web Search, the first large-scale information-rich web dataset, featuring millions of real clicked query-document labels. This dataset closely mimics real-world web document and query distribution, provides rich information for various kinds of downstream tasks and encourages research in various areas, such as generic end-to-end neural indexer models, generic embedding models, and next generation information access system with large language models. MS MARCO Web Search offers a retrieval benchmark with three web retrieval challenge tasks that demands innovations in both machine learning and information retrieval system research domains. As the first dataset that meets large, real and rich data requirements, MS MARCO Web Search paves the way for future advancements in AI and system research. MS MARCO Web Search dataset is available at: https://github.com/microsoft/MS-MARCO-Web-Search.
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 393-402
OneSparse: A Unified System for Multi-index Vector Search
Multi-index vector search has become the cornerstone for many applications, such as recommendation systems. Efficient search in such a multi-modal hybrid vector space is challenging since no single index design performs well for all kinds of vector data. Existing approaches to processing multi-index hybrid queries either suffer from algorithmic limitations or processing inefficiency. In this paper, we propose OneSparse, a unified multi-vector index query system that incorporates multiple posting-based vector indices, which enables highly efficient retrieval of multi-modal data-sets. OneSparse introduces a novel multi-index query engine design of inter-index intersection push-down. It also optimizes the vector posting format to expedite multi-index queries. Our experiments show OneSparse achieves more than 6x search performance improvement while maintaining comparable accuracy. OneSparse has already been integrated into Microsoft online web search and advertising systems with 5x+ latency gain for Bing web search and 2.0% Revenue Per Mille (RPM) gain for Bing sponsored search.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 393-402
Open AccessOneSparse: A Unified System for Multi-index Vector Search
Multi-index vector search has become the cornerstone for many applications, such as recommendation systems. Efficient search in such a multi-modal hybrid vector space is challenging since no single index design performs well for all kinds of vector data. Existing approaches to processing multi-index hybrid queries either suffer from algorithmic limitations or processing inefficiency. In this paper, we propose OneSparse, a unified multi-vector index query system that incorporates multiple posting-based vector indices, which enables highly efficient retrieval of multi-modal data-sets. OneSparse introduces a novel multi-index query engine design of inter-index intersection push-down. It also optimizes the vector posting format to expedite multi-index queries. Our experiments show OneSparse achieves more than 6x search performance improvement while maintaining comparable accuracy. OneSparse has already been integrated into Microsoft online web search and advertising systems with 5x+ latency gain for Bing web search and 2.0% Revenue Per Mille (RPM) gain for Bing sponsored search.
- Open Access
- 21 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 4380-4384
Mitigating Semantic Confusion from Hostile Neighborhood for Graph Active Learning
Graph Active Learning (GAL), which aims to find the most informative nodes in graphs for annotation to maximize the Graph Neural Networks (GNNs) performance, has attracted many research efforts but remains non-trivial challenges. One major challenge is that existing GAL strategies may introduce semantic confusion to the selected training set, particularly when graphs are noisy. Specifically, most existing methods assume all aggregating features to be helpful, ignoring the semantically negative effect between inter-class edges under the message-passing mechanism. In this work, we present Semantic-aware Active learning framework for Graphs (SAG) to mitigate the semantic confusion problem. Pairwise similarities and dissimilarities of nodes with semantic features are introduced to jointly evaluate the node influence. A new prototype-based criterion and query policy are also designed to maintain diversity and class balance of the selected nodes, respectively. Extensive experiments on the public benchmark graphs and a real-world financial dataset demonstrate that SAG significantly improves node classification performances and consistently outperforms previous methods. Moreover, comprehensive analysis and ablation study also verify the effectiveness of the proposed framework.
- 21 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 4380-4384
Open AccessMitigating Semantic Confusion from Hostile Neighborhood for Graph Active Learning
Graph Active Learning (GAL), which aims to find the most informative nodes in graphs for annotation to maximize the Graph Neural Networks (GNNs) performance, has attracted many research efforts but remains non-trivial challenges. One major challenge is that existing GAL strategies may introduce semantic confusion to the selected training set, particularly when graphs are noisy. Specifically, most existing methods assume all aggregating features to be helpful, ignoring the semantically negative effect between inter-class edges under the message-passing mechanism. In this work, we present Semantic-aware Active learning framework for Graphs (SAG) to mitigate the semantic confusion problem. Pairwise similarities and dissimilarities of nodes with semantic features are introduced to jointly evaluate the node influence. A new prototype-based criterion and query policy are also designed to maintain diversity and class balance of the selected nodes, respectively. Extensive experiments on the public benchmark graphs and a real-world financial dataset demonstrate that SAG significantly improves node classification performances and consistently outperforms previous methods. Moreover, comprehensive analysis and ablation study also verify the effectiveness of the proposed framework.
- Open Access
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1280-1290
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Deploying pre-trained transformer models like BERT on downstream tasks in resource-constrained scenarios is challenging due to their high inference cost, which grows rapidly with input sequence length. In this work, we propose a constraint-aware and ranking-distilled token pruning method ToP, which selectively removes unnecessary tokens as input sequence passes through layers, allowing the model to improve online inference speed while preserving accuracy. ToP overcomes the limitation of inaccurate token importance ranking in the conventional self-attention mechanism through a ranking-distilled token distillation technique, which distills effective token rankings from the final layer of unpruned models to early layers of pruned models. Then, ToP introduces a coarse-to-fine pruning approach that automatically selects the optimal subset of transformer layers and optimizes token pruning decisions within these layers through improved L0 regularization. Extensive experiments on GLUE benchmark and SQuAD tasks demonstrate that ToP outperforms state-of-the-art token pruning and model compression methods with improved accuracy and speedups. ToP reduces the average FLOPs of BERT by 8.1X while achieving competitive accuracy on GLUE, and provides a real latency speedup of up to 7.4X on an Intel CPU. Code is available at https://github.com/microsoft/Moonlit/tree/main/ToP
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1280-1290
Open AccessConstraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Deploying pre-trained transformer models like BERT on downstream tasks in resource-constrained scenarios is challenging due to their high inference cost, which grows rapidly with input sequence length. In this work, we propose a constraint-aware and ranking-distilled token pruning method ToP, which selectively removes unnecessary tokens as input sequence passes through layers, allowing the model to improve online inference speed while preserving accuracy. ToP overcomes the limitation of inaccurate token importance ranking in the conventional self-attention mechanism through a ranking-distilled token distillation technique, which distills effective token rankings from the final layer of unpruned models to early layers of pruned models. Then, ToP introduces a coarse-to-fine pruning approach that automatically selects the optimal subset of transformer layers and optimizes token pruning decisions within these layers through improved L0 regularization. Extensive experiments on GLUE benchmark and SQuAD tasks demonstrate that ToP outperforms state-of-the-art token pruning and model compression methods with improved accuracy and speedups. ToP reduces the average FLOPs of BERT by 8.1X while achieving competitive accuracy on GLUE, and provides a real latency speedup of up to 7.4X on an Intel CPU. Code is available at https://github.com/microsoft/Moonlit/tree/main/ToP
- 13 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (7) , 8176-8192
Convolution-Enhanced Evolving Attention Networks
Attention-based neural networks, such as Transformers, have become ubiquitous in numerous applications, including computer vision, natural language processing, and time-series analysis. In all kinds of attention networks, the attention maps are crucial as they encode semantic dependencies between input tokens. However, most existing attention networks perform modeling or reasoning based on representations , wherein the attention maps of different layers are learned separately without explicit interactions. In this paper, we propose a novel and generic evolving attention mechanism, which directly models the evolution of inter-token relationships through a chain of residual convolutional modules. The major motivations are twofold. On the one hand, the attention maps in different layers share transferable knowledge, thus adding a residual connection can facilitate the information flow of inter-token relationships across layers. On the other hand, there is naturally an evolutionary trend among attention maps at different abstraction levels, so it is beneficial to exploit a dedicated convolution-based module to capture this process. Equipped with the proposed mechanism, the convolution-enhanced evolving attention networks achieve superior performance in various applications, including time-series representation, natural language understanding, machine translation, and image classification. Especially on time-series representation tasks, Evolving Attention-enhanced Dilated Convolutional (EA-DC-) Transformer outperforms state-of-the-art models significantly, achieving an average of 17% improvement compared to the best SOTA. To the best of our knowledge, this is the first work that explicitly models the layer-wise evolution of attention maps. Our implementation is available at https://github.com/pkuyym/EvolvingAttention .
- 13 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (7) , 8176-8192
Convolution-Enhanced Evolving Attention Networks
Attention-based neural networks, such as Transformers, have become ubiquitous in numerous applications, including computer vision, natural language processing, and time-series analysis. In all kinds of attention networks, the attention maps are crucial as they encode semantic dependencies between input tokens. However, most existing attention networks perform modeling or reasoning based on representations , wherein the attention maps of different layers are learned separately without explicit interactions. In this paper, we propose a novel and generic evolving attention mechanism, which directly models the evolution of inter-token relationships through a chain of residual convolutional modules. The major motivations are twofold. On the one hand, the attention maps in different layers share transferable knowledge, thus adding a residual connection can facilitate the information flow of inter-token relationships across layers. On the other hand, there is naturally an evolutionary trend among attention maps at different abstraction levels, so it is beneficial to exploit a dedicated convolution-based module to capture this process. Equipped with the proposed mechanism, the convolution-enhanced evolving attention networks achieve superior performance in various applications, including time-series representation, natural language understanding, machine translation, and image classification. Especially on time-series representation tasks, Evolving Attention-enhanced Dilated Convolutional (EA-DC-) Transformer outperforms state-of-the-art models significantly, achieving an average of 17% improvement compared to the best SOTA. To the best of our knowledge, this is the first work that explicitly models the layer-wise evolution of attention maps. Our implementation is available at https://github.com/pkuyym/EvolvingAttention .
- Open Access
- 8 March 2022
- Published by Frontiers Media SA in Frontiers in Environmental Science
- Vol. 10
Financial Annexation, Green Innovation and Carbon Neutrality in China
This article used data from 1980 to 2019 to examine the relationship between financial annexation, financial development, green innovation, and CO2 emissions in China. This review provides vital information to policymakers on how to go to preventative and therapeutic measures to reduce CO2 discharges in many sectors and demonstrates that innovation in the energy sector may help alleviate environmental change through energy productivity. The review used an autoregressive disseminated slack (ARDL) model to determine the dynamic coefficients for both the study’s long- and short-term periods. This study used the autoregressive circulated slack (ARDL) approach since it performs better than any alternative strategies in small cases, as was the case in this research. The ARDL and DARDL models show that CO2 emissions in China are increasing as a result of increased energy consumption and economic growth. Based on the findings, it is advised that China’s approach creators take on such sustainable power sources and improve them so that they may help meet the growing need for energy by replacing outdated traditional energy sources like coal, gas, and oil with them.
- 8 March 2022
- Published by Frontiers Media SA in Frontiers in Environmental Science
- Vol. 10
Open AccessFinancial Annexation, Green Innovation and Carbon Neutrality in China
This article used data from 1980 to 2019 to examine the relationship between financial annexation, financial development, green innovation, and CO2 emissions in China. This review provides vital information to policymakers on how to go to preventative and therapeutic measures to reduce CO2 discharges in many sectors and demonstrates that innovation in the energy sector may help alleviate environmental change through energy productivity. The review used an autoregressive disseminated slack (ARDL) model to determine the dynamic coefficients for both the study’s long- and short-term periods. This study used the autoregressive circulated slack (ARDL) approach since it performs better than any alternative strategies in small cases, as was the case in this research. The ARDL and DARDL models show that CO2 emissions in China are increasing as a result of increased energy consumption and economic growth. Based on the findings, it is advised that China’s approach creators take on such sustainable power sources and improve them so that they may help meet the growing need for energy by replacing outdated traditional energy sources like coal, gas, and oil with them.
- Open Access
- 6 January 2022
- Published by Taylor & Francis in Economic Research-Ekonomska Istraživanja
- Vol. 35 (1) , 5114-5130
Asymmetric impacts of technology innovation and environmental quality on tourism development in emerging economies
Tourism development contributes to higher economic output and is highly integrated with environmental quality and associated technologies. Although many studies explore the impact of tourism on carbon emissions; however, little is known regarding the effects of environmental pollution and technology innovation on tourism growth. Therefore, this study examines the impact of technology innovation and environmental pollution on inbound tourism in emerging economies. In doing so, we employ a recently developed panel quantiles regression and found that technology innovation and economic growth stimulate inbound tourism while increasing emissions limit tourist arrivals. These effects are not equally observed across all quantiles. Particularly, the impact of technology innovation is highest at higher quantiles, while the impact of the emissions is highest at lower quantiles. These results suggest that inbound tourism is asymmetrically affected by technology innovation and environmental quality of host destinations. Hence, emerging economies should encourage sustainable tourism by integrating green technologies and minimizing ecological hazards.
- 6 January 2022
- Published by Taylor & Francis in Economic Research-Ekonomska Istraživanja
- Vol. 35 (1) , 5114-5130
Open AccessAsymmetric impacts of technology innovation and environmental quality on tourism development in emerging economies
Tourism development contributes to higher economic output and is highly integrated with environmental quality and associated technologies. Although many studies explore the impact of tourism on carbon emissions; however, little is known regarding the effects of environmental pollution and technology innovation on tourism growth. Therefore, this study examines the impact of technology innovation and environmental pollution on inbound tourism in emerging economies. In doing so, we employ a recently developed panel quantiles regression and found that technology innovation and economic growth stimulate inbound tourism while increasing emissions limit tourist arrivals. These effects are not equally observed across all quantiles. Particularly, the impact of technology innovation is highest at higher quantiles, while the impact of the emissions is highest at lower quantiles. These results suggest that inbound tourism is asymmetrically affected by technology innovation and environmental quality of host destinations. Hence, emerging economies should encourage sustainable tourism by integrating green technologies and minimizing ecological hazards.