Zheng Chai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zheng Chai has a total of 62 co-authors. Here are the top co-authors by number of shared publications:
- Chunhui Zhao
State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, China
8 shared publications - Yuchao Zheng
ByteDance, Hangzhou, China
8 shared publications - Biao Huang
University of Alberta
6 shared publications - Hui Lu
ByteDance, Hangzhou, China
5 shared publications - Hongtian Chen
Shanghai Jiao Tong University
4 shared publications - Peng Xu
Bytedance, San Jose, USA
4 shared publications
Latest Publications
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 7489-7500
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
While scaling laws for recommendation models have gained significant traction, existing architectures such as Wukong, HiFormer, and DHEN often struggle with sub-optimal designs and hardware under-utilization, limiting their practical scalability. Our previous TokenMixer architecture (introduced in the RankMixer paper) balanced effectiveness and efficiency by replacing self-attention with a lightweight token mixing operator; however, it encountered critical bottlenecks in deeper configurations, including sub-optimal residual paths, vanishing gradients, incomplete MoE sparsification, and constrained scalability. In this paper, we propose TokenMixer-Large, a systematically evolved architecture designed for extreme-scale recommendation. By introducing a mixing-and-reverting operation, inter-layer residuals, and an auxiliary loss, we ensure stable gradient propagation as model depth increases. Furthermore, we incorporate a Sparse Per-token MoE to enable efficient parameter expansion. TokenMixer-Large successfully scales to 7 billion parameters for online traffic and 15 billion in offline experiments. Currently deployed in multiple scenarios at ByteDance, TokenMixer-Large has achieved substantial performance gains: delivering a 1.66% increase in orders and a 2.98% boost in per-capita preview payment GMV for e-commerce; improving ADSS by 2.0% in advertising; and achieving 1.4% revenue growth in live streaming.
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 7489-7500
Open AccessTokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
While scaling laws for recommendation models have gained significant traction, existing architectures such as Wukong, HiFormer, and DHEN often struggle with sub-optimal designs and hardware under-utilization, limiting their practical scalability. Our previous TokenMixer architecture (introduced in the RankMixer paper) balanced effectiveness and efficiency by replacing self-attention with a lightweight token mixing operator; however, it encountered critical bottlenecks in deeper configurations, including sub-optimal residual paths, vanishing gradients, incomplete MoE sparsification, and constrained scalability. In this paper, we propose TokenMixer-Large, a systematically evolved architecture designed for extreme-scale recommendation. By introducing a mixing-and-reverting operation, inter-layer residuals, and an auxiliary loss, we ensure stable gradient propagation as model depth increases. Furthermore, we incorporate a Sparse Per-token MoE to enable efficient parameter expansion. TokenMixer-Large successfully scales to 7 billion parameters for online traffic and 15 billion in offline experiments. Currently deployed in multiple scenarios at ByteDance, TokenMixer-Large has achieved substantial performance gains: delivering a 1.66% increase in orders and a 2.98% boost in per-capita preview payment GMV for e-commerce; improving ADSS by 2.0% in advertising; and achieving 1.4% revenue growth in live streaming.
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 7422-7431
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
As industrial recommender systems enter a scaling-driven regime, Transformer architectures have become increasingly attractive for scaling models towards larger capacity and longer sequence. However, existing Transformer-based recommendation models remain structurally fragmented, where sequence modeling and feature interaction are implemented as separate modules with independent parameterization. Such designs introduce a fundamental co-scaling challenge, as model capacity must be suboptimally allocated between dense feature interaction and sequence modeling under a limited computational budget. In this work, we propose MixFormer, a unified Transformer-style architecture tailored for recommender systems, which jointly models sequential behaviors and feature interactions within a single backbone. Through a unified parameterization, MixFormer enables effective co-scaling across both dense capacity and sequence length, mitigating the trade-off observed in decoupled designs. Moreover, the integrated architecture facilitates deep interaction between sequential and non-sequential representations, allowing high-order feature semantics to directly inform sequence aggregation and enhancing overall expressiveness. To ensure industrial practicality, we further introduce a user-item decoupling strategy for efficiency optimizations that significantly reduce redundant computation and inference latency. Extensive experiments on large-scale industrial datasets demonstrate that MixFormer consistently exhibits superior accuracy and efficiency. Furthermore, large-scale online A/B tests on two production recommender systems, Douyin and Douyin Lite, show consistent improvements in user engagement metrics, including active days and in-app usage duration.
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 7422-7431
Open AccessMixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
As industrial recommender systems enter a scaling-driven regime, Transformer architectures have become increasingly attractive for scaling models towards larger capacity and longer sequence. However, existing Transformer-based recommendation models remain structurally fragmented, where sequence modeling and feature interaction are implemented as separate modules with independent parameterization. Such designs introduce a fundamental co-scaling challenge, as model capacity must be suboptimally allocated between dense feature interaction and sequence modeling under a limited computational budget. In this work, we propose MixFormer, a unified Transformer-style architecture tailored for recommender systems, which jointly models sequential behaviors and feature interactions within a single backbone. Through a unified parameterization, MixFormer enables effective co-scaling across both dense capacity and sequence length, mitigating the trade-off observed in decoupled designs. Moreover, the integrated architecture facilitates deep interaction between sequential and non-sequential representations, allowing high-order feature semantics to directly inform sequence aggregation and enhancing overall expressiveness. To ensure industrial practicality, we further introduce a user-item decoupling strategy for efficiency optimizations that significantly reduce redundant computation and inference latency. Extensive experiments on large-scale industrial datasets demonstrate that MixFormer consistently exhibits superior accuracy and efficiency. Furthermore, large-scale online A/B tests on two production recommender systems, Douyin and Douyin Lite, show consistent improvements in user engagement metrics, including active days and in-app usage duration.
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 8258-8266
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 8258-8266
Open AccessMSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 590-599
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Industrial large-scale recommendation models (LRMs) face the challenge of jointly modeling long-range user behavior sequences and heterogeneous non-sequential features under strict efficiency constraints. However, most existing architectures employ a decoupled pipeline: long sequences are first compressed with a query-token based sequence compressor like LONGER, followed by fusion with dense features through token-mixing modules like RankMixer, which thereby limits both the representation capacity and the interaction flexibility. This paper presents HyFormer, a unified hybrid transformer architecture that tightly integrates long-sequence modeling and feature interaction into a single backbone. From the perspective of sequence modeling, we revisit and redesign query tokens in LRMs, and frame the LRM modeling task as an alternating optimization process that integrates two core components: Query Decoding which expands non-sequential features into Global Tokens and performs long sequence decoding over layer-wise key-value representations of long behavioral sequences; and Query Boosting which enhances cross-query and cross-sequence heterogeneous interactions via efficient token mixing. The two complementary mechanisms are performed iteratively to refine semantic representations across layers. Extensive experiments on billion-scale industrial datasets demonstrate that HyFormer consistently outperforms strong LONGER and RankMixer baselines under comparable parameter and FLOPs budgets, while exhibiting superior scaling behavior with increasing parameters and FLOPs. Large-scale online A/B tests in high-traffic production systems further validate its effectiveness, showing significant gains over deployed state-of-the-art models. These results highlight the practicality and scalability of HyFormer as a unified modeling framework for industrial LRMs.
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 590-599
Open AccessHyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Industrial large-scale recommendation models (LRMs) face the challenge of jointly modeling long-range user behavior sequences and heterogeneous non-sequential features under strict efficiency constraints. However, most existing architectures employ a decoupled pipeline: long sequences are first compressed with a query-token based sequence compressor like LONGER, followed by fusion with dense features through token-mixing modules like RankMixer, which thereby limits both the representation capacity and the interaction flexibility. This paper presents HyFormer, a unified hybrid transformer architecture that tightly integrates long-sequence modeling and feature interaction into a single backbone. From the perspective of sequence modeling, we revisit and redesign query tokens in LRMs, and frame the LRM modeling task as an alternating optimization process that integrates two core components: Query Decoding which expands non-sequential features into Global Tokens and performs long sequence decoding over layer-wise key-value representations of long behavioral sequences; and Query Boosting which enhances cross-query and cross-sequence heterogeneous interactions via efficient token mixing. The two complementary mechanisms are performed iteratively to refine semantic representations across layers. Extensive experiments on billion-scale industrial datasets demonstrate that HyFormer consistently outperforms strong LONGER and RankMixer baselines under comparable parameter and FLOPs budgets, while exhibiting superior scaling behavior with increasing parameters and FLOPs. Large-scale online A/B tests in high-traffic production systems further validate its effectiveness, showing significant gains over deployed state-of-the-art models. These results highlight the practicality and scalability of HyFormer as a unified modeling framework for industrial LRMs.
- Top Cited PapersOpen Access
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 6309-6316
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 6309-6316
Top Cited PapersOpen AccessRankMixer: Scaling Up Ranking Models in Industrial Recommenders
- Top Cited PapersOpen Access
- 22 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 247-256
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
- 22 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 247-256
Top Cited PapersOpen AccessLONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
- Open Access
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 4234-4238
Adaptive Domain Scaling for Personalized Sequential Modeling in Recommenders
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 4234-4238
Open AccessAdaptive Domain Scaling for Personalized Sequential Modeling in Recommenders
- Open Access
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 1162-1166
Large Memory Network for Recommendation
- 8 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 1162-1166
Open AccessLarge Memory Network for Recommendation
- 6 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 8725-8737
Walk Before You Can Run: Sampling-Rate-Aware Sequential Knowledge Transfer for Multirate Process Anomaly Detection
Anomaly detection plays a vital role in monitoring industrial processes. Despite the extensive development of deep anomaly detection approaches, many assume uniform sampling rates for process variables, yet multirate phenomena are common in practice. The resultant large-scale missing data and varying dynamics of variables with different sampling rates pose a challenge to conventional anomaly detection methods. To this end, this paper developed a SAmpling-Rate-aware sequential Knowledge trAnsfer () model for detecting the anomalies in multirate industrial processes. First, the raw multirate dataset is chunked into multiple data blocks, such that samples in each block correspond to a specific sampling rate and can be more readily processed by deep neural networks. Then, a Sampling-rate aware Slow Variational Auto-Encoder () model is developed, in which a sampling-rate-aware slowness principle is devised and integrated to enable both data block-and instance-wise personalized dynamic features characterization. Besides, to alleviate the scarce sample problem in the low-sampling-rate data block due to the multirate phenomena, a Sequential Knowledge Transfer (SKT) strategy is devised to convey the knowledge from the high-rate data to facilitate low-rate data modeling and improve the overall monitoring performance. Experimental results from a real-world coal mill in a thermal power plant multirate process demonstrate the effectiveness of the proposed method. Note to Practitioners —Due to the challenges resulting from large-scale missing data and complex varying dynamics, multirate process anomaly detection is a crucial task in modern industries. To tackle the challenge, this paper presents SARKA, which consists of two modules, i.e., and SKT. Among them, serves as the base model, which integrates a novel devised slowness principle under the probabilistic framework for improved sampling-rate-aware dynamic features characterization. Besides, SKT is elaborated to bridge the multiple base models by conveying the modeling knowledge from those high-rate data to low-rate data, as the amount of the low-rate data is generally scarce to train a valid base model. Therefore, the base model can sufficiently capture both the static and dynamic features of the multirate process, and the SKT further enhances the modeling of the low-rate data, yielding improved performance on the overall multirate process.
- 6 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 8725-8737
Walk Before You Can Run: Sampling-Rate-Aware Sequential Knowledge Transfer for Multirate Process Anomaly Detection
Anomaly detection plays a vital role in monitoring industrial processes. Despite the extensive development of deep anomaly detection approaches, many assume uniform sampling rates for process variables, yet multirate phenomena are common in practice. The resultant large-scale missing data and varying dynamics of variables with different sampling rates pose a challenge to conventional anomaly detection methods. To this end, this paper developed a SAmpling-Rate-aware sequential Knowledge trAnsfer () model for detecting the anomalies in multirate industrial processes. First, the raw multirate dataset is chunked into multiple data blocks, such that samples in each block correspond to a specific sampling rate and can be more readily processed by deep neural networks. Then, a Sampling-rate aware Slow Variational Auto-Encoder () model is developed, in which a sampling-rate-aware slowness principle is devised and integrated to enable both data block-and instance-wise personalized dynamic features characterization. Besides, to alleviate the scarce sample problem in the low-sampling-rate data block due to the multirate phenomena, a Sequential Knowledge Transfer (SKT) strategy is devised to convey the knowledge from the high-rate data to facilitate low-rate data modeling and improve the overall monitoring performance. Experimental results from a real-world coal mill in a thermal power plant multirate process demonstrate the effectiveness of the proposed method. Note to Practitioners —Due to the challenges resulting from large-scale missing data and complex varying dynamics, multirate process anomaly detection is a crucial task in modern industries. To tackle the challenge, this paper presents SARKA, which consists of two modules, i.e., and SKT. Among them, serves as the base model, which integrates a novel devised slowness principle under the probabilistic framework for improved sampling-rate-aware dynamic features characterization. Besides, SKT is elaborated to bridge the multiple base models by conveying the modeling knowledge from those high-rate data to low-rate data, as the amount of the low-rate data is generally scarce to train a valid base model. Therefore, the base model can sufficiently capture both the static and dynamic features of the multirate process, and the SKT further enhances the modeling of the low-rate data, yielding improved performance on the overall multirate process.
- 2 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 19 (4) , 3649-3658
Fault-Prototypical Adapted Network for Cross-Domain Industrial Intelligent Diagnosis
Despite rapid advances in machine learning based fault diagnosis, their identical distribution assumption of the training (source domain) and testing data (target domain) is generally challenged in industrial applications due to the variation of working conditions. In this article, a fault-prototypical adapted network (FPAN) is proposed, which enables cross-domain industrial intelligent fault diagnosis aided by deep transfer learning. First, a similarity learning-based discrimination module is designed to learn fault prototypes (FPs) that are representative for each fault and discriminative across different faults. Then, a fault prototypical-adaptation module is developed, which adapts the multiple FPs to the target dataset and enables more precise category-wise domain invariance. The two modules are trained simultaneously to extract transferrable and discriminative FPs, by which the cross-domain intelligent diagnosis can be readily achieved. Experimental results on two industrial cases illustrate that the proposed approach learns transferable feature representations that better reduce domain discrepancy, and provides improved diagnosis performance on target data. Note to Practitioners—As the distribution of source and target domain data may differ due to the varying working conditions, cross-domain intelligent fault diagnosis plays an increasingly important role in practical industrial cases. This work presents the FPAN model, in which the fault discrimination and the category-wise adaptation are naturally connected and unified by exploring representative virtual FP for each category, irrespective of which domain that the category comes from. Connected by the adaptive FPs, the similarity learning-based discrimination and the fault-prototypical adaptation modules can benefit from each other to boost the target learning. Finally, the cross-domain fault diagnosis can be performed based on the similarities with different FP representations. The proposed method is readily applicable to various intelligent fault diagnosis problems, including the mechanical cases and the industrial processes.
- 2 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 19 (4) , 3649-3658
Fault-Prototypical Adapted Network for Cross-Domain Industrial Intelligent Diagnosis
Despite rapid advances in machine learning based fault diagnosis, their identical distribution assumption of the training (source domain) and testing data (target domain) is generally challenged in industrial applications due to the variation of working conditions. In this article, a fault-prototypical adapted network (FPAN) is proposed, which enables cross-domain industrial intelligent fault diagnosis aided by deep transfer learning. First, a similarity learning-based discrimination module is designed to learn fault prototypes (FPs) that are representative for each fault and discriminative across different faults. Then, a fault prototypical-adaptation module is developed, which adapts the multiple FPs to the target dataset and enables more precise category-wise domain invariance. The two modules are trained simultaneously to extract transferrable and discriminative FPs, by which the cross-domain intelligent diagnosis can be readily achieved. Experimental results on two industrial cases illustrate that the proposed approach learns transferable feature representations that better reduce domain discrepancy, and provides improved diagnosis performance on target data. Note to Practitioners—As the distribution of source and target domain data may differ due to the varying working conditions, cross-domain intelligent fault diagnosis plays an increasingly important role in practical industrial cases. This work presents the FPAN model, in which the fault discrimination and the category-wise adaptation are naturally connected and unified by exploring representative virtual FP for each category, irrespective of which domain that the category comes from. Connected by the adaptive FPs, the similarity learning-based discrimination and the fault-prototypical adaptation modules can benefit from each other to boost the target learning. Finally, the cross-domain fault diagnosis can be performed based on the similarities with different FP representations. The proposed method is readily applicable to various intelligent fault diagnosis problems, including the mechanical cases and the industrial processes.