CSChangjian Shui

Changjian Shui

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Changjian Shui has a total of 43 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Unraveling the Mysteries of Label Noise in Source-Free Domain Adaptation: Theory and Practice

    by Gezheng Xu, Li Yi, Pengcheng Xu, Jiaqi Li, Ruizhi Pu, Changjian Shui, A. Ian McLeod, Boyu Wang, Charles Ling

    Recent source-free domain adaptation (SFDA) methods have focused on learning meaningful cluster structures in feature space, successfully adapting the knowledge from the source domain to the unlabeled target domain without accessing the private source data. However, existing methods rely on pseudo-labels generated by source models that can be noisy due to domain shift, presenting a significant challenge to their efficacy. In this paper, we study SFDA from the perspective of learning with label noise (LLN) and prove that the label noise in SFDA, unlike in conventional LLN scenarios, follows a different distribution assumption. This discrepancy renders some existing LLN methods less effective in SFDA. To address this issue and comprehensively improve adaptation performance, we tackle label noise in SFDA from two perspectives. First, we demonstrate that the early-time training phenomenon (ETP), previously observed in LLN settings, still exists in SFDA. Hence, we introduce a simple yet effective approach to leveraging ETP to improve current SFDA algorithms. Second, we propose a noise and variance control module, mitigating the label noise discrepancy between SFDA and LLN and enhancing the effectiveness of LLN methods in SFDA. Extensive empirical evaluation and analysis of four benchmarks show that our methods substantially outperform existing baselines.


  • Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning

    by Jiaqi Li, Yuanhao Lai, Rui Wang, Changjian Shui, Sabyasachi Sahoo, Charles X. Ling, Shichun Yang, Boyu Wang, Christian Gagné, Fan Zhou

    Continual learning aims to learn a series of tasks sequentially without forgetting the knowledge acquired from the previous ones. In this work, we propose the Hessian Aware Low-Rank Perturbation algorithm for continual learning. By modeling the parameter transitions along the sequential tasks with the weight matrix transformation, we propose to apply the low-rank approximation on the task-adaptive parameters in each layer of the neural networks. Specifically, we theoretically demonstrate the quantitative relationship between the Hessian and the proposed low-rank approximation. The approximation ranks are then globally determined according to the marginal change of the empirical loss estimated by the layer-specific gradient and low-rank approximation error. Furthermore, we control the model capacity by pruning less important parameters to diminish the parameter growth. We conduct extensive experiments on various benchmarks, including a dataset with large-scale tasks, and compare our method against some recent state-of-the-art methods to demonstrate the effectiveness and scalability of our proposed method. Empirical results show that our method performs better on different benchmarks, especially in achieving task order robustness and handling the forgetting issue.


  • Label shift conditioned hybrid querying for deep active learning

    by Jiaqi Li, Haojia Kong, Gezheng Xu, Changjian Shui, Ruizhi Pu, Zhao Kang, Charles X. Ling, Boyu Wang


  • Towards More General Loss and Setting in Unsupervised Domain Adaptation

    by Changjian Shui, Ruizhi Pu, Gezheng Xu, Jun Wen, Fan Zhou, Christian Gagné, Charles X. Ling, Boyu Wang

    In this article, we present an analysis of unsupervised domain adaptation with a series of theoretical and algorithmic results. We derive a novel Rényi- $\alpha$ divergence-based generalization bound, which is tailored to domain adaptation algorithms with arbitrary loss functions in a stochastic setting. Moreover, our theoretical results provide new insights into the assumptions for successful domain adaptation: the closeness between the conditional distributions of the domains and the Lipschitzness on the source domain. With these assumptions, we reveal the following: if their conditional generation distributions are close, the Lipschitzness property of the target domain can be transferred from the Lipschitzness on the source domain, without knowing the exact target distribution. Motivated by our analysis and assumptions, we further derive practical principles for deep domain adaptation: 1) Rényi-2 adversarial training for marginal distributions matching and 2) Lipschitz regularization for the classifier. Our experimental results on both synthetic and real-world datasets support our theoretical findings and the practical efficiency of the proposed principles.


  • Open Access

    Lifelong Online Learning from Accumulated Knowledge

    by Changjian Shui, William Wei Wang, Ihsen Hedhli, Chi Man Wong, Feng Wan, Boyu Wang, Christian Gagné

    In this article, we formulate lifelong learning as an online transfer learning procedure over consecutive tasks, where learning a given task depends on the accumulated knowledge. We propose a novel theoretical principled framework, lifelong online learning, where the learning process for each task is in an incremental manner. Specifically, our framework is composed of two-level predictions: the prediction information that is solely from the current task; and the prediction from the knowledge base by previous tasks. Moreover, this article tackled several fundamental challenges: arbitrary or even non-stationary task generation process, an unknown number of instances in each task, and constructing an efficient accumulated knowledge base. Notably, we provide a provable bound of the proposed algorithm, which offers insights on the how the accumulated knowledge improves the predictions. Finally, empirical evaluations on both synthetic and real datasets validate the effectiveness of the proposed algorithm.


  • Open Access

    Clinically Plausible Pathology-Anatomy Disentanglement in Patient Brain MRI with Structured Variational Priors

    by Anjun Hu, Jean-Pierre R. Falet, Brennan S. Nichyporuk, Changjian Shui, Douglas L. Arnold, Sotirios A. Tsaftaris, Tal Arbel

    We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain lesions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternative training configuration where we provide supervision to a subset of latent units. It is shown that (1) a partially supervised latent space achieves a higher degree of disentanglement between evidence of disease and subject-specific anatomy; (2) when the prior is formulated with an autoregressive structure, knowledge from the supervision can propagate to the unsupervised latent units, resulting in more informative latent representations capable of modelling anatomy-pathology interdependencies.

  • A novel domain adaptation theory with Jensen–Shannon divergence

    by Changjian Shui, Qi Chen, Jun Wen, Fan Zhou, Christian Gagné, Boyu Wang


  • Open Access

    On the benefits of representation regularization in invariance based domain generalization

    by Changjian Shui, Boyu Wang, Christian Gagné

    A crucial aspect of reliable machine learning is to design a deployable system for generalizing new related but unobserved environments. Domain generalization aims to alleviate such a prediction gap between the observed and unseen environments. Previous approaches commonly incorporated learning the invariant representation for achieving good empirical performance. In this paper, we reveal that merely learning the invariant representation is vulnerable to the related unseen environment. To this end, we derive a novel theoretical analysis to control the unseen test environment error in the representation learning, which highlights the importance of controlling the smoothness of representation. In practice, our analysis further inspires an efficient regularization method to improve the robustness in domain generalization. The proposed regularization is orthogonal to and can be straightforwardly adopted in existing domain generalization algorithms that ensure invariant representation learning. Empirical results show that our algorithm outperforms the base versions in various datasets and invariance criteria.


  • On the Benefits of Two Dimensional Metric Learning

    by Di Wu, Fan Zhou, Boyu Wang, Chi Man Wong, Changjian Shui, Yuan Zhou, Qicheng Lao, Feng Wan

    In this paper, we study two dimensional metric learning (2DML) for matrix data from both theoretical and algorithmic perspectives. We first investigate the generalization bounds of 2DML based on the notion of Rademacher complexity, which theoretically justifies the benefits of learning from matrices directly. Furthermore, we present a novel boosting-based algorithm that scales well with the feature dimension. Finally, we introduce an efficient rank-one correction algorithm, which is tailored to our boosting learning procedure to produce a low-rank solution to 2DML. As our algorithm works directly on the data in matrix representation, it scales well with the feature dimension, keeps the structure and dependence in the data, and has a more compact structure and much fewer parameters to optimize. Extensive evaluations on several benchmark data sets also empirically verify the effectiveness and efficiency of our algorithm.

  • Discriminative active learning for domain adaptation

    by Fan Zhou, Changjian Shui, Shichun Yang, Bincheng Huang, Boyu Wang, Brahim Chaib-Draa