Kaushik Roy
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Kaushik Roy has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Mehrtash Harandi
Monash University, Melbourne, VIC, Australia
5 shared publications - Peyman Moghadam
CSIRO
4 shared publications - Christian Simon
The Australian National University
3 shared publications - Oksam Chae
Kyung Hee University
2 shared publications - Farkhod Makhmudkhujaev
Department of Information and Communication Engineering, Inha University, Incheon, South Korea
1 shared publication - Gihun Song
LG Electronics, AI Lab, Seoul- 06772, South Korea
1 shared publication
Latest Publications
- Open Access
- 27 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 204072-204086
DUAL3D-Fed: Dual Distillation for 3D Continual Federated Learning With Vision–Language Models
Continual Federated Learning (CFL) for 3D object classification requires learning generalizable representations across distributed clients while maintaining feature-space consistency as new classes emerge. However, existing CFL methods, which are primarily designed for 2D vision tasks, fail to address the geometric complexity of 3D point clouds and lack mechanisms to simultaneously mitigate intra-task and inter-task forgetting. To address these limitations, we propose DUAL3D-Fed, a dual-distillation framework that supports scalable, privacy-preserving, and semantically coherent 3D continual federated learning. First, we propose an embedding-space distillation, where we align local and global representation distributions using Sinkhorn divergence, a regularized optimal transport metric that enforces consistency across client models and over time. Next, we introduce semantic grounding withVision-Language Models (VLMs), which uses fixed language prompts to anchor the 3D feature space to a rich semantic prior. This promotes clustering of point cloud samples with similar textual descriptions and separation of semantically dissimilar ones. Finally, a Logit-space distillation is added, where we stabilize class predictions using a hybrid distillation scheme that incorporates buffered exemplars to retain historical knowledge while adapting to novel classes. Experiments on four synthetic and real-world 3D benchmarks under CFL settings show that DUAL3D-Fed improves classification accuracy, enhanced feature representations, and exhibits greater robustness to class drift and forgetting than baseline methods.
- 27 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 204072-204086
Open AccessDUAL3D-Fed: Dual Distillation for 3D Continual Federated Learning With Vision–Language Models
Continual Federated Learning (CFL) for 3D object classification requires learning generalizable representations across distributed clients while maintaining feature-space consistency as new classes emerge. However, existing CFL methods, which are primarily designed for 2D vision tasks, fail to address the geometric complexity of 3D point clouds and lack mechanisms to simultaneously mitigate intra-task and inter-task forgetting. To address these limitations, we propose DUAL3D-Fed, a dual-distillation framework that supports scalable, privacy-preserving, and semantically coherent 3D continual federated learning. First, we propose an embedding-space distillation, where we align local and global representation distributions using Sinkhorn divergence, a regularized optimal transport metric that enforces consistency across client models and over time. Next, we introduce semantic grounding withVision-Language Models (VLMs), which uses fixed language prompts to anchor the 3D feature space to a rich semantic prior. This promotes clustering of point cloud samples with similar textual descriptions and separation of semantically dissimilar ones. Finally, a Logit-space distillation is added, where we stabilize class predictions using a hybrid distillation scheme that incorporates buffered exemplars to retain historical knowledge while adapting to novel classes. Experiments on four synthetic and real-world 3D benchmarks under CFL settings show that DUAL3D-Fed improves classification accuracy, enhanced feature representations, and exhibits greater robustness to class drift and forgetting than baseline methods.
- Open Access
- 23 November 2023
- Published by MDPI AG in Journal of Imaging
- Vol. 9 (12) , 259
CL3: Generalization of Contrastive Loss for Lifelong Learning
Lifelong learning portrays learning gradually in nonstationary environments and emulates the process of human learning, which is efficient, robust, and able to learn new concepts incrementally from sequential experience. To equip neural networks with such a capability, one needs to overcome the problem of catastrophic forgetting, the phenomenon of forgetting past knowledge while learning new concepts. In this work, we propose a novel knowledge distillation algorithm that makes use of contrastive learning to help a neural network to preserve its past knowledge while learning from a series of tasks. Our proposed generalized form of contrastive distillation strategy tackles catastrophic forgetting of old knowledge, and minimizes semantic drift by maintaining a similar embedding space, as well as ensures compactness in feature distribution to accommodate novel tasks in a current model. Our comprehensive study shows that our method achieves improved performances in the challenging class-incremental, task-incremental, and domain-incremental learning for supervised scenarios.
- 23 November 2023
- Published by MDPI AG in Journal of Imaging
- Vol. 9 (12) , 259
Open AccessCL3: Generalization of Contrastive Loss for Lifelong Learning
Lifelong learning portrays learning gradually in nonstationary environments and emulates the process of human learning, which is efficient, robust, and able to learn new concepts incrementally from sequential experience. To equip neural networks with such a capability, one needs to overcome the problem of catastrophic forgetting, the phenomenon of forgetting past knowledge while learning new concepts. In this work, we propose a novel knowledge distillation algorithm that makes use of contrastive learning to help a neural network to preserve its past knowledge while learning from a series of tasks. Our proposed generalized form of contrastive distillation strategy tackles catastrophic forgetting of old knowledge, and minimizes semantic drift by maintaining a similar embedding space, as well as ensures compactness in feature distribution to accommodate novel tasks in a current model. Our comprehensive study shows that our method achieves improved performances in the challenging class-incremental, task-incremental, and domain-incremental learning for supervised scenarios.
- 1 October 2023
- Published by Springer Nature
- p. 123-133
L3DMC: Lifelong Learning Using Distillation via Mixed-Curvature Space
- 1 October 2023
- Published by Springer Nature
- p. 123-133
L3DMC: Lifelong Learning Using Distillation via Mixed-Curvature Space
- Open Access
- 7 August 2023
- Published by Elsevier in Neural Networks
- Vol. 167, 65-79
Subspace distillation for continual learning
- 7 August 2023
- Published by Elsevier in Neural Networks
- Vol. 167, 65-79
Open AccessSubspace distillation for continual learning
- Open Access
Subspace Distillation for Continual Learning
An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL.
Open AccessSubspace Distillation for Continual Learning
An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL.
- Open Access
- 1 January 2019
- Published by Institution of Engineering and Technology (IET) in IET Image Processing
- Vol. 13 (1) , 224-232
Structural pattern‐based approach for Betacam dropout detection in degraded archived media
Detection of Betacam dropout defects that can occur in the digitisation process of old archived media has importance in the restoration of degraded data to a higher quality. Most of the existing methods rely on the temporal information of multiple consecutive frames to detect Betacam dropouts, which sometimes may not work well as several successive frames may contain a Betacam error at the same position. In this study, an automatic method of Betacam dropout error detection is proposed based on vertical patterns in a single frame. Hence, it is also applicable when temporal information‐based detectors fail. The results of performance tests done in real working environments demonstrate that the proposed Betacam dropout detection method performs much better than the existing methods.
- 1 January 2019
- Published by Institution of Engineering and Technology (IET) in IET Image Processing
- Vol. 13 (1) , 224-232
Open AccessStructural pattern‐based approach for Betacam dropout detection in degraded archived media
Detection of Betacam dropout defects that can occur in the digitisation process of old archived media has importance in the restoration of degraded data to a higher quality. Most of the existing methods rely on the temporal information of multiple consecutive frames to detect Betacam dropouts, which sometimes may not work well as several successive frames may contain a Betacam error at the same position. In this study, an automatic method of Betacam dropout error detection is proposed based on vertical patterns in a single frame. Hence, it is also applicable when temporal information‐based detectors fail. The results of performance tests done in real working environments demonstrate that the proposed Betacam dropout detection method performs much better than the existing methods.
- Open Access
- 13 June 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 6, 39917-39926
Background Subtraction Using Dominant Directional Pattern
Background subtraction technique is commonly used in foreground segmentation problem. Most of the existing color or intensity feature-based background subtraction methods suffer in non-stationary environments, such as presence of illumination variations. To address the issue, we introduce a new spatial feature descriptor that extracts the prominent directional information in a local neighborhood of a pixel. We use this local feature along with color information as the core component of a sample consensus-based model evolved from visual background extractor method. We also introduce an adaptive way to fuse color and local feature at each pixel to determine if the current observed features match to its corresponding background model while classification. The discriminative nature of our proposed local feature descriptor makes our model robust against the change of illumination. Extensive experiments on CDnet 2012 (changedetection.net) data set demonstrate that our background subtraction method outperforms many state-of-the-art methods.
- 13 June 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 6, 39917-39926
Open AccessBackground Subtraction Using Dominant Directional Pattern
Background subtraction technique is commonly used in foreground segmentation problem. Most of the existing color or intensity feature-based background subtraction methods suffer in non-stationary environments, such as presence of illumination variations. To address the issue, we introduce a new spatial feature descriptor that extracts the prominent directional information in a local neighborhood of a pixel. We use this local feature along with color information as the core component of a sample consensus-based model evolved from visual background extractor method. We also introduce an adaptive way to fuse color and local feature at each pixel to determine if the current observed features match to its corresponding background model while classification. The discriminative nature of our proposed local feature descriptor makes our model robust against the change of illumination. Extensive experiments on CDnet 2012 (changedetection.net) data set demonstrate that our background subtraction method outperforms many state-of-the-art methods.