Xian Wei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xian Wei has a total of 177 co-authors. Here are the top co-authors by number of shared publications:
- Mingsong Chen
East China Normal University, Shanghai, China
13 shared publications - Xuan Tang
School of Communication and Electronic Engineering, East China Normal University, Shanghai, China
13 shared publications Xuan Tang
School of Communication and Electronic Engineering, East China Normal University
7 shared publications- Jian Yang
School of Geospatial Information, Information Engineering University, Zhengzhou, China
6 shared publications Mingsong Chen
Software Engineering Institute, East China Normal University
5 shared publicationsXiaoliang Tang
Zhejiang University of Science and Technology
5 shared publications
Latest Publications
- Open Access
- 25 April 2026
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 58 (11) , 1-38
Robot Navigation via Foundation Language Models: A Review
Recently, with advances in Large Language Models(LLMs), robot navigation models have demonstrated superior generalization capabilities across environment perception, decision-making, reasoning, planning, instruction understanding, and human-robot interaction. In this article, we systematically review recent LLM-based robot navigation research articles and categorize them into a novel taxonomy comprising perception, planning, control, interaction, and coordination. We also present an overview of the principal datasets, simulations, and metrics used in robot navigation, analyzing the distinctive characteristics of the datasets and the performance of the main LLM-based methods. Furthermore, we discuss the challenges hindering the integration of LLMs into robot navigation and provide opportunities and potential directions for future development.
- 25 April 2026
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 58 (11) , 1-38
Open AccessRobot Navigation via Foundation Language Models: A Review
Recently, with advances in Large Language Models(LLMs), robot navigation models have demonstrated superior generalization capabilities across environment perception, decision-making, reasoning, planning, instruction understanding, and human-robot interaction. In this article, we systematically review recent LLM-based robot navigation research articles and categorize them into a novel taxonomy comprising perception, planning, control, interaction, and coordination. We also present an overview of the principal datasets, simulations, and metrics used in robot navigation, analyzing the distinctive characteristics of the datasets and the performance of the main LLM-based methods. Furthermore, we discuss the challenges hindering the integration of LLMs into robot navigation and provide opportunities and potential directions for future development.
- 30 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. PP (10518215) , 1
Zero-Shot Visual Grounding via Cascade Semantic Prototype Learning
Compared to standard visual grounding (VG), zero-shot visual grounding (ZSVG) requires robust localization of novel image–query pairs unseen during training, creating a clear gap from supervised settings. Existing two-stage approaches rely too heavily on external knowledge from pre-trained vision-language models and underexplore in-context semantics, while current one-stage models fail to leverage region-level cues to enhance discriminative representation learning. In this paper, inspired by the information processing procedure of human beings, we propose a novel one-stage network, dubbed Cascade Semantic Prototype Learning (CaSePro), to tackle these limitations. Specifically, CaSePro builds two types of prototypes that capture prototypical multimodal representations for global contexts and local semantics, respectively. With global context prototypes, CaSePro explores context-related cues to enhance the coarsely fused global multimodal features, enabling the model to learn more comprehensive representations. With local semantic prototypes, CaSePro searches for region-level multimodal correspondences to augment the prototypical global patterns, thereby strengthening the cross-modal relationship between the query and visual representations. Moreover, to enhance the generalization ability of the proposed approach on sample-imbalanced benchmark datasets, CaSePro further incorporates an Adaptive Focal Loss that dynamically reweights hard and easy samples by accounting for the model’s grounding uncertainty. Extensive experiments on ten benchmarks demonstrate that CaSePro outperforms all existing one-stage approaches and achieves performance comparable to state-of-the-art two-stage methods.
- 30 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. PP (10518215) , 1
Zero-Shot Visual Grounding via Cascade Semantic Prototype Learning
Compared to standard visual grounding (VG), zero-shot visual grounding (ZSVG) requires robust localization of novel image–query pairs unseen during training, creating a clear gap from supervised settings. Existing two-stage approaches rely too heavily on external knowledge from pre-trained vision-language models and underexplore in-context semantics, while current one-stage models fail to leverage region-level cues to enhance discriminative representation learning. In this paper, inspired by the information processing procedure of human beings, we propose a novel one-stage network, dubbed Cascade Semantic Prototype Learning (CaSePro), to tackle these limitations. Specifically, CaSePro builds two types of prototypes that capture prototypical multimodal representations for global contexts and local semantics, respectively. With global context prototypes, CaSePro explores context-related cues to enhance the coarsely fused global multimodal features, enabling the model to learn more comprehensive representations. With local semantic prototypes, CaSePro searches for region-level multimodal correspondences to augment the prototypical global patterns, thereby strengthening the cross-modal relationship between the query and visual representations. Moreover, to enhance the generalization ability of the proposed approach on sample-imbalanced benchmark datasets, CaSePro further incorporates an Adaptive Focal Loss that dynamically reweights hard and easy samples by accounting for the model’s grounding uncertainty. Extensive experiments on ten benchmarks demonstrate that CaSePro outperforms all existing one-stage approaches and achieves performance comparable to state-of-the-art two-stage methods.
- Open Access
- 2 March 2026
- Published by MDPI AG in Electronics
- Vol. 15 (5) , 1033
Structure-Aware Topological Exploration: A Semantic Seeded Voronoi Approach for Unstructured Environments
In autonomous exploration tasks in unstructured terrain, exploration efficiency and map topology quality have been a difficult problem to balance. Among the current autonomous exploration methods, geometry-based exploration methods only focus on exploration efficiency but not map quality, which not only leads to frequent backtracking by the robot, but also tends to ignore non-geometric risks such as negative obstacles. To address this pain point, we propose the Structure-Aware Topology Exploration framework. Unlike pure geometric exploration, we utilize U-Net to semantically analyze the unmanned aerial vehicle aerial images, and force the robot’s path to be anchored to the geometric axis of the safe area through the Semantic Seeded Voronoi mechanism. To avoid map redundancy leading to backtracking, we directly introduce topological sparsity constraints in the decision function to realize online structural pruning during exploration. Simulation experiments based on real-world aerial imagery demonstrate that the proposed framework effectively overcomes the late-stage exploration plateau: compared with purely geometric baselines (Rapidly exploring Random Tree and Frontier), it reduces average path length to 278.4 m (45% reduction) and improves exploration efficiency by 80%; compared with the semantic frontier-based baseline, it achieves 28.6% higher efficiency and 13% shorter path length, maximizing information gain per unit travel distance.
- 2 March 2026
- Published by MDPI AG in Electronics
- Vol. 15 (5) , 1033
Open AccessStructure-Aware Topological Exploration: A Semantic Seeded Voronoi Approach for Unstructured Environments
In autonomous exploration tasks in unstructured terrain, exploration efficiency and map topology quality have been a difficult problem to balance. Among the current autonomous exploration methods, geometry-based exploration methods only focus on exploration efficiency but not map quality, which not only leads to frequent backtracking by the robot, but also tends to ignore non-geometric risks such as negative obstacles. To address this pain point, we propose the Structure-Aware Topology Exploration framework. Unlike pure geometric exploration, we utilize U-Net to semantically analyze the unmanned aerial vehicle aerial images, and force the robot’s path to be anchored to the geometric axis of the safe area through the Semantic Seeded Voronoi mechanism. To avoid map redundancy leading to backtracking, we directly introduce topological sparsity constraints in the decision function to realize online structural pruning during exploration. Simulation experiments based on real-world aerial imagery demonstrate that the proposed framework effectively overcomes the late-stage exploration plateau: compared with purely geometric baselines (Rapidly exploring Random Tree and Frontier), it reduces average path length to 278.4 m (45% reduction) and improves exploration efficiency by 80%; compared with the semantic frontier-based baseline, it achieves 28.6% higher efficiency and 13% shorter path length, maximizing information gain per unit travel distance.
- Open Access
- 1 September 2025
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 9, 7724-7732
PDDFormer: Pairwise Distance Distribution Graph Transformer for Crystal Material Property Prediction
Crystal structures can be simplified as a periodic point set that repeats across three-dimensional space along an underlying lattice. Traditionally, crystal representation methods rely on descriptors such as lattice parameters, symmetry, and space groups to characterize the structure. However, in reality, atoms in materials always vibrate above absolute zero, causing their positions to fluctuate continuously. This dynamic behavior disrupts the fundamental periodicity of the lattice, making crystal graphs based on static lattice parameters and conventional descriptors discontinuous under slight perturbations. Chemists proposed the pairwise distance distribution (PDD) method to address this. However, the completeness of PDD requires defining a large number of neighboring atoms, leading to high computational costs. Additionally, PDD does not account for atomic information, making it challenging to apply it directly to crystal material property prediction tasks. To tackle these challenges, we introduce the atom-weighted Pairwise Distance Distribution (WPDD) and Unit cell Pairwise Distance Distribution (UPDD) for the first time, applying them to the construction of multi-edge crystal graphs. We demonstrate the continuity and general completeness of crystal graphs under slight atomic position perturbations. Moreover, by modeling PDD as global information and integrating it into matrix-based message passing, we significantly reduce computational costs. Comprehensive evaluation results show that WPDDFormer achieves state-of-the-art predictive accuracy across tasks on benchmark datasets such as the Materials Project and JARVIS-DFT.
- 1 September 2025
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 9, 7724-7732
Open AccessPDDFormer: Pairwise Distance Distribution Graph Transformer for Crystal Material Property Prediction
Crystal structures can be simplified as a periodic point set that repeats across three-dimensional space along an underlying lattice. Traditionally, crystal representation methods rely on descriptors such as lattice parameters, symmetry, and space groups to characterize the structure. However, in reality, atoms in materials always vibrate above absolute zero, causing their positions to fluctuate continuously. This dynamic behavior disrupts the fundamental periodicity of the lattice, making crystal graphs based on static lattice parameters and conventional descriptors discontinuous under slight perturbations. Chemists proposed the pairwise distance distribution (PDD) method to address this. However, the completeness of PDD requires defining a large number of neighboring atoms, leading to high computational costs. Additionally, PDD does not account for atomic information, making it challenging to apply it directly to crystal material property prediction tasks. To tackle these challenges, we introduce the atom-weighted Pairwise Distance Distribution (WPDD) and Unit cell Pairwise Distance Distribution (UPDD) for the first time, applying them to the construction of multi-edge crystal graphs. We demonstrate the continuity and general completeness of crystal graphs under slight atomic position perturbations. Moreover, by modeling PDD as global information and integrating it into matrix-based message passing, we significantly reduce computational costs. Comprehensive evaluation results show that WPDDFormer achieves state-of-the-art predictive accuracy across tasks on benchmark datasets such as the Materials Project and JARVIS-DFT.
- Open Access
- 1 September 2025
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 12, 10723-10732
Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review
Embodied multi-agent systems (EMAS) have attracted growing attention for their potential to address complex, real-world challenges in areas such as logistics and robotics. Recent advances in foundation models pave the way for generative agents capable of richer communication and adaptive problem-solving. This survey provides a systematic examination of how EMAS can benefit from these generative capabilities. We propose a taxonomy that categorizes EMAS by system architectures and embodiment modalities, emphasizing how collaboration spans both physical and virtual contexts. Central building blocks, perception, planning, communication, and feedback, are then analyzed to illustrate how generative techniques bolster system robustness and flexibility. Through concrete examples, we demonstrate the transformative effects of integrating foundation models into embodied, multi-agent frameworks. Finally, we discuss challenges and future directions, underlining the significant promise of EMAS to reshape the landscape of AI-driven collaboration.
- 1 September 2025
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 12, 10723-10732
Open AccessGenerative Multi-Agent Collaboration in Embodied AI: A Systematic Review
Embodied multi-agent systems (EMAS) have attracted growing attention for their potential to address complex, real-world challenges in areas such as logistics and robotics. Recent advances in foundation models pave the way for generative agents capable of richer communication and adaptive problem-solving. This survey provides a systematic examination of how EMAS can benefit from these generative capabilities. We propose a taxonomy that categorizes EMAS by system architectures and embodiment modalities, emphasizing how collaboration spans both physical and virtual contexts. Central building blocks, perception, planning, communication, and feedback, are then analyzed to illustrate how generative techniques bolster system robustness and flexibility. Through concrete examples, we demonstrate the transformative effects of integrating foundation models into embodied, multi-agent frameworks. Finally, we discuss challenges and future directions, underlining the significant promise of EMAS to reshape the landscape of AI-driven collaboration.
- 22 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (8) , 12372-12386
FuseFormer: A Manifold Metric Fusing Attention for Pedestrian Trajectory Prediction
Accurate pedestrian trajectory prediction is critical for ensuring the safety of autonomous vehicles and advancing higher levels of driving automation. However, the complex interpersonal interactions and highly dynamic trajectory patterns in real-world scenarios pose significant challenges to achieving precise predictions. Recently, Transformers have shown remarkable success in pedestrian trajectory prediction, primarily due to their effective modeling of temporal and spatial dependencies via Multi-Head Self-Attention (MHA) mechanisms. Despite these advancements, existing self-attention methods often rely on Euclidean distance-based metrics and dot-product operations, which are inadequate for capturing interaction-induced trajectory curvatures. To address this limitation, we propose a novel hybrid Transformer architecture, FuseFormer, that incorporates Geodesic Self-Attention (GSA) mechanisms. GSA utilizes geodesic distances to characterize interaction features effectively, complementing MHA, which excels in capturing local features and maintaining temporal correlations. FuseFormer employs a gating network to adaptively combine GSA and MHA embeddings, leveraging their complementary strengths. Additionally, FuseFormer integrates a Transformer-based Neural Ordinary Differential Equation (ODE) decoder to model trajectory temporal dynamics. This design enables the generation of future trajectories that align closely with motion trends while adapting the network depth to input sequence lengths. Experimental results demonstrate that FuseFormer achieves state-of-the-art performance across widely used pedestrian trajectory prediction datasets, including ETH/UCY, SDD, and NBA. These results underscore the model’s effectiveness and generalization capability in capturing complex interaction patterns and handling diverse scenarios.
- 22 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (8) , 12372-12386
FuseFormer: A Manifold Metric Fusing Attention for Pedestrian Trajectory Prediction
Accurate pedestrian trajectory prediction is critical for ensuring the safety of autonomous vehicles and advancing higher levels of driving automation. However, the complex interpersonal interactions and highly dynamic trajectory patterns in real-world scenarios pose significant challenges to achieving precise predictions. Recently, Transformers have shown remarkable success in pedestrian trajectory prediction, primarily due to their effective modeling of temporal and spatial dependencies via Multi-Head Self-Attention (MHA) mechanisms. Despite these advancements, existing self-attention methods often rely on Euclidean distance-based metrics and dot-product operations, which are inadequate for capturing interaction-induced trajectory curvatures. To address this limitation, we propose a novel hybrid Transformer architecture, FuseFormer, that incorporates Geodesic Self-Attention (GSA) mechanisms. GSA utilizes geodesic distances to characterize interaction features effectively, complementing MHA, which excels in capturing local features and maintaining temporal correlations. FuseFormer employs a gating network to adaptively combine GSA and MHA embeddings, leveraging their complementary strengths. Additionally, FuseFormer integrates a Transformer-based Neural Ordinary Differential Equation (ODE) decoder to model trajectory temporal dynamics. This design enables the generation of future trajectories that align closely with motion trends while adapting the network depth to input sequence lengths. Experimental results demonstrate that FuseFormer achieves state-of-the-art performance across widely used pedestrian trajectory prediction datasets, including ETH/UCY, SDD, and NBA. These results underscore the model’s effectiveness and generalization capability in capturing complex interaction patterns and handling diverse scenarios.
- 5 February 2025
- Published by World Scientific Pub Co Pte Ltd in Journal of Circuits, Systems and Computers
- Vol. 34 (07)
CyclicFL: Efficient Federated Learning with Cyclic Model Pre-training
Federated learning (FL) has been proposed to enable distributed learning on artificial intelligence Internet of Things (AIoT) devices with guarantees of high-level data privacy. Since random initial models in FL can easily result in unregulated stochastic gradient descent (SGD) processes, existing FL methods greatly suffer from both slow convergence and poor accuracy, especially in non-IID scenarios. To address this problem, we propose a novel method named CyclicFL, which can quickly derive effective initial models to guide the SGD processes, thus improving the overall FL training performance. We formally analyze the significance of data consistency between the pre-training and training stages of CyclicFL, showing the limited Lipschitzness of loss for the pre-trained models by CyclicFL. Moreover, we systematically prove that our method can achieve a faster convergence speed under various convexity assumptions. Unlike traditional centralized pre-training methods that require public proxy data, CyclicFL pre-trains initial models on selected AIoT devices cyclically without exposing their local data. Therefore, they can be easily integrated into any security-critical FL methods. Comprehensive experimental results show that CyclicFL can not only improve maximum classification accuracy by up to 14.11%, but also significantly accelerate the overall FL training process.
- 5 February 2025
- Published by World Scientific Pub Co Pte Ltd in Journal of Circuits, Systems and Computers
- Vol. 34 (07)
CyclicFL: Efficient Federated Learning with Cyclic Model Pre-training
Federated learning (FL) has been proposed to enable distributed learning on artificial intelligence Internet of Things (AIoT) devices with guarantees of high-level data privacy. Since random initial models in FL can easily result in unregulated stochastic gradient descent (SGD) processes, existing FL methods greatly suffer from both slow convergence and poor accuracy, especially in non-IID scenarios. To address this problem, we propose a novel method named CyclicFL, which can quickly derive effective initial models to guide the SGD processes, thus improving the overall FL training performance. We formally analyze the significance of data consistency between the pre-training and training stages of CyclicFL, showing the limited Lipschitzness of loss for the pre-trained models by CyclicFL. Moreover, we systematically prove that our method can achieve a faster convergence speed under various convexity assumptions. Unlike traditional centralized pre-training methods that require public proxy data, CyclicFL pre-trains initial models on selected AIoT devices cyclically without exposing their local data. Therefore, they can be easily integrated into any security-critical FL methods. Comprehensive experimental results show that CyclicFL can not only improve maximum classification accuracy by up to 14.11%, but also significantly accelerate the overall FL training process.
- Top Cited PapersOpen Access
- 24 January 2025
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 57 (5) , 1-37
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold
Deep Learning (DL) has achieved remarkable success in tackling complex Artificial Intelligence tasks. The standard training of neural networks employs backpropagation to compute gradients and utilizes various optimization algorithms in the Euclidean space ℛ n . However, this optimization process faces challenges, such as the local optimal issues and the problem of gradient vanishing and exploding. To address these problems, Riemannian optimization offers a powerful extension to solve optimization problems in deep learning. By incorporating the prior constraint structure and the metric information of the underlying geometric information, Riemannian optimization-based DL offers a more stable and reliable optimization process, as well as enhanced adaptability to complex data structures. This article presents a comprehensive survey of applying geometric optimization in DL, including the basic procedure of geometric optimization, various geometric optimizers, and some concepts of the Riemannian manifold. In addition, it investigates various applications of geometric optimization in different DL networks for diverse tasks and discusses typical public toolboxes that implement optimization on the manifold. This article also includes a performance comparison among different deep geometric optimization methods in image recognition scenarios. Finally, this article elaborates on future opportunities and challenges in this field.
- 24 January 2025
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 57 (5) , 1-37
Top Cited PapersOpen AccessA Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold
Deep Learning (DL) has achieved remarkable success in tackling complex Artificial Intelligence tasks. The standard training of neural networks employs backpropagation to compute gradients and utilizes various optimization algorithms in the Euclidean space ℛ n . However, this optimization process faces challenges, such as the local optimal issues and the problem of gradient vanishing and exploding. To address these problems, Riemannian optimization offers a powerful extension to solve optimization problems in deep learning. By incorporating the prior constraint structure and the metric information of the underlying geometric information, Riemannian optimization-based DL offers a more stable and reliable optimization process, as well as enhanced adaptability to complex data structures. This article presents a comprehensive survey of applying geometric optimization in DL, including the basic procedure of geometric optimization, various geometric optimizers, and some concepts of the Riemannian manifold. In addition, it investigates various applications of geometric optimization in different DL networks for diverse tasks and discusses typical public toolboxes that implement optimization on the manifold. This article also includes a performance comparison among different deep geometric optimization methods in image recognition scenarios. Finally, this article elaborates on future opportunities and challenges in this field.
- Open Access
Hyperbolic Contrastive Learning for Hierarchical 3D Point Cloud Embedding
Hyperbolic spaces allow for more efficient modeling of complex, hierarchical structures, which is particularly beneficial in tasks involving multi-modal data. Although hyperbolic geometries have been proven effective for language-image pre-training, their capabilities to unify language, image, and 3D Point Cloud modalities are under-explored. We extend the 3D Point Cloud modality in hyperbolic multi-modal contrastive pre-training. Additionally, we explore the entailment, modality gap, and alignment regularizers for learning hierarchical 3D embeddings and facilitating the transfer of knowledge from both Text and Image modalities. These regularizers enable the learning of intra-modal hierarchy within each modality and inter-modal hierarchy across text, 2D images, and 3D Point Clouds. Experimental results demonstrate that our proposed training strategy yields an outstanding 3D Point Cloud encoder, and the obtained 3D Point Cloud hierarchical embeddings significantly improve performance on various downstream tasks.
Open AccessHyperbolic Contrastive Learning for Hierarchical 3D Point Cloud Embedding
Hyperbolic spaces allow for more efficient modeling of complex, hierarchical structures, which is particularly beneficial in tasks involving multi-modal data. Although hyperbolic geometries have been proven effective for language-image pre-training, their capabilities to unify language, image, and 3D Point Cloud modalities are under-explored. We extend the 3D Point Cloud modality in hyperbolic multi-modal contrastive pre-training. Additionally, we explore the entailment, modality gap, and alignment regularizers for learning hierarchical 3D embeddings and facilitating the transfer of knowledge from both Text and Image modalities. These regularizers enable the learning of intra-modal hierarchy within each modality and inter-modal hierarchy across text, 2D images, and 3D Point Clouds. Experimental results demonstrate that our proposed training strategy yields an outstanding 3D Point Cloud encoder, and the obtained 3D Point Cloud hierarchical embeddings significantly improve performance on various downstream tasks.
- 18 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (7) , 12483-12496
Integrating Convolution and Sparse Coding for Learning Low-Dimensional Discriminative Image Representations
This work investigates the problem of efficiently learning discriminative low-dimensional (LD) representations of multiclass image objects. We propose a generic end-to-end approach that jointly optimizes sparse dictionary and convolutions for learning LOW-dimensional discriminative image representations, named SparConvLow, taking advantage of c onvolutional n eural n etworks ( CNN s), dictionary learning, and orthogonal projections. The whole learning process can be summarized as follows. First, a CNN module is employed to extract high-dimensional (HD) preliminary convolutional features. Second, to avoid the high computational cost of direct sparse coding on HD CNN features, we learn sparse representation (SR) over a task-driven dictionary in the space with the feature being orthogonally projected. We then exploit the discriminative projection on SR. The whole learning process is consistently treated as an end-to-end joint optimization problem of trace quotient maximization. The cost function is well-defined on the product of the CNN parameters space, the Stiefel manifold, the Oblique manifold, and the Grassmann manifold. By using the explicit gradient delivery, the cost function is optimized via a geometrical stochastic gradient descent (SGD) algorithm along with the chain rule and the backpropagation. The experimental results show that the proposed method can achieve a highly competitive performance with the state-of-the-art (SOTA) image classification, object categorization, and face recognition methods, under both supervised and semi-supervised settings. The code is available at https://github.com/MVPR-Group/SparConvLow.
- 18 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (7) , 12483-12496
Integrating Convolution and Sparse Coding for Learning Low-Dimensional Discriminative Image Representations
This work investigates the problem of efficiently learning discriminative low-dimensional (LD) representations of multiclass image objects. We propose a generic end-to-end approach that jointly optimizes sparse dictionary and convolutions for learning LOW-dimensional discriminative image representations, named SparConvLow, taking advantage of c onvolutional n eural n etworks ( CNN s), dictionary learning, and orthogonal projections. The whole learning process can be summarized as follows. First, a CNN module is employed to extract high-dimensional (HD) preliminary convolutional features. Second, to avoid the high computational cost of direct sparse coding on HD CNN features, we learn sparse representation (SR) over a task-driven dictionary in the space with the feature being orthogonally projected. We then exploit the discriminative projection on SR. The whole learning process is consistently treated as an end-to-end joint optimization problem of trace quotient maximization. The cost function is well-defined on the product of the CNN parameters space, the Stiefel manifold, the Oblique manifold, and the Grassmann manifold. By using the explicit gradient delivery, the cost function is optimized via a geometrical stochastic gradient descent (SGD) algorithm along with the chain rule and the backpropagation. The experimental results show that the proposed method can achieve a highly competitive performance with the state-of-the-art (SOTA) image classification, object categorization, and face recognition methods, under both supervised and semi-supervised settings. The code is available at https://github.com/MVPR-Group/SparConvLow.