Mingyu Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mingyu Yang has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Hun-Seok Kim
University of Michigan
6 shared publications Hun-Seok Kim
University of Michigan
3 shared publications- Yu Chen
EECS Department, University of Michigan, Ann Arbor, MI, USA
3 shared publications - Bowen Liu
Department of Electrical and Computer Engineering, University of Michigan, Ann Arbor, MI, USA
2 shared publications Boyang Wang
Department of Electrical and Computer Engineering, University of Michigan, Ann Arbor, MI, USA
2 shared publications- Chenghong Bian
Department of Electrical and Electronic Engineering, Imperial College London, London, U.K.
1 shared publication
Latest Publications
- Open Access
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
HyperLiDAR: Adaptive Post-Deployment LiDAR Segmentation via Hyperdimensional Computing
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessHyperLiDAR: Adaptive Post-Deployment LiDAR Segmentation via Hyperdimensional Computing
- Open Access
- 3 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Machine Learning in Communications and Networking
- Vol. 3, 1227-1243
Diffusion-Aided Joint Source Channel Coding for High Realism Wireless Image Transmission
Deep learning-based joint source-channel coding (deep JSCC) has been demonstrated to be an effective approach for wireless image transmission. However, many current approaches utilize an autoencoder framework to optimize conventional metrics such as Mean Squared Error (MSE) and Structural Similarity Index (SSIM), which are inadequate for preserving the perceptual quality of reconstructed images. Such an issue is more prominent under stringent bandwidth constraints or low signal-to-noise ratio (SNR) conditions. To tackle this challenge, we propose DiffJSCC, a novel framework that leverages the prior knowledge of the pre-trained Stable Diffusion model to produce high-realism images via the conditional diffusion denoising process. First, our DiffJSCC employs an autoencoder structure similar to prior deep JSCC works to generate an initial image reconstruction from the noisy channel symbols. This preliminary reconstruction serves as an intermediate step where robust multimodal spatial and textual features are extracted. In the following diffusion step, DiffJSCC uses the derived multimodal features, together with channel state information such as the signal-to-noise ratio (SNR) and channel gain, to guide the diffusion denoising process through a novel control module. To maintain the balance between realism and fidelity, an optional intermediate guidance approach using the initial image reconstruction is implemented. Extensive experiments on diverse datasets reveal that our method significantly surpasses prior deep JSCC approaches on both perceptual metrics and downstream task performance, showcasing its ability to preserve the semantics of the original transmitted images. Notably, DiffJSCC can achieve highly realistic reconstructions for 768×512 pixel Kodak images with only 3072 symbols (<0.008 symbols per pixel) under 1dB SNR channels. The source code is available at https://github.com/mingyuyng/DiffJSCC.
- 3 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Machine Learning in Communications and Networking
- Vol. 3, 1227-1243
Open AccessDiffusion-Aided Joint Source Channel Coding for High Realism Wireless Image Transmission
Deep learning-based joint source-channel coding (deep JSCC) has been demonstrated to be an effective approach for wireless image transmission. However, many current approaches utilize an autoencoder framework to optimize conventional metrics such as Mean Squared Error (MSE) and Structural Similarity Index (SSIM), which are inadequate for preserving the perceptual quality of reconstructed images. Such an issue is more prominent under stringent bandwidth constraints or low signal-to-noise ratio (SNR) conditions. To tackle this challenge, we propose DiffJSCC, a novel framework that leverages the prior knowledge of the pre-trained Stable Diffusion model to produce high-realism images via the conditional diffusion denoising process. First, our DiffJSCC employs an autoencoder structure similar to prior deep JSCC works to generate an initial image reconstruction from the noisy channel symbols. This preliminary reconstruction serves as an intermediate step where robust multimodal spatial and textual features are extracted. In the following diffusion step, DiffJSCC uses the derived multimodal features, together with channel state information such as the signal-to-noise ratio (SNR) and channel gain, to guide the diffusion denoising process through a novel control module. To maintain the balance between realism and fidelity, an optional intermediate guidance approach using the initial image reconstruction is implemented. Extensive experiments on diverse datasets reveal that our method significantly surpasses prior deep JSCC approaches on both perceptual metrics and downstream task performance, showcasing its ability to preserve the semantics of the original transmitted images. Notably, DiffJSCC can achieve highly realistic reconstructions for 768×512 pixel Kodak images with only 3072 symbols (<0.008 symbols per pixel) under 1dB SNR channels. The source code is available at https://github.com/mingyuyng/DiffJSCC.
- 6 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (4) , 3054-3061
H-PCC: Point Cloud Compression With Hybrid Mode Selection and Content Adaptive Down-Sampling
LiDAR sensors are integral to autonomous driving and augmented reality applications, providing essential depth information. However, managing the substantial volume of LiDAR point cloud data is crucial for practical application, necessitating efficient compression algorithms. Similar to other data compression domains, point cloud compression aims to spatially and temporally decorrelate information while preserving reconstruction quality. Addressing real-world challenges such as sensor pose retrieval inaccuracies or motion estimation failures, we introduce H-PCC, a hybrid LiDAR data compression pipeline that combines a range image prediction-based dynamic compression method with a novel point clustering octree-based static method. Additionally, we demonstrate that our content adaptive down-sampling technique using associated camera images can significantly enhance the LiDAR data compression rate with minimal impact on the accuracy of machine perception tasks. Extensive evaluations on the KITTI and Oxford datasets, using both traditional perceptual quality metrics and machine vision tasks, show that H-PCC surpasses state-of-the-art LiDAR data compression methods.
- 6 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (4) , 3054-3061
H-PCC: Point Cloud Compression With Hybrid Mode Selection and Content Adaptive Down-Sampling
LiDAR sensors are integral to autonomous driving and augmented reality applications, providing essential depth information. However, managing the substantial volume of LiDAR point cloud data is crucial for practical application, necessitating efficient compression algorithms. Similar to other data compression domains, point cloud compression aims to spatially and temporally decorrelate information while preserving reconstruction quality. Addressing real-world challenges such as sensor pose retrieval inaccuracies or motion estimation failures, we introduce H-PCC, a hybrid LiDAR data compression pipeline that combines a range image prediction-based dynamic compression method with a novel point clustering octree-based static method. Additionally, we demonstrate that our content adaptive down-sampling technique using associated camera images can significantly enhance the LiDAR data compression rate with minimal impact on the accuracy of machine perception tasks. Extensive evaluations on the KITTI and Oxford datasets, using both traditional perceptual quality metrics and machine vision tasks, show that H-PCC surpasses state-of-the-art LiDAR data compression methods.
- Open Access
SAM-guided Pseudo Label Enhancement for Multi-modal 3D Semantic Segmentation
Multi-modal 3D semantic segmentation is vital for applications such as autonomous driving and virtual reality (VR). To effectively deploy these models in real-world scenarios, it is essential to employ cross-domain adaptation techniques that bridge the gap between training data and real-world data. Recently, self-training with pseudo-labels has emerged as a predominant method for cross-domain adaptation in multi-modal 3D semantic segmentation. However, generating reliable pseudo-labels necessitates stringent constraints, which often result in sparse pseudo-labels after pruning. This sparsity can potentially hinder performance improvement during the adaptation process. We propose an image-guided pseudo-label enhancement approach that leverages the complementary 2D prior knowledge from the Segment Anything Model (SAM) to introduce more reliable pseudo-labels, thereby boosting domain adaptation performance. Specifically, given a 3D point cloud and the SAM masks from its paired image data, we collect all 3D points covered by each SAM mask that potentially belong to the same object. Then our method refines the pseudo-labels within each SAM mask in two steps. First, we determine the class label for each mask using majority voting and employ various constraints to filter out unreliable mask labels. Next, we introduce Geometry-Aware Progressive Propagation (GAPP) which propagates the mask label to all 3D points within the SAM mask while avoiding outliers caused by 2D-3D misalignment. Experiments conducted across multiple datasets and domain adaptation scenarios demonstrate that our proposed method significantly increases the quantity of high-quality pseudo-labels and enhances the adaptation performance over baseline methods.
Open AccessSAM-guided Pseudo Label Enhancement for Multi-modal 3D Semantic Segmentation
Multi-modal 3D semantic segmentation is vital for applications such as autonomous driving and virtual reality (VR). To effectively deploy these models in real-world scenarios, it is essential to employ cross-domain adaptation techniques that bridge the gap between training data and real-world data. Recently, self-training with pseudo-labels has emerged as a predominant method for cross-domain adaptation in multi-modal 3D semantic segmentation. However, generating reliable pseudo-labels necessitates stringent constraints, which often result in sparse pseudo-labels after pruning. This sparsity can potentially hinder performance improvement during the adaptation process. We propose an image-guided pseudo-label enhancement approach that leverages the complementary 2D prior knowledge from the Segment Anything Model (SAM) to introduce more reliable pseudo-labels, thereby boosting domain adaptation performance. Specifically, given a 3D point cloud and the SAM masks from its paired image data, we collect all 3D points covered by each SAM mask that potentially belong to the same object. Then our method refines the pseudo-labels within each SAM mask in two steps. First, we determine the class label for each mask using majority voting and employ various constraints to filter out unreliable mask labels. Next, we introduce Geometry-Aware Progressive Propagation (GAPP) which propagates the mask label to all 3D points within the SAM mask while avoiding outliers caused by 2D-3D misalignment. Experiments conducted across multiple datasets and domain adaptation scenarios demonstrate that our proposed method significantly increases the quantity of high-quality pseudo-labels and enhances the adaptation performance over baseline methods.
- Open Access
Efficient Computation Sharing for Multi-Task Visual Scene Understanding
Solving multiple visual tasks using individual models can be resource-intensive, while multi-task learning can conserve resources by sharing knowledge across different tasks. Despite the benefits of multi-task learning, such techniques can struggle with balancing the loss for each task, leading to potential performance degradation. We present a novel computation- and parameter-sharing framework that balances efficiency and accuracy to perform multiple visual tasks utilizing individually-trained single-task transformers. Our method is motivated by transfer learning schemes to reduce computational and parameter storage costs while maintaining the desired performance. Our approach involves splitting the tasks into a base task and the other sub-tasks, and sharing a significant portion of activations and parameters/weights between the base and sub-tasks to decrease inter-task redundancies and enhance knowledge sharing. The evaluation conducted on NYUD-v2 and PASCAL-context datasets shows that our method is superior to the state-of-the-art transformer-based multi-task learning techniques with higher accuracy and reduced computational resources. Moreover, our method is extended to video stream inputs, further reducing computational costs by efficiently sharing information across the temporal domain as well as the task domain. Our codes and models will be publicly available.
Open AccessEfficient Computation Sharing for Multi-Task Visual Scene Understanding
Solving multiple visual tasks using individual models can be resource-intensive, while multi-task learning can conserve resources by sharing knowledge across different tasks. Despite the benefits of multi-task learning, such techniques can struggle with balancing the loss for each task, leading to potential performance degradation. We present a novel computation- and parameter-sharing framework that balances efficiency and accuracy to perform multiple visual tasks utilizing individually-trained single-task transformers. Our method is motivated by transfer learning schemes to reduce computational and parameter storage costs while maintaining the desired performance. Our approach involves splitting the tasks into a base task and the other sub-tasks, and sharing a significant portion of activations and parameters/weights between the base and sub-tasks to decrease inter-task redundancies and enhance knowledge sharing. The evaluation conducted on NYUD-v2 and PASCAL-context datasets shows that our method is superior to the state-of-the-art transformer-based multi-task learning techniques with higher accuracy and reduced computational resources. Moreover, our method is extended to video stream inputs, further reducing computational costs by efficiently sharing information across the temporal domain as well as the task domain. Our codes and models will be publicly available.
- 23 October 2022
- Published by Springer Nature
- p. 233-250
Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
- 23 October 2022
- Published by Springer Nature
- p. 233-250
Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
- Open Access
Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
In recent years, deep learning-based approaches for visual-inertial odometry (VIO) have shown remarkable performance outperforming traditional geometric methods. Yet, all existing methods use both the visual and inertial measurements for every pose estimation incurring potential computational redundancy. While visual data processing is much more expensive than that for the inertial measurement unit (IMU), it may not always contribute to improving the pose estimation accuracy. In this paper, we propose an adaptive deep-learning based VIO method that reduces computational redundancy by opportunistically disabling the visual modality. Specifically, we train a policy network that learns to deactivate the visual feature extractor on the fly based on the current motion state and IMU readings. A Gumbel-Softmax trick is adopted to train the policy network to make the decision process differentiable for end-to-end system training. The learned strategy is interpretable, and it shows scenario-dependent decision patterns for adaptive complexity reduction. Experiment results show that our method achieves a similar or even better performance than the full-modality baseline with up to 78.8% computational complexity reduction for KITTI dataset evaluation. The code is available at https://github.com/mingyuyng/Visual-Selective-VIO.
Open AccessEfficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
In recent years, deep learning-based approaches for visual-inertial odometry (VIO) have shown remarkable performance outperforming traditional geometric methods. Yet, all existing methods use both the visual and inertial measurements for every pose estimation incurring potential computational redundancy. While visual data processing is much more expensive than that for the inertial measurement unit (IMU), it may not always contribute to improving the pose estimation accuracy. In this paper, we propose an adaptive deep-learning based VIO method that reduces computational redundancy by opportunistically disabling the visual modality. Specifically, we train a policy network that learns to deactivate the visual feature extractor on the fly based on the current motion state and IMU readings. A Gumbel-Softmax trick is adopted to train the policy network to make the decision process differentiable for end-to-end system training. The learned strategy is interpretable, and it shows scenario-dependent decision patterns for adaptive complexity reduction. Experiment results show that our method achieves a similar or even better performance than the full-modality baseline with up to 78.8% computational complexity reduction for KITTI dataset evaluation. The code is available at https://github.com/mingyuyng/Visual-Selective-VIO.
- Top Cited Papers
- 16 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive Communications and Networking
- Vol. 8 (2) , 584-599
OFDM-Guided Deep Joint Source Channel Coding for Wireless Multipath Fading Channels
We investigate joint source channel coding (JSCC) for wireless image transmission over multipath fading channels. Inspired by recent works on deep learning based JSCC and model-based learning methods, we combine an autoencoder with orthogonal frequency division multiplexing (OFDM) to cope with multipath fading. The proposed encoder and decoder use convolutional neural networks (CNNs) and directly map the source images to complex-valued baseband samples for OFDM transmission. The multipath channel and OFDM are represented by non-trainable (deterministic) but differentiable layers so that the system can be trained end-to-end. Furthermore, our JSCC decoder further incorporates explicit channel estimation, equalization, and additional subnets to enhance the performance. The proposed method exhibits 2.5 – 4 dB SNR gain for the equivalent image quality compared to conventional schemes that employ state-of-the-art but separate source and channel coding such as Better Portable Graphics (BPG) and Low-Density Parity-Check (LDPC) schemes. The performance further improves when the system incorporates the channel state information (CSI) feedback. The proposed scheme is robust against OFDM signal clipping and parameter mismatch for the channel model used in training and evaluation.
- 16 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive Communications and Networking
- Vol. 8 (2) , 584-599
Top Cited PapersOFDM-Guided Deep Joint Source Channel Coding for Wireless Multipath Fading Channels
We investigate joint source channel coding (JSCC) for wireless image transmission over multipath fading channels. Inspired by recent works on deep learning based JSCC and model-based learning methods, we combine an autoencoder with orthogonal frequency division multiplexing (OFDM) to cope with multipath fading. The proposed encoder and decoder use convolutional neural networks (CNNs) and directly map the source images to complex-valued baseband samples for OFDM transmission. The multipath channel and OFDM are represented by non-trainable (deterministic) but differentiable layers so that the system can be trained end-to-end. Furthermore, our JSCC decoder further incorporates explicit channel estimation, equalization, and additional subnets to enhance the performance. The proposed method exhibits 2.5 – 4 dB SNR gain for the equivalent image quality compared to conventional schemes that employ state-of-the-art but separate source and channel coding such as Better Portable Graphics (BPG) and Low-Density Parity-Check (LDPC) schemes. The performance further improves when the system incorporates the channel state information (CSI) feedback. The proposed scheme is robust against OFDM signal clipping and parameter mismatch for the channel model used in training and evaluation.
- Open Access
Deep Learning Based Near-Orthogonal Superposition Code for Short Message Transmission
Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning based near-orthogonal superposition (NOS) coding scheme is proposed for reliable transmission of short messages in the additive white Gaussian noise (AWGN) channel for mMTC applications. Similar to recent hyper-dimensional modulation (HDM), the NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector for transmission. The NOS decoder first estimates the information vectors and then performs a cyclic redundancy check (CRC)-assisted K-best tree-search algorithm to further reduce the packet error rate. The proposed NOS encoder and decoder are deep neural networks (DNNs) jointly trained as an auto encoder and decoder pair to learn a new NOS coding scheme with near-orthogonal codewords. Simulation results show the proposed deep learning-based NOS scheme outperforms HDM and Polar code with CRC-aided list decoding for short(32-bit) message transmission.
Open AccessDeep Learning Based Near-Orthogonal Superposition Code for Short Message Transmission
Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning based near-orthogonal superposition (NOS) coding scheme is proposed for reliable transmission of short messages in the additive white Gaussian noise (AWGN) channel for mMTC applications. Similar to recent hyper-dimensional modulation (HDM), the NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector for transmission. The NOS decoder first estimates the information vectors and then performs a cyclic redundancy check (CRC)-assisted K-best tree-search algorithm to further reduce the packet error rate. The proposed NOS encoder and decoder are deep neural networks (DNNs) jointly trained as an auto encoder and decoder pair to learn a new NOS coding scheme with near-orthogonal codewords. Simulation results show the proposed deep learning-based NOS scheme outperforms HDM and Polar code with CRC-aided list decoding for short(32-bit) message transmission.