Weizhi Lu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Weizhi Lu has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
Mingrui Chen
School of Control Science and Engineering, Shandong University, Jinan, China
5 shared publicationsWeiyu Li
4 shared publications- Wei Zhang
Shandong University
3 shared publications - Weiyu Li
Zhongtai Securities Institute for Financial Studies, Shandong University, Jinan, People’s Republic of China
3 shared publications Kai Guo
2 shared publications- Kai Guo
School of Control Science and Engineering, Shandong University, Jinan, China
2 shared publications
Latest Publications
- 23 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (7) , 9610-9620
Collaborative Dictionary Learning for Compressed Sensing
In this article, we study the multiple dictionaries-based compressed sensing, which has sparse signals recovered with several different dictionaries, and then takes the average as the final recovery. Statistically, the averaging method will reduce the errors with zero mean, but can hardly deal with the errors with nonzero means. In practice, the nonzero-mean errors are usually inevitable since sparse recovery will drop relatively small elements. To reduce such kind of errors, we analytically show that the error distributions of different dictionaries should be different to each other. Interestingly, the condition is hard to achieve, even though the dictionaries are learned independently. This is because the dictionaries, which are trained on the same dataset tend to converge to close points and, thus, yield similar recovery errors. In an attempt to address the issue, we propose a collaborative dictionary learning model to enhance the difference between dictionaries. The model is implemented with a block coordinate descent algorithm, with guaranteed convergence. Experiments show that our collaboratively learned dictionaries perform better than the independently learned ones.
- 23 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (7) , 9610-9620
Collaborative Dictionary Learning for Compressed Sensing
In this article, we study the multiple dictionaries-based compressed sensing, which has sparse signals recovered with several different dictionaries, and then takes the average as the final recovery. Statistically, the averaging method will reduce the errors with zero mean, but can hardly deal with the errors with nonzero means. In practice, the nonzero-mean errors are usually inevitable since sparse recovery will drop relatively small elements. To reduce such kind of errors, we analytically show that the error distributions of different dictionaries should be different to each other. Interestingly, the condition is hard to achieve, even though the dictionaries are learned independently. This is because the dictionaries, which are trained on the same dataset tend to converge to close points and, thus, yield similar recovery errors. In an attempt to address the issue, we propose a collaborative dictionary learning model to enhance the difference between dictionaries. The model is implemented with a block coordinate descent algorithm, with guaranteed convergence. Experiments show that our collaboratively learned dictionaries perform better than the independently learned ones.
- 31 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 30 (10709908) , 364-368
Cascaded Compressed Sensing Networks
For deep networks with complex nonlinearity, the structure analysis and design remain challenging. In the letter, we propose to understand and build deep networks as a cascade of compressed sensing. Each compressed sensing module consists of two layers, corresponding to the two data transforms involved in compressed sensing, namely the sparse transform over dictionaries and the measurement over random projection. The two transforms can be viewed as a process of feature generation and extraction, thus amenable to network construction. The proposed construction has two advantages. First, it can be analyzed with compressed sensing theory. Second, it enables networks to be learned layer by layer by back-propagating the target via compressed sensing, and empirically the layerwise learning method tends to outperform the conventional error back-propagation method in the presence of noise.
- 31 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 30 (10709908) , 364-368
Cascaded Compressed Sensing Networks
For deep networks with complex nonlinearity, the structure analysis and design remain challenging. In the letter, we propose to understand and build deep networks as a cascade of compressed sensing. Each compressed sensing module consists of two layers, corresponding to the two data transforms involved in compressed sensing, namely the sparse transform over dictionaries and the measurement over random projection. The two transforms can be viewed as a process of feature generation and extraction, thus amenable to network construction. The proposed construction has two advantages. First, it can be analyzed with compressed sensing theory. Second, it enables networks to be learned layer by layer by back-propagating the target via compressed sensing, and empirically the layerwise learning method tends to outperform the conventional error back-propagation method in the presence of noise.
- Open Access
- 24 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Quantization: Is It Possible to Improve Classification?
Large-scale classification poses great challenges to storage and computation. There are two major solutions to address the problem: data compression and quantization. In the paper, we study the method of first reducing data dimension by random projection and then quantizing the projections to ternary and binary codes, which has been widely applied in practice. Often, the extreme quantization would degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could result in performance improvement, rather than degradation, if the data for quantization are preprocessed by sparse transform. Also, the quantization gain could be obtained with the random projections of the data, if both the data and random projection matrices are sparse enough, such that the resulting projections remain sparse. The intriguing performance is verified and analyzed with extensive experiments.
- 24 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessQuantization: Is It Possible to Improve Classification?
Large-scale classification poses great challenges to storage and computation. There are two major solutions to address the problem: data compression and quantization. In the paper, we study the method of first reducing data dimension by random projection and then quantizing the projections to ternary and binary codes, which has been widely applied in practice. Often, the extreme quantization would degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could result in performance improvement, rather than degradation, if the data for quantization are preprocessed by sparse transform. Also, the quantization gain could be obtained with the random projections of the data, if both the data and random projection matrices are sparse enough, such that the resulting projections remain sparse. The intriguing performance is verified and analyzed with extensive experiments.
- Open Access
- 6 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Deep Learning to Ternary Hash Codes by Continuation
Recently, it has been observed that $\{0,\pm1\}$-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform $\{-1, 1\}$-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes.
- 6 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessDeep Learning to Ternary Hash Codes by Continuation
Recently, it has been observed that $\{0,\pm1\}$-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform $\{-1, 1\}$-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes.
- Open Access
- 25 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Cascaded Compressed Sensing Networks: A Reversible Architecture for Layerwise Learning
Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The target propagation is an ill-posed problem, due to involving the inversion of nonlinear activations from low-dimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it leads to higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network’s each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.
- 25 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessCascaded Compressed Sensing Networks: A Reversible Architecture for Layerwise Learning
Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The target propagation is an ill-posed problem, due to involving the inversion of nonlinear activations from low-dimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it leads to higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network’s each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.
- Open Access
- 17 September 2021
- Published by Institution of Engineering and Technology (IET) in Electronics Letters
- Vol. 57 (24) , 925-926
Deep learning to ternary hash codes by continuation
Recently, ithas been observed that -ternary codes, which are simply generated from deep features by hard thresholding, tend to outperform -binary codes in image retrieval. To obtain better ternary codes, the authors for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes. For the first time, the authors propose to generate ternary hash codes by jointly learning the codes with deep features via a continuation method. Experiments show that the proposed method outperforms existing methods.- 17 September 2021
- Published by Institution of Engineering and Technology (IET) in Electronics Letters
- Vol. 57 (24) , 925-926
Open AccessDeep learning to ternary hash codes by continuation
Recently, ithas been observed that -ternary codes, which are simply generated from deep features by hard thresholding, tend to outperform -binary codes in image retrieval. To obtain better ternary codes, the authors for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes. For the first time, the authors propose to generate ternary hash codes by jointly learning the codes with deep features via a continuation method. Experiments show that the proposed method outperforms existing methods. - 24 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 29 (10577149) , 3365-3373
A Multi-Scale Spatial-Temporal Attention Model for Person Re-Identification in Videos
In this paper, we propose a novel deep neural network based attention model to learn the representative local regions from a video sequence for person re-identification. Specifically, we propose a multi-scale spatial-temporal attention (MSTA) model to measure the regions of each frame in different scales from the perspective of whole video sequence. Compared to traditional temporal attention models, MSTA focuses on exploiting the importance of local regions of each frame to the whole video representation in both spatial and temporal domains. A new training strategy is designed for the proposed model by incorporating the image-to-image mode with the videoto- video mode. Extensive experiments on benchmark datasets demonstrate the superiority of the proposed model over state-ofthe- art methods.
- 24 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 29 (10577149) , 3365-3373
A Multi-Scale Spatial-Temporal Attention Model for Person Re-Identification in Videos
In this paper, we propose a novel deep neural network based attention model to learn the representative local regions from a video sequence for person re-identification. Specifically, we propose a multi-scale spatial-temporal attention (MSTA) model to measure the regions of each frame in different scales from the perspective of whole video sequence. Compared to traditional temporal attention models, MSTA focuses on exploiting the importance of local regions of each frame to the whole video representation in both spatial and temporal domains. A new training strategy is designed for the proposed model by incorporating the image-to-image mode with the videoto- video mode. Extensive experiments on benchmark datasets demonstrate the superiority of the proposed model over state-ofthe- art methods.
- 16 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 22 (2) , 515-523
Exploring Discriminative Representations for Image Emotion Recognition With CNNs
Image emotion recognition aims to automatically categorize the emotion conveyed by an image. The potential of deep representation has been demonstrated in recent research on image emotion recognition. To better understand how CNNs work in emotion recognition, we investigate the deep features by visualizing them in this work. The study shows that the deep models mainly rely on the image content but miss the image style information such as color, texture and shapes that are low-level visual features but are vital for evoking emotions. To form a more discriminative representation for emotion recognition, we propose a novel CNN model that learns and integrates the content information from the high layers of the deep network with the style information from the lower layers. The uncertainty of image emotion labels is also investigated in this paper. Rather than using the emotion labels for training directly, as in previous work, a new loss function is designed by including the emotion labeling quality to optimize the proposed inference model. Extensive experiments on benchmark datasets are conducted to demonstrate the superiority of the proposed representation.
- 16 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 22 (2) , 515-523
Exploring Discriminative Representations for Image Emotion Recognition With CNNs
Image emotion recognition aims to automatically categorize the emotion conveyed by an image. The potential of deep representation has been demonstrated in recent research on image emotion recognition. To better understand how CNNs work in emotion recognition, we investigate the deep features by visualizing them in this work. The study shows that the deep models mainly rely on the image content but miss the image style information such as color, texture and shapes that are low-level visual features but are vital for evoking emotions. To form a more discriminative representation for emotion recognition, we propose a novel CNN model that learns and integrates the content information from the high layers of the deep network with the style information from the lower layers. The uncertainty of image emotion labels is also investigated in this paper. Rather than using the emotion labels for training directly, as in previous work, a new loss function is designed by including the emotion labeling quality to optimize the proposed inference model. Extensive experiments on benchmark datasets are conducted to demonstrate the superiority of the proposed representation.
- 21 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal and Information Processing over Networks
- Vol. 5 (3) , 418-427
Expander Recovery Performance of Bipartite Graphs With Girth Greater Than 4
Expander recovery is an iterative algorithm de- signed to recover sparse signals measured with 0-1 binary matrices with linear complexity. In the paper, we study the expander recovery performance of the bipartite graph with girth greater than 4, which can be associated with a binary matrix with column correlations equal to either 0 or 1. For such graph, expander recovery is proved to achieve the same performance as the traditional basis pursuit recovery, as the signal is dissociated. Compared to random graphs widely used for expander recovery, the graph we study tends to present better empirical performance. Furthermore, its special structure enables reducing the iteration number of expander recovery from O(nlogk) times to exactly k times in serial recovery, and from O(logk) times to exactly one time in parallel recovery.
- 21 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal and Information Processing over Networks
- Vol. 5 (3) , 418-427
Expander Recovery Performance of Bipartite Graphs With Girth Greater Than 4
Expander recovery is an iterative algorithm de- signed to recover sparse signals measured with 0-1 binary matrices with linear complexity. In the paper, we study the expander recovery performance of the bipartite graph with girth greater than 4, which can be associated with a binary matrix with column correlations equal to either 0 or 1. For such graph, expander recovery is proved to achieve the same performance as the traditional basis pursuit recovery, as the signal is dissociated. Compared to random graphs widely used for expander recovery, the graph we study tends to present better empirical performance. Furthermore, its special structure enables reducing the iteration number of expander recovery from O(nlogk) times to exactly k times in serial recovery, and from O(logk) times to exactly one time in parallel recovery.