Wenjia Niu
Research Subjects
Wenjia Niu has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Kewen Xia
School of Electronic and Information Engineering, Hebei University of Technology, Tianjin, China
3 shared publications Dylan Zhang
Quriosity Pty Ltd
2 shared publicationsQixuan Zhang
Australian National University
2 shared publicationsSabrina Caldwell
Australian National University
2 shared publications- Tom Gedeon
Curtin University, Perth, Australia
2 shared publications - Wenhan Luo
The Hong Kong University of Science and Technology
2 shared publications
Latest Publications
- Open Access
Visual Prompting in LLMs for Enhancing Emotion Recognition
Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remains largely unexplored and untapped. Traditional methods in VLLMs struggle with spatial localization and often discard valuable global context. To address this problem, we propose a Set-of-Vision prompting (SoV) approach that enhances zero-shot emotion recognition by using spatial information, such as bounding boxes and facial landmarks, to mark targets precisely. SoV improves accuracy in face count and emotion categorization while preserving the enriched image context. Through a battery of experimentation and analysis of recent commercial or open-source VLLMs, we evaluate the SoV model's ability to comprehend facial expressions in natural environments. Our findings demonstrate the effectiveness of integrating spatial visual prompts into VLLMs for improving emotion recognition performance.
Open AccessVisual Prompting in LLMs for Enhancing Emotion Recognition
Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remains largely unexplored and untapped. Traditional methods in VLLMs struggle with spatial localization and often discard valuable global context. To address this problem, we propose a Set-of-Vision prompting (SoV) approach that enhances zero-shot emotion recognition by using spatial information, such as bounding boxes and facial landmarks, to mark targets precisely. SoV improves accuracy in face count and emotion categorization while preserving the enriched image context. Through a battery of experimentation and analysis of recent commercial or open-source VLLMs, we evaluate the SoV model's ability to comprehend facial expressions in natural environments. Our findings demonstrate the effectiveness of integrating spatial visual prompts into VLLMs for improving emotion recognition performance.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 4484-4499
Visual Prompting in LLMs for Enhancing Emotion Recognition
Qixuan Zhang, Zhifeng Wang, Dylan Zhang, Wenjia Niu, Sabrina Caldwell, Tom Gedeon, Yang Liu, Zhenyue Qin. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 4484-4499
Open AccessVisual Prompting in LLMs for Enhancing Emotion Recognition
Qixuan Zhang, Zhifeng Wang, Dylan Zhang, Wenjia Niu, Sabrina Caldwell, Tom Gedeon, Yang Liu, Zhenyue Qin. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.
- 22 June 2022
- Published by Elsevier in Knowledge-Based Systems
- Vol. 251
Four-player GroupGAN for weak expression recognition via latent expression magnification
- 22 June 2022
- Published by Elsevier in Knowledge-Based Systems
- Vol. 251
Four-player GroupGAN for weak expression recognition via latent expression magnification
- Open Access
Blind Motion Deblurring Super-Resolution: When Dynamic Spatio-Temporal Learning Meets Static Image Understanding
Single-image super-resolution (SR) and multi-frame SR are two ways to super resolve low-resolution images. Single-Image SR generally handles each image independently, but ignores the temporal information implied in continuing frames. Multi-frame SR is able to model the temporal dependency via capturing motion information. However, it relies on neighbouring frames which are not always available in the real world. Meanwhile, slight camera shake easily causes heavy motion blur on long-distance-shot low-resolution images. To address these problems, a Blind Motion Deblurring Super-Reslution Networks, BMDSRNet, is proposed to learn dynamic spatio-temporal information from single static motion-blurred images. Motion-blurred images are the accumulation over time during the exposure of cameras, while the proposed BMDSRNet learns the reverse process and uses three-streams to learn Bidirectional spatio-temporal information based on well designed reconstruction loss functions to recover clean high-resolution images. Extensive experiments demonstrate that the proposed BMDSRNet outperforms recent state-of-the-art methods, and has the ability to simultaneously deal with image deblurring and SR.
Open AccessBlind Motion Deblurring Super-Resolution: When Dynamic Spatio-Temporal Learning Meets Static Image Understanding
Single-image super-resolution (SR) and multi-frame SR are two ways to super resolve low-resolution images. Single-Image SR generally handles each image independently, but ignores the temporal information implied in continuing frames. Multi-frame SR is able to model the temporal dependency via capturing motion information. However, it relies on neighbouring frames which are not always available in the real world. Meanwhile, slight camera shake easily causes heavy motion blur on long-distance-shot low-resolution images. To address these problems, a Blind Motion Deblurring Super-Reslution Networks, BMDSRNet, is proposed to learn dynamic spatio-temporal information from single static motion-blurred images. Motion-blurred images are the accumulation over time during the exposure of cameras, while the proposed BMDSRNet learns the reverse process and uses three-streams to learn Bidirectional spatio-temporal information based on well designed reconstruction loss functions to recover clean high-resolution images. Extensive experiments demonstrate that the proposed BMDSRNet outperforms recent state-of-the-art methods, and has the ability to simultaneously deal with image deblurring and SR.
- Open Access
Contiguous Loss for Motion-Based, Non-Aligned Image Deblurring
In general dynamic scenes, blurring is the result of the motion of multiple objects, camera shaking or scene depth variations. As an inverse process, deblurring extracts a sharp video sequence from the information contained in one single blurry image—it is itself an ill-posed computer vision problem. To reconstruct these sharp frames, traditional methods aim to build several convolutional neural networks (CNN) to generate different frames, resulting in expensive computation. To vanquish this problem, an innovative framework which can generate several sharp frames based on one CNN model is proposed. The motion-based image is put into our framework and the spatio-temporal information is encoded via several convolutional and pooling layers, and the output of our model is several sharp frames. Moreover, a blurry image does not have one-to-one correspondence with any sharp video sequence, since different video sequences can create similar blurry images, so neither the traditional pixel2pixel nor perceptual loss is suitable for focusing on non-aligned data. To alleviate this problem and model the blurring process, a novel contiguous blurry loss function is proposed which focuses on measuring the loss of non-aligned data. Experimental results show that the proposed model combined with the contiguous blurry loss can generate sharp video sequences efficiently and perform better than state-of-the-art methods.
Open AccessContiguous Loss for Motion-Based, Non-Aligned Image Deblurring
In general dynamic scenes, blurring is the result of the motion of multiple objects, camera shaking or scene depth variations. As an inverse process, deblurring extracts a sharp video sequence from the information contained in one single blurry image—it is itself an ill-posed computer vision problem. To reconstruct these sharp frames, traditional methods aim to build several convolutional neural networks (CNN) to generate different frames, resulting in expensive computation. To vanquish this problem, an innovative framework which can generate several sharp frames based on one CNN model is proposed. The motion-based image is put into our framework and the spatio-temporal information is encoded via several convolutional and pooling layers, and the output of our model is several sharp frames. Moreover, a blurry image does not have one-to-one correspondence with any sharp video sequence, since different video sequences can create similar blurry images, so neither the traditional pixel2pixel nor perceptual loss is suitable for focusing on non-aligned data. To alleviate this problem and model the blurring process, a novel contiguous blurry loss function is proposed which focuses on measuring the loss of non-aligned data. Experimental results show that the proposed model combined with the contiguous blurry loss can generate sharp video sequences efficiently and perform better than state-of-the-art methods.
- Open Access
- 8 February 2021
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2021, 1-19
Semisupervised SVM by Hybrid Whale Optimization Algorithm and Its Application in Oil Layer Recognition
In many fields, such as oil logging, it is expensive to obtain labeled data, and a large amount of inexpensive unlabeled data are not used. Therefore, it is necessary to use semisupervised learning to obtain accurate classification with limited labeled data and many unlabeled data. The semisupervised support vector machine (S3VM) is the most useful method in semisupervised learning. Nevertheless, S3VM model performance will degrade when the sample number of categories is not even or have lots of unlabeled samples. Thus, a new semisupervised SVM by hybrid whale optimization algorithm (HWOA-S3VM) is proposed in this paper. Firstly, a tradeoff control parameter is added in S3VM to deal with an uneven sample of category which can cause S3VM to degrade. Then, a hybrid whale optimization algorithm (HWOA) is used to optimize the model parameters of S3VM to increase the classification accuracy. For HWOA improvement, an opposition-based cubic mapping is used to initialize the WOA population to improve the convergence speed, and the catfish effect is used to help WOA jump out of the local optimum and obtain the global optimization ability. In the experiments, firstly, the HWOA is tested by 12 classic benchmark functions of CEC2005 and four functions of CEC2014 compared with the other five algorithms. Then, six UCI datasets are used to test the performance of HWOA-S3VM and compared with the other four algorithms. Finally, we applied HWOA-S3VM to perform oil layer recognition of three oil well datasets. These experimental results show that (1) HWOA has a higher convergence speed and better global searchability than other algorithms. (2) HWOA-S3VM model has higher classification accuracy on UCI datasets than other algorithms when combined, labeled, and unlabeled data are used as the training dataset. (3) The recognition accuracy and speed of the HWOA-S3VM model are superior to the other four algorithms when applied in oil layer recognition.
- 8 February 2021
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2021, 1-19
Open AccessSemisupervised SVM by Hybrid Whale Optimization Algorithm and Its Application in Oil Layer Recognition
In many fields, such as oil logging, it is expensive to obtain labeled data, and a large amount of inexpensive unlabeled data are not used. Therefore, it is necessary to use semisupervised learning to obtain accurate classification with limited labeled data and many unlabeled data. The semisupervised support vector machine (S3VM) is the most useful method in semisupervised learning. Nevertheless, S3VM model performance will degrade when the sample number of categories is not even or have lots of unlabeled samples. Thus, a new semisupervised SVM by hybrid whale optimization algorithm (HWOA-S3VM) is proposed in this paper. Firstly, a tradeoff control parameter is added in S3VM to deal with an uneven sample of category which can cause S3VM to degrade. Then, a hybrid whale optimization algorithm (HWOA) is used to optimize the model parameters of S3VM to increase the classification accuracy. For HWOA improvement, an opposition-based cubic mapping is used to initialize the WOA population to improve the convergence speed, and the catfish effect is used to help WOA jump out of the local optimum and obtain the global optimization ability. In the experiments, firstly, the HWOA is tested by 12 classic benchmark functions of CEC2005 and four functions of CEC2014 compared with the other five algorithms. Then, six UCI datasets are used to test the performance of HWOA-S3VM and compared with the other four algorithms. Finally, we applied HWOA-S3VM to perform oil layer recognition of three oil well datasets. These experimental results show that (1) HWOA has a higher convergence speed and better global searchability than other algorithms. (2) HWOA-S3VM model has higher classification accuracy on UCI datasets than other algorithms when combined, labeled, and unlabeled data are used as the training dataset. (3) The recognition accuracy and speed of the HWOA-S3VM model are superior to the other four algorithms when applied in oil layer recognition.
- Open Access
- 1 January 2017
- Published by Wiley in Computational Intelligence and Neuroscience
- Vol. 2017, 1-11
A Novel Graph Constructor for Semisupervised Discriminant Analysis: Combined Low-Rank and -Nearest Neighbor GraphSemisupervised Discriminant Analysis (SDA) is a semisupervised dimensionality reduction algorithm, which can easily resolve the out-of-sample problem. Relative works usually focus on the geometric relationships of data points, which are not obvious, to enhance the performance of SDA. Different from these relative works, the regularized graph construction is researched here, which is important in the graph-based semisupervised learning methods. In this paper, we propose a novel graph for Semisupervised Discriminant Analysis, which is called combined low-rank and -nearest neighbor (LRKNN) graph. In our LRKNN graph, we map the data to the LR feature space and then the is adopted to satisfy the algorithmic requirements of SDA. Since the low-rank representation can capture the global structure and the -nearest neighbor algorithm can maximally preserve the local geometrical structure of the data, the LRKNN graph can significantly improve the performance of SDA. Extensive experiments on several real-world databases show that the proposed LRKNN graph is an efficient graph constructor, which can largely outperform other commonly used baselines.- 1 January 2017
- Published by Wiley in Computational Intelligence and Neuroscience
- Vol. 2017, 1-11
Open AccessA Novel Graph Constructor for Semisupervised Discriminant Analysis: Combined Low-Rank and -Nearest Neighbor GraphSemisupervised Discriminant Analysis (SDA) is a semisupervised dimensionality reduction algorithm, which can easily resolve the out-of-sample problem. Relative works usually focus on the geometric relationships of data points, which are not obvious, to enhance the performance of SDA. Different from these relative works, the regularized graph construction is researched here, which is important in the graph-based semisupervised learning methods. In this paper, we propose a novel graph for Semisupervised Discriminant Analysis, which is called combined low-rank and -nearest neighbor (LRKNN) graph. In our LRKNN graph, we map the data to the LR feature space and then the is adopted to satisfy the algorithmic requirements of SDA. Since the low-rank representation can capture the global structure and the -nearest neighbor algorithm can maximally preserve the local geometrical structure of the data, the LRKNN graph can significantly improve the performance of SDA. Extensive experiments on several real-world databases show that the proposed LRKNN graph is an efficient graph constructor, which can largely outperform other commonly used baselines. - 2 July 2015
- Published by Springer Nature in Knowledge and Information Systems
- Vol. 47 (3) , 647-669
A differentially private algorithm for location data release
- 2 July 2015
- Published by Springer Nature in Knowledge and Information Systems
- Vol. 47 (3) , 647-669
A differentially private algorithm for location data release