Jiayuan He
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Jiayuan He has a total of 68 co-authors. Here are the top co-authors by number of shared publications:
- William Shieh
Westlake University
11 shared publications An Li
Department of Electrical and Electronics Engineering, University of Melbourne, Melbourne, Australia
9 shared publicationsXi Chen
Department of Electrical and Electronics Engineering, The University of Melbourne, Melbourne, Vic., Australia
8 shared publications- Ke Wang
RMIT University
6 shared publications Di Che
Department of Electrical and Electronics Engineering, The University of Melbourne, Melbourne, Vic., Australia
5 shared publications- Jianzhong Qi
The University of Melbourne, Melbourne, VIC, Australia
5 shared publications
Latest Publications
- Open Access
- 19 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 155-159
On the Challenge of Generating Multivariate Time Series Data from Distributed Sensors in IoT-enabled Scenarios
- 19 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 155-159
Open AccessOn the Challenge of Generating Multivariate Time Series Data from Distributed Sensors in IoT-enabled Scenarios
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 447-455
Generating Multimodal Metaphorical Features for Meme Understanding
Understanding a meme is a challenging task, due to the metaphorical information contained in the meme that requires intricate interpretation to grasp its intended meaning fully. In previous works, attempts have been made to facilitate computational understanding of memes through introducing human-annotated metaphors as extra input features into machine learning models. However, these approaches mainly focus on formulating linguistic representation of a metaphor (extracted from the texts appearing in memes), while ignoring the connection between the metaphor and corresponding visual features (e.g., objects in meme images). In this paper, we argue that a more comprehensive understanding of memes can only be achieved through a joint modelling of both visual and linguistic features of memes. To this end, we propose an approach to generate Multimodal Metaphorical feature for Meme Classification, named MMMC. MMMC derives visual characteristics from linguistic attributes of metaphorical concepts, which more effectively convey the underlying metaphorical concept, leveraging a text-conditioned generative adversarial network. The linguistic and visual features are then integrated into a set of multimodal metaphorical features for classification purpose. We perform extensive experiments on a benchmark metaphorical meme dataset, MET-Meme. Experimental results show that MMMC significantly outperforms existing baselines on the task of emotion classification and intention detection. Our code and dataset are available at https://github.com/liaolianfoka/MMMC.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 447-455
Open AccessGenerating Multimodal Metaphorical Features for Meme Understanding
Understanding a meme is a challenging task, due to the metaphorical information contained in the meme that requires intricate interpretation to grasp its intended meaning fully. In previous works, attempts have been made to facilitate computational understanding of memes through introducing human-annotated metaphors as extra input features into machine learning models. However, these approaches mainly focus on formulating linguistic representation of a metaphor (extracted from the texts appearing in memes), while ignoring the connection between the metaphor and corresponding visual features (e.g., objects in meme images). In this paper, we argue that a more comprehensive understanding of memes can only be achieved through a joint modelling of both visual and linguistic features of memes. To this end, we propose an approach to generate Multimodal Metaphorical feature for Meme Classification, named MMMC. MMMC derives visual characteristics from linguistic attributes of metaphorical concepts, which more effectively convey the underlying metaphorical concept, leveraging a text-conditioned generative adversarial network. The linguistic and visual features are then integrated into a set of multimodal metaphorical features for classification purpose. We perform extensive experiments on a benchmark metaphorical meme dataset, MET-Meme. Experimental results show that MMMC significantly outperforms existing baselines on the task of emotion classification and intention detection. Our code and dataset are available at https://github.com/liaolianfoka/MMMC.
- Open Access
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 19-24
Generating Multivariate Synthetic Time Series Data for Absent Sensors from Correlated Sources
Missing sensor data in human activity recognition is an active field of research that is being targeted with generative models for synthetic data generation. In contrast to most previous approaches, we aim to generate data of a sensor exclusively from data available at sensors in different body locations. Particularly, we evaluate existing approaches proposed in the literature for their suitability in this scenario. In this paper, we focus on the prediction of acceleration data and generate machine learning models based on generative adversarial networks and trained using correlated data from sensors in different body positions to generate synthetic sensor data that can replace the missing data from a sensor in a specific body position. The accuracy of the generated synthetic data is evaluated using a classification model based on a convolutional neural network for human activity recognition.
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 19-24
Open AccessGenerating Multivariate Synthetic Time Series Data for Absent Sensors from Correlated Sources
Missing sensor data in human activity recognition is an active field of research that is being targeted with generative models for synthetic data generation. In contrast to most previous approaches, we aim to generate data of a sensor exclusively from data available at sensors in different body locations. Particularly, we evaluate existing approaches proposed in the literature for their suitability in this scenario. In this paper, we focus on the prediction of acceleration data and generate machine learning models based on generative adversarial networks and trained using correlated data from sensors in different body positions to generate synthetic sensor data that can replace the missing data from a sensor in a specific body position. The accuracy of the generated synthetic data is evaluated using a classification model based on a convolutional neural network for human activity recognition.
- 25 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Terahertz Science and Technology
- Vol. 14 (3) , 354-363
Frequency-Diverse Antenna With Convolutional Neural Networks for Direction-of-Arrival Estimation in Terahertz Communications
The IEEE 802.15.3d standard for point-to-point wireless terahertz communications is defined to support high-capacity channels. By nature, terahertz signal transmission requires line-of-sight propagation and terahertz communications operates within a challenging power budget limitation. Therefore, accurate and efficient direction of arrival (DoA) estimation for maximizing received power becomes paramount to achieve reliable terahertz communications. In this paper, we present a frequency-diverse antenna with a machine learning-based approach utilizing convolutional neural networks (CNNs) to estimate DoA in the terahertz communications band. The antenna is deliberately designed to break symmetry, generating quasi-random radiation patterns, while the CNN captures the relationship between the radiation patterns and their respective angles of arrival. Based on experiments, the DoA estimation results converge to a minimum validation mean squared error (MSE) of 3.9° and root mean squared error (RMSE) of 1.9°. The estimation efficacy is further substantiated by a consistent performance demonstrated across diverse scenarios, encompassing various obstacles and absorbers around the propagation path. The proposed DoA estimation method shows considerable advantages as a compact, integrable, cost-effective solution for practical terahertz communications.
- 25 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Terahertz Science and Technology
- Vol. 14 (3) , 354-363
Frequency-Diverse Antenna With Convolutional Neural Networks for Direction-of-Arrival Estimation in Terahertz Communications
The IEEE 802.15.3d standard for point-to-point wireless terahertz communications is defined to support high-capacity channels. By nature, terahertz signal transmission requires line-of-sight propagation and terahertz communications operates within a challenging power budget limitation. Therefore, accurate and efficient direction of arrival (DoA) estimation for maximizing received power becomes paramount to achieve reliable terahertz communications. In this paper, we present a frequency-diverse antenna with a machine learning-based approach utilizing convolutional neural networks (CNNs) to estimate DoA in the terahertz communications band. The antenna is deliberately designed to break symmetry, generating quasi-random radiation patterns, while the CNN captures the relationship between the radiation patterns and their respective angles of arrival. Based on experiments, the DoA estimation results converge to a minimum validation mean squared error (MSE) of 3.9° and root mean squared error (RMSE) of 1.9°. The estimation efficacy is further substantiated by a consistent performance demonstrated across diverse scenarios, encompassing various obstacles and absorbers around the propagation path. The proposed DoA estimation method shows considerable advantages as a compact, integrable, cost-effective solution for practical terahertz communications.
- Open Access
- 7 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 193-196
An Application Programming Interface for Android to support dedicated 5G network slicing
- 7 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 193-196
Open AccessAn Application Programming Interface for Android to support dedicated 5G network slicing
- 3 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (7) , 3047-3061
Focused Contrastive Loss for Classification With Pre-Trained Language Models
Contrastive learning, which learns data representations by contrasting similar and dissimilar instances, has achieved great success in various domains including natural language processing (NLP). Recently, it has been demonstrated that incorporating class labels into contrastive learning, i.e., supervised contrastive learning (SCL), can further enhance the quality of the learned data representations. Although several works have shown empirically that incorporating SCL into classification models leads to better performance, the mechanism of how SCL works for classification is less studied. In this paper, we first investigate how SCL facilitates the classifier learning, where we show that the contrastive region, i.e., the data instances involved in each contrasting operation, has a crucial link to the mechanism of SCL. We reveal that the vanilla SCL is suboptimal since its behavior can be altered by variances in class distributions. Based on this finding, we propose a Fo cused C ontrastive L oss (FoCL) for classification. Compared with SCL, FoCL defines a finer contrastive region, focusing on the data instances surrounding decision boundaries. We conduct extensive experiments on three NLP tasks: text classification, named entity recognition, and relation extraction. Experimental results show consistent and significant improvements of FoCL over strong baselines on various benchmark datasets, especially in few-shot scenarios.
- 3 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (7) , 3047-3061
Focused Contrastive Loss for Classification With Pre-Trained Language Models
Contrastive learning, which learns data representations by contrasting similar and dissimilar instances, has achieved great success in various domains including natural language processing (NLP). Recently, it has been demonstrated that incorporating class labels into contrastive learning, i.e., supervised contrastive learning (SCL), can further enhance the quality of the learned data representations. Although several works have shown empirically that incorporating SCL into classification models leads to better performance, the mechanism of how SCL works for classification is less studied. In this paper, we first investigate how SCL facilitates the classifier learning, where we show that the contrastive region, i.e., the data instances involved in each contrasting operation, has a crucial link to the mechanism of SCL. We reveal that the vanilla SCL is suboptimal since its behavior can be altered by variances in class distributions. Based on this finding, we propose a Fo cused C ontrastive L oss (FoCL) for classification. Compared with SCL, FoCL defines a finer contrastive region, focusing on the data instances surrounding decision boundaries. We conduct extensive experiments on three NLP tasks: text classification, named entity recognition, and relation extraction. Experimental results show consistent and significant improvements of FoCL over strong baselines on various benchmark datasets, especially in few-shot scenarios.
- Open Access
- 25 March 2021
- Published by Frontiers Media SA in Frontiers in Research Metrics and Analytics
- Vol. 6
ChEMU 2020: Natural Language Processing Methods Are Effective for Information Extraction From Chemical Patents
Chemical patents represent a valuable source of information about new chemical compounds, which is critical to the drug discovery process. Automated information extraction over chemical patents is, however, a challenging task due to the large volume of existing patents and the complex linguistic properties of chemical patents. The Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2020, part of the Conference and Labs of the Evaluation Forum 2020 (CLEF2020), was introduced to support the development of advanced text mining techniques for chemical patents. The ChEMU 2020 lab proposed two fundamental information extraction tasks focusing on chemical reaction processes described in chemical patents: (1)chemical named entity recognition, requiring identification of essential chemical entities and their roles in chemical reactions, as well as reaction conditions; and (2)event extraction, which aims at identification of event steps relating the entities involved in chemical reactions. The ChEMU 2020 lab received 37 team registrations and 46 runs. Overall, the performance of submissions for these tasks exceeded our expectations, with the top systems outperforming strong baselines. We further show the methods to be robust to variations in sampling of the test data. We provide a detailed overview of the ChEMU 2020 corpus and its annotation, showing that inter-annotator agreement is very strong. We also present the methods adopted by participants, provide a detailed analysis of their performance, and carefully consider the potential impact of data leakage on interpretation of the results. The ChEMU 2020 Lab has shown the viability of automated methods to support information extraction of key information in chemical patents.
- 25 March 2021
- Published by Frontiers Media SA in Frontiers in Research Metrics and Analytics
- Vol. 6
Open AccessChEMU 2020: Natural Language Processing Methods Are Effective for Information Extraction From Chemical Patents
Chemical patents represent a valuable source of information about new chemical compounds, which is critical to the drug discovery process. Automated information extraction over chemical patents is, however, a challenging task due to the large volume of existing patents and the complex linguistic properties of chemical patents. The Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2020, part of the Conference and Labs of the Evaluation Forum 2020 (CLEF2020), was introduced to support the development of advanced text mining techniques for chemical patents. The ChEMU 2020 lab proposed two fundamental information extraction tasks focusing on chemical reaction processes described in chemical patents: (1)chemical named entity recognition, requiring identification of essential chemical entities and their roles in chemical reactions, as well as reaction conditions; and (2)event extraction, which aims at identification of event steps relating the entities involved in chemical reactions. The ChEMU 2020 lab received 37 team registrations and 46 runs. Overall, the performance of submissions for these tasks exceeded our expectations, with the top systems outperforming strong baselines. We further show the methods to be robust to variations in sampling of the test data. We provide a detailed overview of the ChEMU 2020 corpus and its annotation, showing that inter-annotator agreement is very strong. We also present the methods adopted by participants, provide a detailed analysis of their performance, and carefully consider the potential impact of data leakage on interpretation of the results. The ChEMU 2020 Lab has shown the viability of automated methods to support information extraction of key information in chemical patents.
- Open Access
- 12 October 2020
- Published by American Chemical Society (ACS)
ChEMU shared task: chemical entity recognition and event extraction of chemical reactions from patents
- 12 October 2020
- Published by American Chemical Society (ACS)
Open AccessChEMU shared task: chemical entity recognition and event extraction of chemical reactions from patents
- 14 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Journal of Lightwave Technology
- Vol. 38 (17) , 4632-4640
Delay-Tolerant Indoor Optical Wireless Communication Systems Based on Attention-Augmented Recurrent Neural Network
The optical wireless communication (OWC) system has been widely studied as a promising candidate for indoor high-speed wireless communications. In particular, the spatial diversity scheme has shown to be effective to improve the performance of OWC systems. However, the delay amongst multiple channels in this scheme may result in severe inter-symbol-interference (ISI) and degradations of the system performance. Recent studies have shown that the symbol decision schemes based on recurrent neural networks (RNNs) can mitigate the impact of delays. However, their performances are limited when the channel delays are long. In this paper, we propose a delay-tolerant indoor OWC system, which utilizes an attention-augmented long-short term memory (ALSTM) RNN decision scheme to handle long channel delays. A 10 Gb/s repetition-coded indoor OWC system with the proposed ALSTM RNN decision scheme is experimentally demonstrated. Compared with traditional OWC systems which add cyclic prefix (CP) /zero-postfix (ZP) to combat the impact of channel delays, the proposed system does not reduce the effective data rate or throughput. Compared with the previous study which adopts the vanilla RNN decision scheme, the proposed ALSTM RNN decision scheme can better learn dependencies amongst received symbols with the help of attention mechanism, especially amongst non-neighboring symbols. Hence, it is more robust against the ISI induced by long channel delays. Experimental results show that compared with the previously studied vanilla RNN, over one-order-of-magnitude bit-error-rate improvement is achieved when the channel delay is more than 4.7-symbol-period.
- 14 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Journal of Lightwave Technology
- Vol. 38 (17) , 4632-4640
Delay-Tolerant Indoor Optical Wireless Communication Systems Based on Attention-Augmented Recurrent Neural Network
The optical wireless communication (OWC) system has been widely studied as a promising candidate for indoor high-speed wireless communications. In particular, the spatial diversity scheme has shown to be effective to improve the performance of OWC systems. However, the delay amongst multiple channels in this scheme may result in severe inter-symbol-interference (ISI) and degradations of the system performance. Recent studies have shown that the symbol decision schemes based on recurrent neural networks (RNNs) can mitigate the impact of delays. However, their performances are limited when the channel delays are long. In this paper, we propose a delay-tolerant indoor OWC system, which utilizes an attention-augmented long-short term memory (ALSTM) RNN decision scheme to handle long channel delays. A 10 Gb/s repetition-coded indoor OWC system with the proposed ALSTM RNN decision scheme is experimentally demonstrated. Compared with traditional OWC systems which add cyclic prefix (CP) /zero-postfix (ZP) to combat the impact of channel delays, the proposed system does not reduce the effective data rate or throughput. Compared with the previous study which adopts the vanilla RNN decision scheme, the proposed ALSTM RNN decision scheme can better learn dependencies amongst received symbols with the help of attention mechanism, especially amongst non-neighboring symbols. Hence, it is more robust against the ISI induced by long channel delays. Experimental results show that compared with the previously studied vanilla RNN, over one-order-of-magnitude bit-error-rate improvement is achieved when the channel delay is more than 4.7-symbol-period.
- 20 April 2020
- Published by Optica Publishing Group in Optics Express
- Vol. 28 (9) , 13384-13400
FPGA-based neural network accelerators for millimeter-wave radio-over-fiber systems
With rapidly developing high-speed wireless communications, the 60 GHz millimeter-wave (mm-wave) frequency range has attracted extensive interests, and radio-over-fiber (RoF) systems have been widely investigated as a promising solution to deliver mm-wave signals. Neural networks have been proposed and studied to improve the mm-wave RoF system performances at the receiver side by suppressing both linear and nonlinear impairments. However, previous studies of neural networks in mm-wave RoF systems all focus on the use of off-line processing with high-end GPUs or CPUs, which are not practical for low power-consumption, low-cost and limited computation platform applications. To solve this issue, in this paper we investigate neural network hardware accelerator implementations for mm-wave RoF systems for the first time using the field programmable gate array (FPGA), taking advantage of the low power consumption, parallel computation, and reconfigurablity features of FPGA. Both the convolutional neural network (CNN) and binary convolutional neural network (BCNN) hardware accelerators are demonstrated. In addition, to satisfy the low-latency requirement in mm-wave RoF systems and to enable the use of low-cost compact FPGA devices, a novel inner parallel computation optimization method for implementing CNN and BCNN on FPGA is proposed. It is shown that compared with the popular embedded processor (ARM Cortex A9) execution latency, the proposed FPGA-based hardware accelerator reduces the processing delay in mm-wave RoF systems by about 99.45% and 92.79% for CNN and BCNN, respectively. Compared with non-optimized FPGA implementations, results show that the proposed inner parallel computation method reduces the processing latency by about 44.93% and 45.85% for CNN and BCNN, respectively. In addition, compared with the GPU implementation, the latency of CNN implementation with the proposed optimization method is reduced by 85.49%, while the power consumption is reduced by 86.91%. Although the latency of BCNN implementation with the proposed optimization method is larger compared with the GPU implementation, the power consumption is reduced by 86.14%. The demonstrated FPGA-based neural network hardware accelerators provide a promising solution for mm-wave RoF systems.
- 20 April 2020
- Published by Optica Publishing Group in Optics Express
- Vol. 28 (9) , 13384-13400
FPGA-based neural network accelerators for millimeter-wave radio-over-fiber systems
With rapidly developing high-speed wireless communications, the 60 GHz millimeter-wave (mm-wave) frequency range has attracted extensive interests, and radio-over-fiber (RoF) systems have been widely investigated as a promising solution to deliver mm-wave signals. Neural networks have been proposed and studied to improve the mm-wave RoF system performances at the receiver side by suppressing both linear and nonlinear impairments. However, previous studies of neural networks in mm-wave RoF systems all focus on the use of off-line processing with high-end GPUs or CPUs, which are not practical for low power-consumption, low-cost and limited computation platform applications. To solve this issue, in this paper we investigate neural network hardware accelerator implementations for mm-wave RoF systems for the first time using the field programmable gate array (FPGA), taking advantage of the low power consumption, parallel computation, and reconfigurablity features of FPGA. Both the convolutional neural network (CNN) and binary convolutional neural network (BCNN) hardware accelerators are demonstrated. In addition, to satisfy the low-latency requirement in mm-wave RoF systems and to enable the use of low-cost compact FPGA devices, a novel inner parallel computation optimization method for implementing CNN and BCNN on FPGA is proposed. It is shown that compared with the popular embedded processor (ARM Cortex A9) execution latency, the proposed FPGA-based hardware accelerator reduces the processing delay in mm-wave RoF systems by about 99.45% and 92.79% for CNN and BCNN, respectively. Compared with non-optimized FPGA implementations, results show that the proposed inner parallel computation method reduces the processing latency by about 44.93% and 45.85% for CNN and BCNN, respectively. In addition, compared with the GPU implementation, the latency of CNN implementation with the proposed optimization method is reduced by 85.49%, while the power consumption is reduced by 86.91%. Although the latency of BCNN implementation with the proposed optimization method is larger compared with the GPU implementation, the power consumption is reduced by 86.14%. The demonstrated FPGA-based neural network hardware accelerators provide a promising solution for mm-wave RoF systems.