Hongxu Zhu

In the past decade, benefiting from the exposure of Internet-of-Things (IoT) and information and communications technology (ICT), consumers are experiencing an increased link to electronics in the digital world and enjoying the drastically convenient and smart applications. The human-machine interface (HMI) serves as a channel for users to communicate their intentions, get feedback, and manage numerous systems and gadgets. For consumer electronics, the graphical user interface (GUI) has been widely adopted and continues to be dominant. However, with increasing sophisticated functions has been integrated in consumer electronics, GUIs sometimes become complicated and time-consuming. In most cases, users need to read the instruction or be pre-trained on how to operate the GUI. Thus, people have been forced to adapt to devices in this manner, which is inefficient, indirect, and more machine-centric than human-centric. Meanwhile, the continuous innovating breakthrough in AI (artificial intelligence) has been the focus for technical research across the globe. Deloitte survey indicates that the global AI market has reached USD1.9 trillion by 2019 and is expected to exceed USD6 trillion by 2025, with a compound growth rate of 30% from 2017 to 2025. Artificial intelligence has a profound impact across industries, especially since people are beginning to implement its capabilities across verticals. This is truer in the case of HMI design. One of the most seamless ways that AI is enhancing HMI design is with the use of AI-powered assistants. This includes Siri, Alexa, Google Assistant, and more which allow human interactions with devices using natural language. Indeed, technologies such as touchscreens, voice recognition, gesture control, virtual reality (VR), augmented reality (AR), brain-computer interfaces (BCI), are all included and expected to diversify the next generation of HMIs.

Transformer network architecture has proven effective in speech enhancement. However, as its core module, self-attention suffers from quadratic complexity, making it infeasible for training on long speech utterances. In practical scenarios, speech enhancement models are often required to perform on noisy speech at run-time that is substantially longer than the training utterances. It remains a challenge how a Transformer-based speech enhancement model can generalize to long speech utterances. In this paper, extensive empirical studies are conducted to explore the model's length generalization ability. In particular, we conduct speech enhancement experiments on four training objectives and evaluate with five metrics. Our studies establish that positional encoding is an effective instrument to dampen the effect of utterance length on speech enhancement. We first explore several existing positional encoding methods, and the results show that relative positional encoding methods exhibit a better length generalization property than absolute positional encoding methods. Additionally, we also explore a simpler and more effective positional encoding scheme, i.e. LearnLin, that uses only one trainable parameter for each attention head to scale the real relative position between time frames, which learns the different preferences on short- or long-term dependencies of these heads. The results demonstrate that our proposal exhibits excellent length generalization ability with comparable or superior performance than other state-of-the-art positional encoding strategies.

Transformer network architecture has proven effective in speech enhancement. However, as its core module, self-attention suffers from quadratic complexity, making it infeasible for training on long speech utterances. In practical scenarios, speech enhancement models are often required to perform on noisy speech at run-time that is substantially longer than the training utterances. It remains a challenge how a Transformer-based speech enhancement model can generalize to long speech utterances. In this paper, extensive empirical studies are conducted to explore the model's length generalization ability. In particular, we conduct speech enhancement experiments on four training objectives and evaluate with five metrics. Our studies establish that positional encoding is an effective instrument to dampen the effect of utterance length on speech enhancement. We first explore several existing positional encoding methods, and the results show that relative positional encoding methods exhibit a better length generalization property than absolute positional encoding methods. Additionally, we also explore a simpler and more effective positional encoding scheme, i.e. LearnLin, that uses only one trainable parameter for each attention head to scale the real relative position between time frames, which learns the different preferences on short- or long-term dependencies of these heads. The results demonstrate that our proposal exhibits excellent length generalization ability with comparable or superior performance than other state-of-the-art positional encoding strategies.

Previous studies on neurosteered hearing aids employed a neural decoder to reconstruct the speech stimulus from EEG signals to establish the attended sound source. However, this approach presents several limitations, such as the need for clean speech stimuli—which are often unavailable in real-world scenarios—and long processing windows. To address these challenges, we propose a novel EEG-based neurosteered speaker extraction (ENSE) mechanism that performs a joint action of speech separation and direct attention classification without the need for explicit speech stimulus reconstruction. Specifically, a typical speech separation model is first pretrained on a large speech corpus. We then train a speech-EEG match detector to perform direct attention classification by detecting which of the separated speech stimuli, or which of the speakers, induces the observed EEG signals. Experimental results show that ENSE effectively identifies and extracts the attended speech while suppressing unattended ones in a mixture. With time-domain speech separation and direct attention classification, ENSE offers a low-latency solution that marks an important step towards practical neurosteered hearing prostheses.

Human brain possesses an extraordinary ability to attend to a specific sound source in a multi-talk, noisy environment such as a cocktail party. Auditory attention detection (AAD) aims to automatically identify such attentive neural activity from brain signals, such as electroencephalography (EEG). Given the dynamic and nonlinear nature of EEG signals, we propose a spiking long short-term memory (LSTM) network to capture the temporal features from EEG data. Additionally, we introduce a spiking temporal attention mechanism that dynamically assigns differentiated weights, thereby enhancing the representation of EEG features. We evaluate our proposed spiking temporal LSTM model, named ST-LSTM, on a widely used AAD dataset through a wide range of experiments. The experiments demonstrate that ST-LSTM outperforms other competing models, especially in low-latency settings. Moreover, with low power consumption, ST-LSTM offers a practical solution for edge computing implementations such as neuro-steered hearing aids, and other portable brain-computer interfaces.

Nowadays, our society has been experiencing the development of Healthcare 4.0, focusing on digitalization by utilizing cloud/fog computing-based consumer Internet-of-Things (CIoT). Meanwhile, Industry 5.0 has been defined recently, targeting improving the current Industry 4.0. Inspired by Industry 5.0, we are among the first wave of researchers to propose Healthcare 5.0, specifically in the perspective of fog/cloud computing. Concretely, we perform a comprehensive vertical and horizontal summary of Healthcare 1.0 -4.0 and Industry 1.0 -4.0, which enables us to be among the first research efforts to provide a unique prediction of the corresponding upcoming Healthcare 5.0, based on Industry 5.0. In particular, we predict the potential development in Healthcare 5.0, specifically for fog/cloud computing, on top of a comprehensive overview of the role and state-of-the-art development in Healthcare 4.0. We also give our unique insight of the potential technologies of CIoT and fog/cloud computing in Healthcare 5.0, as well as the corresponding opportunities and challenges. Therefore, for the first time, this article provides deep and well-thought insight to open a new era towards the future fog/cloud computing for healthcare system development towards Healthcare 5.0.

The Internet of Things (IoT) has been attracting people to its capability to deal with smart applications. However, with the development of IoT, there are more attacks to threaten IoT systems. Especially, the distributed denial of service (DDoS) attack can lead to mighty destruction to IoT servers, causing the whole IoT network to be out-of-service. Hitherto, given the lack of a common standard for defining the IoT-driven DDoS (IoT-DDoS) attack, the DDoS defense system for IoT is developed without accurate guidance. Additionally, defense against multi-layer IoT-DDoS attacks is rarely covered by previous works. To address these issues, a deep reinforcement learning-based multi-layer IoT-DDoS defense system (DRL-MLDS) is proposed with the reward metrics in compliance with IEEE P2668 – the first of its kind. In addition, to provide a resilient blocking time configuration for false-positive samples, a new power-law-based blocking time mechanism is developed to cooperate with the DRL-MLDS. The outcome reveals that the DRL-MLDS can reach the same accuracy level (i.e., more than 96%) as previous works under single protocol-based IoT DDoS attack, as well as providing around 97% defense accuracy on multi-layer IoT-DDoS attack, which was rarely discussed in previous works. Additionally, by applying the IEEE P2668-compliant reward metrics, the applicability index (ADex) of DRL-MLDS can be improved from 3.2 to 4.4, fulfilling the recommendation of ADex (e.g., $\mathbf {>}3.5$ ) toward IoT best practices. The DRL-MLDS can be extended to Metaverse design and applications.

The adoption of voice user interface (VUI) will promote network automation with enhanced efficiency with reduced simplicity and operating expense in Industry 5.0. Given the noisy environments, speech denoising is indispensable for the VUI in Internet of Things (IoT) or Industrial IoT (IIoT). Despite Transformer's recent success in speech denoising, the adopted full self-attention suffers from quadratic complexity, which challenges the computational power of the IoT/IIoT components. Considering the strong local correlations of speech signals, a speech-oriented sparse attention denoising scheme is developed to keep the meaningful local and global dependencies while mitigating the redundant attentions, resulting in a significant reduction in computational complexity. With the full self-attention as the baseline, experimental results revealed that the proposed scheme achieves a better denoising performance and yields a lower computational cost, indicating the strong potential for various VUI application scenarios in IoT and IIoT toward Industry 5.0.

LoRa wide area network (LoRaWAN), an emerging IoT protocol, has been popularized in large-scale applications, given its long-range and low-power properties. Hitherto, there is no appropriate traffic model for LoRaWAN to estimate the heterogeneous arriving traffic at the network server cluster (NSC). Inefficient computation power planning or even processing failure might be further caused. Radio replication, commonly existed in the arriving traffic at NSC in LoRaWAN, also causes difficulty estimating the makespan (i.e., mean processing time in NSC). To overcome the abovementioned limitations, a heterogeneous radio-replication-aware traffic aggregation model is proposed to estimate the arriving traffic for LoRaWAN. In addition, a radio-replication-combined supermarket model (RRC-SM), on top of HTAM, is proposed to achieve load balancing among servers in LoRaWAN. Furthermore, a nondominated sorting genetic algorithm based on multiobjective optimization is developed to simultaneously minimize cost and latency on NSC. Experiments reveal that the proposed HTAM and RRC-SM agree well with the simulation outcome. Under the arriving traffic estimated as 6.16 erlangs with four radio replications of each arriving packet on average, the proposed RRC-SM provides more than 50% reduction on the total processing latency and 75% reduction on the number of servers in NSC than other existing models.

The Received signal strength (RSS) fingerprint-based approaches are widely used for indoor location-based services. The emerging Long Range Wide Area Network (LoRaWAN) is a cost-effective solution for indoor latency-tolerant location-based services attributed to its long-range property. In general, there are serious RSS fluctuations due to fadings along the communication path, thus significantly jeopardizing the localization accuracy. To overcome the challenge, we propose the extreme RSS to stabilize the fingerprint database and formulate boundary autocorrelation to downsize tremendously the searching complexity and thus proliferating localization accuracy. In essence, the RSS fluctuations are modeled as a Bernoulli random process so that the RSS stability can be estimated by a newly defined fluctuation analytic function. To mitigate the impact of the perturbative fluctuation, the extreme RSS is further defined to cultivate a highly stable and robust fingerprint database which withstand environmental dynamics. In addition, boundary autocorrelation is developed to measure and compare the similarity between the measured RSS values versus the prestored fingerprint database. RSS values with low autocorrelation coefficients are eradicated from the typically lengthy searching. The downsized complexity significantly improves the localization accuracy. Experiments were carried out and the results revealed that the proposed method achieved sub-10-meter localization accuracy in indoor environments. Such accuracy is encouraging and superior in contemporary LoRaWAN measurements.