Wei Jiang

Integrated Sensing And Communication (ISAC) has been applied to the Internet of Things (IoT) network as a promising 6G technology due to its ability to enhance spectrum utilization and reduce resource consumption, making it ideal for high-precision sensing applications. However, while the introduction of millimeter Wave (mmWave) and massive Multiple-Input Multiple-Output (MIMO) technologies can enhance the performance of ISAC systems, they extend the near-field region, rendering traditional channel parameter estimation algorithms ineffective due to the spherical wavefront channel model. Aiming to address the challenge, we propose a tensor-based channel parameter estimation and localization algorithm for the near-field mmWave massive MIMO-Orthogonal Frequency Division Multiplexing (OFDM) ISAC systems. Firstly, the received signal at the User Terminal (UT) is constructed as a third-order tensor to retain the multi-dimensional features of the data. Then, the proposed tensor-based algorithm achieves the channel parameter estimation and target localization by exploiting the second-order Taylor expansion and intrinsic structure of tensor factor matrices. Furthermore, the Cramér–Rao Bounds (CRBs) of channel parameters and position are derived to establish the lower bound of errors. Simulation results show that the proposed tensor-based algorithm is superior compared to the existing algorithms in terms of channel parameter estimation and localization accuracy in ISAC systems for IoT network, achieving errors that approach the CRBs. Specifically, the proposed algorithm attains a 79.8% improvement in UT positioning accuracy compared to suboptimal methods at SNR = 5 dB.

Most existing intelligent editing tools for music and video rely on the cross-modal matching technology of the affective consistency or the similarity of feature representations. However, these methods are not fully applicable to complex audiovisual matching scenarios, resulting in low matching accuracy and suboptimal audience perceptual effects due to ambiguous matching rules and associated factors. To address these limitations, this paper focuses on both the similarity and integration of affective distribution for the artistic audiovisual works of movie and television video and music. Based on the rich emotional perception elements, we propose a hybrid matching model based on feature canonical correlation analysis (CCA) and fine-grained affective similarity. The model refines KCCA fusion features by analyzing both matched and unmatched music–video pairs. Subsequently, the model employs XGBoost to predict relevance and to compute similarity by considering fine-grained affective semantic distance as well as affective factor distance. Ultimately, the matching prediction values are obtained through weight allocation. Experimental results on a self-built dataset demonstrate that the proposed affective matching model balances feature parameters and affective semantic cognitions, yielding relatively high prediction accuracy and better subjective experience of audiovisual association. This paper is crucial for exploring the affective association mechanisms of audiovisual objects from a sensory perspective and improving related intelligent tools, thereby offering a novel technical approach to retrieval and matching in music–video editing.

Film and TV video scenes contain rich art and design elements such as light and shadow, color, composition, and complex affects. To recognize the fine-grained affects of the art carrier, this paper proposes a multitask affective value prediction model based on an attention mechanism. After comparing the characteristics of different models, a multitask prediction framework based on the improved progressive layered extraction (PLE) architecture (multi-headed attention and factor correlation-based PLE), incorporating a multi-headed self-attention mechanism and correlation analysis of affective factors, is constructed. Both the dynamic and static features of a video are chosen as fusion input, while the regression of fine-grained affects and classification of whether a character exists in a video are designed as different training tasks. Considering the correlation between different affects, we propose a loss function based on association constraints, which effectively solves the problem of training balance within tasks. Experimental results on a self-built video dataset show that the algorithm can give full play to the complementary advantages of different features and improve the accuracy of prediction, which is more suitable for fine-grained affect mining of film and TV scenes.

Since existing music-driven dance generation methods have abnormal motion when generating dance sequences which leads to unnatural overall dance movements, a music-driven dance generation method based on a spatial-temporal refinement model is proposed to optimize the abnormal frames. Firstly, the cross-modal alignment model is used to learn the correspondence between the two modalities of audio and dance video and based on the learned correspondence, the corresponding dance segments are matched with the input music segments. Secondly, an abnormal frame optimization algorithm is proposed to carry out the optimization of the abnormal frames in the dance sequence. Finally, a temporal refinement model is used to constrain the music beats and dance rhythms in the temporal perspective to further strengthen the consistency between the music and the dance movements. The experimental results show that the proposed method can generate realistic and natural dance video sequences, with the FID index reduced by 1.2 and the diversity index improved by 1.7.

The point cloud is a form of three-dimensional data that comprises various detailed features at multiple scales. Due to this characteristic and its irregularity, point cloud analysis based on deep learning is challenging. While previous works utilize the sampling-grouping operation of PointNet++ for feature description and then explore geometry by means of sophisticated feature extractors or deep networks, such operations fail to describe multi-scale features effectively. Additionally, these techniques have led to performance saturation. And it is difficult for standard MLPs to directly "mine" point cloud. To address above problems, we propose the Detail Activation (DA) module, which encodes data based on Fourier transform after sampling and grouping. We activate the channels at different frequency levels from low to high in the DA module to gradually recover finer point cloud details. As training progresses, the proposed Point-MDA can uncover local and global geometries of point cloud progressively. Our experiments show that Point-MDA achieves superior classification accuracy, outperforming PointNet++ by 3.3% and 7.9% in terms of overall accuracy on the ModelNet40 and ScanObjectNN dataset, respectively. Furthermore, it accomplishes this without employing complicated operations, while exploring the full potential of PointNet++.

A multi-swarm-evolutionary structure based on the parasitic relationship in the biosphere is proposed in this paper and, according to the conception, the Para-PSO-ABC algorithm (ParaPA), combined with merits of the modified particle swarm optimization (MPSO) and artificial bee colony algorithm (ABC), is conducted with the multimodal routing strategy to enhance the safety and the cost issue for the mobile robot path planning problem. The evolution is divided into three stages, where the first is the independent evolutionary stage, with the same evolution strategies for each swarm. The second is the fusion stage, in which individuals are evolved hierarchically in the parasitism structure. Finally, in the interaction stage, a multi-swarm-elite strategy is used to filter the information through a predefined cross function among swarms. Meanwhile, the segment obstacle-avoiding strategy is proposed to accelerate the searching speed with two fitness functions. The best path is selected according to the performance on the safety and consumption issues. The introduced algorithm is examined with different obstacle allocations and simulated in the real routing environment compared with some typical algorithms. The results verify the productiveness of the parasitism-relation-based structure and the stage-based evolution strategy in path planning.

This article models multicolor perceptual features to investigate human perceptual responses to multiple colors based on a subjective evaluation experiment. First, an image set containing 164 multicolor samples was constructed, and multicolor objective features containing two categories with 60 dimensions were designed and quantified. Based on this, the 164 samples were evaluated by 30 subjects based on nine perceptual descriptive variables, including “swell‐shrink,” “light‐dark,” and “far‐near.” Then, a mathematical model of multicolor perceptual features (nine descriptive variables) was established by support vector regression. Finally, this article conducted model evaluation and factor analysis, and the results showed that the model could predict multicolor perceptual features well. It was also observed that, unlike single‐color perception, space perception was produced when applying multiple colors.

A novel technique is proposed for the analysis and modeling of timbre perception features, including a new terminology system for evaluating timbre in musical instruments. This database consists of 16 expert and novice evaluation terms, including five pairs with opposite polarity. In addition, a material library containing 72 samples (including 37 Chinese orchestral instruments, 11 Chinese minority instruments, and 24 Western orchestral instruments) and a 54-sample objective acoustic parameter set were developed as part of the study. The method of successive categories was applied to each term for subjective assessment. A mathematical model of timbre perception features (i.e., bright or dark, raspy or mellow, sharp or vigorous, coarse or pure, and hoarse or consonant) was then developed for the first time using linear regression, support vector regression, a neural network, and random forest algorithms. Experimental results showed the proposed model accurately predicted these attributes. Finally, an improved technique for 3D timbre space construction is proposed. Auditory perception attributes for this 3D timbre space were determined by analyzing the correlation between each spatial dimension and the 16 timbre evaluation terms.