Yewen Huang

Multimodal sentiment analysis predicts continuous or ordinal affect from language, acoustic, and visual cues, but cross-modal alignment and per-stream reliability vary by sample. Previous fusion often collapses modalities into a symmetric or text-fixed representation, which may amplify inherent noise and ignore important information. Therefore, we propose PRISM, a primary-centric framework:(i) each modality is independently encoded by a parallel \emph{Variational Token Bottleneck} (\VTB) under a variational information-bottleneck objective, producing compact token-level latents together with token-level confidence estimates that quantify per-token reliability;(ii) a \emph{Differentiable Primary Router} (\DPR) then applies a Gumbel--Softmax relaxation to select a sample-dependent primary pathway from these bottleneck representations, rather than assuming a fixed dominant modality, and routes prediction exclusively through the enhanced primary stream instead of a pooled multimodal vector; and(iii) stacked InfoGate layers refine the selected primary by confidence-aware auxiliary-to-primary attention and adaptive auxiliary gating, using the \VTB confidence maps to downweight unreliable auxiliary tokens.We evaluate the model on both regression and classification datasets. While showing strong performance on regression tasks, the architecture also generalizes to binary humor and sarcasm classification.

Aiming at the problem of slow retrieval speed due to a large amount of data in person re-identification, this paper introduces Taylor expansion into a deep neural network, and obtains Taylor deep neural network for feature extraction, which is combined with the proposed hash probability coding algorithm, and get a fast retrieval algorithm for person re-identification based on the hash. The algorithm can not only extract the feature map information well but also perform image retrieval at a faster speed, which is a new method to solve the fast image retrieval of person re-identification. First, a Taylor deep neural network framework is proposed, and the feature representation extracted by each layer of the deep network is fused by Taylor expansion weighting to retain the overall features of the image and make the feature representation more comprehensive ; then, the triplet loss function Triplet Loss is optimized to improve the When the distance between groups cannot be optimized, the group loss function Group Loss is proposed. Finally, a new hash probability compression coding method is used to make the image retrieval fast. Experiments show that the algorithm model in this paper is 26.7% and 8.18% higher than the BaseLine model on the large data sets Cuhk03 and Market1501, respectively. It also has a good retrieval speed on images retrieved by person re-identification..

The past few years in the fields of Person Re-Identification (RE-ID) have seen attention mechanism receives enormous interest as it has superior performance in obtaining discriminative feature representations. However, a wide range of state-of-the-art RE-ID attention models only focus on one-dimensional attention design method, e.g. spatial attention and channels attention, hence the produced attention maps are neither detailed enough nor discriminative enough to capture complicated interactions of visual parts. Developing multi-scale attention mechanism for RE-ID, an under-studied approach, becomes a practicable method to overcome this deficiency. Toward this goal, we propose a Multiscale Omnibearing Attention Networks (MOAN) for RE-ID which is capable of utilizing the complex fusion information acquired from the multiscale attention mechanism with features being more representative. Specifically, MOAN takes full advantage of multi-sized convolution filters to obtain discriminative holistic and local feature maps, and adaptively conducts feature information augmentation by introducing an Omnibearing Attention (OA) module. Through the OA module, spatial attention and channel attention are integrated together in a unique way where they work in a complementary way. To sum up, MOAN not only inherits the merit of two kinds of attention mechanism but also performs well in extracting comprehensive feature information. Furthermore, taking into account the robustness of model performance, we formulate a Random Drop (RD) Function to facilitate training MOAN and further increase the diversity of training model for adaptation. Furthermore, to achieve end-to-end training, we utilize trainable parameters to take place of initial fixed parameters, and the model performance is experimentally promoted. Extensive experiments have been carried out on the four mainstream RE-ID datasets. As the result shows, our method with re-ranking achieves rank-1 accuracy of 92.29% on CUHK03-NP, 97.45% on Market-1501, 93.81% on DukeMTMC-reID and 81.53% on MSMT17-V2, outperforming the state-of-the-art methods and confirming the effectiveness of our method.

In this paper, we propose a Resetting-Label Network based on Fast Group Loss for Person Re-Identification (RLFGL-ReID). The major challenge of Re-ID lies in how to preserve the similarity of the same person against large variations caused by complex background, different illuminations, and various view angles while discriminating different individuals. To address the above-mentioned problems, we propose the RLFGL-ReID that includes resetting-label (RL) and fast group loss (FGL). Two main contributions of our network are as follows. First, a new method, the resetting-label method, which resets the ID labels, is proposed for the Re-ID network. Resetting the ID labels of each pedestrian is beneficial in maximizing the inter-group distances between each people, achieving better performance on classifying different individuals. Second, a fast group loss, i.e., an advanced version of variance group loss (VGL), is proposed to simplify the training process and accelerate the loss computation. By doing so, the network can eliminate the restriction of inputting the whole group of data when training the network. To confirm the effectiveness of our method, we extensively conduct our method on several widely used person Re-ID benchmark datasets. As the result shows, our method achieves rank@1 accuracy of 98.38% on CUHK03, 95.46% on Market1501, and 91.2% on DukeMTMC-reID, outperforming the state-of-the-art methods and confirming the advantage of our method.