Leida Li

Video Quality Assessment (VQA) technology is of significant importance for improving video transmission, storage, and processing, etc. Although Convolutional Neural Networks (CNNs) based and Transformer based methods have achieved significant progress, them still suffer from some drawbacks. The previous methods treated video data as independent samples, thereby neglecting the close or distant relationships between different quality levels and consequently constraining the model's discriminative ability; meanwhile, most existing fusion strategies utilize fixed architectures that lack the ability to adapt to the data for optimal integration, resulting in insufficient utilization of spatio-temporal information and limited expression capabilities of the fused features. To address the above issues, this paper proposes a VQA method with In-Batch Contrastive Learning and Two-Phase Feature Fusion (IBCL-VQA). Firstly, the spatial features are extracted through two branches, which not only preserve the global semantics but also focus on the local regions. The temporal characteristics are obtained through a pre-trained video recognition model. Secondly, we propose an in-batch contrastive learning mechanism which, through the principles of maximizing intra-class similarity and minimizing inter-class similarity, combined with a dynamically adjusted penalty strategy, models the correlation between video quality levels. Thirdly, a two-phase feature fusion strategy, consisting of the Gated Spatio-temporal Attention Unit (GSTU) and the Adaptive Fusion Cell (AFC), is further proposed. The former achieves spatiotemporal feature fusion through dynamic weight allocation, and the latter adaptively integrates the features from the two branches based on data characteristics. Finally, a regression module outputs the quality score. Experimental results on five real-world VQA datasets demonstrate the superior performance of the IBCL-VQA. Furthermore, the strong generalizability is verified through cross-database testing. The code and pre-trained weights will be publicly available at: https://github.com/BoHu90/IBCL-VQA .

Artistic Image Aesthetics Assessment (AIAA) aims to emulate human artistic perception to evaluate the aesthetics of artistic images. Due to the highly specialized nature of human artistic perception, obtaining large-scale aesthetic annotations for model analysis presents significant challenges. Furthermore, the subjectivity of artistic aesthetics makes it difficult for existing AIAA methods to quantify aesthetic scores solely based on visual features. To address the two challenges, we introduce an AIAA model based on multi-level text prompts generation. Firstly, we leverage a text prompt-based self-supervised learning approach to augment artistic image data and adopt a multi-task learning paradigm to pre-train our multi-modal AIAA model. To further capture the abstract aesthetic characteristics of artistic images, we then adopt a domain-specific multi-modal large language model (MLLM) to simulate human artistic perception and generate aesthetic textual descriptions for images, and employ a multi-modal fusion module to integrate image features with the text features of artistic aesthetics for better feature representation. Finally, the proposed multi-modal AIAA model is trained by text-prompt learning based on the aesthetic quality levels of artistic images. By generating multi-level text prompts, our method can introduce human-perceived artistic aesthetic knowledge and obtain a more effective AIAA model. Experimental results on several AIAA datasets demonstrate that our method is superior to the state-of-the-art AIAA methods. The code of our model is available at https://github.com/cicixuxy/MTPG.

In recent years, small-sample blind image quality assessment (BIQA) has gained considerable attention for its ability to minimize dependence on expensive labeled data. However, as the sample size shrinks, both prediction accuracy and generalization performance decline significantly. A key challenge is that models trained with small datasets struggle to handle diverse image content and lack a clear understanding of how content affects quality. Motivated by the above facts, we propose a content understanding and reasoning enhanced small-sample BIQA framework. In terms of content understanding, we preserve existing understanding capabilities while enhancing adaptability to new scenarios. Specifically, a large multimodal model (LMM) is employed to generate detailed content descriptions, while a distribution-based preservation strategy is further proposed to maintain the image-text similarity understanding capability, thereby mitigating overfitting and catastrophic forgetting. Meanwhile, we propose a contrastive learning framework in which content descriptions are refined by a large language model (LLM) into contrastive ‘real-opposite’ text pairs, enabling the BIQA model to adapt to unseen scenarios by aligning with authentic semantics while distinguishing counterexamples. In terms of content-quality relation learning, we first design quality-guided chain-of-thought strategy to explicitly uncover the relation between content and quality, and distill these reasoning patterns into concise quality-content relations, which then guide the prediction process. Extensive experimental results show that the proposed method not only achieved state-of-art prediction accuracy but also achieved decent generalization ability. The code can be available at https://github.com/esnthere/CURE-IQA.

Video Quality Assessment (VQA) strives to computationally emulate human perceptual judgments and has garnered significant attention given its widespread applicability. However, existing methodologies face two primary impediments: (1) limited proficiency in evaluating samples at quality extremes (e.g., severely degraded or near-perfect videos), and (2) insufficient sensitivity to nuanced quality variations arising from a misalignment with human perceptual mechanisms. Although vision-language models offer promising semantic understanding, their reliance on visual encoders pre-trained for high-level tasks often compromises their sensitivity to low-level distortions. To surmount these challenges, we propose the Restoration-Assisted Multi-modality VQA (RAM-VQA) framework. Uniquely, our approach leverages video restoration as a proxy to explicitly model distortion-sensitive features. The framework operates through two synergistic stages: a prompt learning stage that constructs a quality-aware textual space using triple-level references (degraded, restored, and pristine) derived from the restoration process, and a dual-branch evaluation stage that integrates semantic cues with technical quality indicators via spatio-temporal differential analysis. Extensive experiments demonstrate that RAM-VQA achieves state-of-the-art performance across diverse benchmarks, exhibiting superior capability in handling extreme-quality content while ensuring robust generalization.

Image aesthetic assessment (IAA) is a challenging task due to the subjectivity and abstraction of aesthetic perception. Psychological studies reveal that aesthetic experiences often trigger emotional responses, while comment texts directly reflect people’s expressions of aesthetics and emotions. However, existing multimodal IAA methods neglect the alignment between modalities. To address this, we propose a multimodal emotion-alignment cognitive network (MEC-Net) for IAA, employing strategies of emotion alignment, subjective–objective interaction, and multimodal fusion. First, an emotion alignment module is introduced to align image and text modalities using emotional stimuli, enhancing the consistency of heterogeneous modal features. Then, a subjective and objective representation module is proposed to extract multi-source information from text and images separately. Next, a subjective-objective interactive LSTM (SO-LSTM) is designed to capture the deep interaction between images and text in aesthetic understanding. Finally, an dynamic multimodal fusion (DMF) based on low-rank decomposition is proposed to integrate subjective, objective, and subjective-objective interactive modal features for aesthetic distribution prediction. Extensive experiments and qualitative analysis on image aesthetic benchmarks indicate that the proposed MEC-Net outperforms the state-of-the-art on three IAA tasks. Further, we increase emotion classification task-driven evaluation metrics to verify the strong generalizability of the proposed MEC-Net.