Umapada Pal

The rise of social media has amplified activities like uploading and downloading text and images, making it challenging to monitor and analyze the vast amount of data. Automatically studying user behavior from shared images and text is complex, which motivated us to develop a novel method for personality trait identification. Current methods typically depend on a single modality, such as text, images, or video, and fail to leverage the multimodal nature of data available on social media. In this article, we introduce a multimodal approach that integrates features from various image types (e.g., uploaded images, banners, and profile pictures) and textual content (e.g., posted text, text within images, and image descriptions) to classify social media images based on the Big Five Factors (BFF) of Personality Traits. Our model uses gradients in the frequency domain to detect focused image regions through a combination of quad trees and CNNs. Color features are extracted using a blend of a color synergy model and median cut clustering, while textual features are derived from a sentence inference model combining fuzzy vectors and CNNs. These features are fused using a Fully Connected Neural Network (FCNN) for personality trait classification. Experimental results on our dataset and two benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in classification accuracy.

Sentiment analysis is essential for several real-world applications, such as opinion mining and predicting a person’s intent and personality. Most existing work aims to address challenges of sentiment analysis using normal text and images uploaded on social media. This work aims to use scene text images for sentiment analysis to assist in understanding the intentions of captured scenes. We present the Transformer-based Semantic Refinement Block (TSRB), which comprises a multimodal approach and semantic gating. The proposed method constructs hierarchically fused image and text representations and then routes them through a TSRB and a learned three-way Semantic Gating module. The image branch encodes both the full meme image and text image extracted from the input image through a convolutional network with spatial attention; the text branch encodes OCR text, raw tweet text, and image captions via three independent DistilBERT[Formula: see text]CNN encoders and hierarchically fuses them. The resulting visual and textual embeddings are jointly refined by three stacked Transformer encoder layers within the proposed TSRB and then selectively blended by a softmax-weighted Semantic Gate that dynamically arbitrates among the post-attention, visual, and textual streams. Experiments are conducted on two standard datasets (MVSA-Single and Memotion) and compared with state-of-the-art models to demonstrate the effectiveness of the proposed method.

Object detectors often perform well in-distribution, yet degrade sharply on a different benchmark. We study cross-dataset object detection (CD-OD) through a lens of setting specificity. We group benchmarks into setting-agnostic datasets with diverse everyday scenes and setting-specific datasets tied to a narrow environment, and evaluate a standard detector family across all train--test pairs. This reveals a clear structure in CD-OD: transfer within the same setting type is relatively stable, while transfer across setting types drops substantially and is often asymmetric. The most severe breakdowns occur when transferring from specific sources to agnostic targets, and persist after open-label alignment, indicating that domain shift dominates in the hardest regimes. To disentangle domain shift from label mismatch, we compare closed-label transfer with an open-label protocol that maps predicted classes to the nearest target label using CLIP similarity. Open-label evaluation yields consistent but bounded gains, and many corrected cases correspond to semantic near-misses supported by the image evidence. Overall, we provide a principled characterization of CD-OD under setting specificity and practical guidance for evaluating detectors under distribution shift. Code will be released at \href{[https://github.com/Ritabrata04/cdod-icpr.git}{https://github.com/Ritabrata04/cdod-icpr}.