Xiaoming Liu

Multimodal large language models (MLLMs) achieve strong performance in multimodal information extraction (MIE). In healthcare and elderly-care applications, restricted data access, privacy constraints, and scarce domain-specific annotations often favor inference with frozen models. Standard evaluation measures whether extracted results match reference labels but does not test whether a result changes when a text span identified by the model as its support is modified. Controlled Text-Anchored Evidence-Dependence Verification for Low-Resource Multimodal Information Extraction (CEDV-MIE) implements this test as a bounded inference-time procedure. A text span reported as support for an initial schema-constrained extraction is modified under validity constraints, while the paired image, visual contextual records, and non-target text remain fixed. The original and perturbed extractions are compared to produce a prediction-level consistency record and a final schema-constrained extraction. No parameter updates or additional evidence annotations are required. Across four public MIE benchmarks and four frozen backbones, CEDV-MIE improves average F₁ over direct prompting in all 16 dataset–backbone configurations, with a mean gain of 3.85 points (range: 1.48–10.78). Against a self-consistency control using seven chain-of-thought samples per input (SC-7), CEDV-MIE achieves higher average F₁ in 14 of 16 configurations under the same maximum allowance of seven backbone calls. The post-hoc Consistency Check Score (CCS) increases in all eight diagnostic configurations evaluated with two backbones (mean +0.195); evidence-stability accuracy and F₁ also increase on the validated diagnostic subset.

The Medical Visual Question Answering (MVQA) task integrates computer vision with the medical field, aiming to answer questions related to medical images using natural language processing techniques. Unlike general visual question answering, MVQA presents unique challenges, particularly in terms of the medical context, dataset characteristics, and model methodologies. This paper provides a comprehensive review of the latest advancements in MVQA, aiming to promote the development of the field and provide valuable insights for researchers. The review begins by introducing the basic concepts and objectives of MVQA, followed by a retrospective on its development, the current state of research, and the distinctions between MVQA and general visual question answering. A detailed examination of the latest public datasets is also presented, analyzing their advantages and limitations from multiple perspectives while exploring their applicability in various medical contexts. Furthermore, the review delves into research methods since 2018, focusing on three key aspects: 1) model architecture, 2) modality fusion methods, and 3) data utilization strategies. These methods are discussed in terms of their origins, innovations, performance, and potential for improvement, especially in enhancing model generalizability and reducing annotation costs. Finally, the article identifies unresolved challenges in the MVQA field and outlines promising research didirections, aiming to provide valuable guidance for researchers and further advance the fields of medical visual question answering and medical artificial intelligence.

Multimodal fusion models are widely applied in practical tasks such as image classification and sentiment analysis. However, real-world data often contain noisy or imbalanced modality inputs, significantly affecting the performance of traditional fusion methods. To address this challenge, we propose an Uncertainty-Aware Multimodal Fusion (UAMF) framework designed to enhance robustness under noisy conditions. UAMF jointly models aleatoric uncertainty from noisy data and epistemic uncertainty from model limitations. It integrates a Dynamic Direct Computation (DDC) module for adaptive uncertainty estimation and an Uncertainty Fusion Mechanism (UFM) that fuses predictions using Bayesian updates. We evaluate the effectiveness of our method on two practical tasks: image classification (FOOD101, CUB-200) and multimodal sentiment analysis (MOSI, MOSEI). Compared with the baseline model QMF, UAMF achieves an average accuracy improvement of 2.3% across tasks (e.g., +3.1% on CUB-200 and +1.2% on MVSA-Single under Gaussian noise). In addition, it reduces inference time by 52%, from 187.2 ms to 89.5 ms, compared to the recent state-of-the-art model EAU. The results demonstrate that UAMF outperforms existing methods across both application domains, particularly under noisy or modality-impaired conditions. These findings highlight the robustness and generalizability of the proposed method in real-world multimodal learning scenarios.

Adapting Large Language Models (LLMs) to specialized domains like geriatric care remains a significant challenge due to the limited availability of domain-specific data and the difficulty of achieving efficient yet effective fine-tuning. Current methods often fail to effectively harness domain-specific causal insights, which are crucial for understanding and solving complex problems in low-resource domains.To address these challenges, we propose Causality-Aware, Parameter-Efficient Learning (CPEL), a novel framework that leverages domain-specific causal relationships to guide a multi-layer, parameter-efficient fine-tuning process for more effective domain adaptation. By embedding causal reasoning into the model’s adaptation pipeline, CPEL enables efficient specialization in the target domain while maintaining strong task-specific performance. Specifically, the Causal Prompt Generator of CPEL extracts and applies domain-specific causal structures, generating adaptive prompts that effectively guide the model’s learning process. Complementing this, the MPEFT module employs a dual-adapter mechanism to balance domain-level adaptation with downstream task optimization. This cohesive design ensures that CPEL achieves resource efficiency while capturing domain knowledge in a structured and interpretable manner. Based on this framework, we delved into its application in the field of geriatric care and trained a specialized large language model (Geriatric Care LLaMA) tailored for the aged-care domain, leveraging its capacity to efficiently integrate domain expertise. Experimental results from question-answering tasks demonstrate that CPEL improves ROUGE scores by 9–14% compared to mainstream LLMs and outperforms frontier models by 1–2 points in auto-scoring tasks. In summary, CPEL demonstrates robust generalization and cross-domain adaptability, highlighting its scalability and effectiveness as a transformative solution for domain adaptation in specialized, resource-constrained fields.

Large language models cross-domain named entity recognition task in the face of the scarcity of large language labeled data in a specific domain, due to the entity bias arising from the variation of entity information between different domains, which makes large language models prone to spurious correlations problems when dealing with specific domains and entities. In order to solve this problem, this paper proposes a cross-domain named entity recognition method based on causal graph structure enhancement, which captures the cross-domain invariant causal structural representations between feature representations of text sequences and annotation sequences by establishing a causal learning and intervention module, so as to improve the utilization of causal structural features by the large language models in the target domains, and thus effectively alleviate the false entity bias triggered by the false relevance problem; meanwhile, through the semantic feature fusion module, the semantic information of the source and target domains is effectively combined. The results show an improvement of 2.47% and 4.12% in the political and medical domains, respectively, compared with the benchmark model, and an excellent performance in small-sample scenarios, which proves the effectiveness of causal graph structural enhancement in improving the accuracy of cross-domain entity recognition and reducing false correlations.

Background Aspiration pneumonia (AP) is a common complication in the intensive care unit (ICU), which is associated with significantly increased morbidity and mortality and has a significant impact on patient prognosis. Antibiotics are commonly used in the clinical treatment of AP. However, the prognostic impact of antibiotics on patients with AP has not been adequately characterized. The purpose of this study is to illustrate the relationship between the use of antibiotics and in-hospital mortality of AP patients, as well as to analyze the effects of different antibiotic treatment regimens on the prognosis of the patients, and to further understand the distribution of pathogens and drug resistance in AP patients, so as to provide guidance information for the rational use of medication for patients in the clinic. Methods Clinical data of AP patients were extracted from the MIMIC-IV database. Statistical methods included multivariate logistic regression, propensity score matching (PSM), and inverse probability weighting (IPW) based on propensity scores to ensure the robustness of the findings. In addition, the characteristics of the medications used by patients with AP were described using statistical graphs and tables. Results A total of 4132 patients with AP were included. In-hospital mortality was significantly lower in the group using antibiotics compared to the group not using antibiotics (odds ratio [OR] = 0.44, 95% confidence interval [CI] 0.27- 0.71, P = 0.001). Furthermore, in the group using mechanical ventilation (MV), antibiotics use significantly reduced in-hospital mortality (OR = 0.30, 95% CI 0.15-0.57, P < 0.001). Vancomycin and cephalosporins are the most commonly used antibiotics to treat AP. Specifically, vancomycin in combination with piperacillin-tazobactam was used most frequently with 396 cases. The highest survival rate (97.6\%) was observed in patients treated with levofloxacin combined with metronidazole. Additionally, vancomycin combined with piperacillin-tazobactam had many inflammation related features that differed significantly from those in patients who did not receive medication. Conclusions Antibiotics use is closely associated with lower in-hospital mortality in ICU patients with AP. Moreover, understanding antibiotics use, the composition of pathogenic bacteria, and the rates of drug resistance in patients with AP can aid in disease prevention and prompt infection control.