Yuexin Wu

Visual retrievers such as visual retrieval-augmented generation (RAG) have recently emerged as a powerful model for retrieving multimodal documents without the need to convert page images into text. Existing visual retrievers typically encode every document page separately, ignoring the inherent rich context information across pages within multi-page documents. However, some crucial semantic information often spans multiple pages in a document, and should be effectively encoded for better retrieval. To address this problem, this paper proposes a novel approach utilizing dynamically fusing visual context (DFVC), which adaptively encodes the semantic information across pages. In the proposed DFVC approach, a lightweight plug-and-play adapter is designed; in addition, a contrastive loss function incorporating the positive fused embedding vectors and negative embedding vectors is designed to constrain the adapter, allowing it to learn the weights for the context pages. Together, the designed adapter and loss function allow the retriever to effectively encode useful semantic information across pages while excluding distracting noise. The proposed DFVC is validated on commonly used challenging multi-page document benchmarks. Extensive experimental results demonstrate that it significantly boosts retrieval performance. In addition, the proposed DFVC is highly parameter-efficient since it employs frozen vision-language backbones, allowing it to be easily integrated into existing visual RAG pipelines for finer document retrieval.

The traditional data services of prediction for emergency or non-periodic events usually cannot generate satisfying result or fulfill the correct prediction purpose. However, these events are influenced by external causes, which mean certain a priori information of these events generally can be collected through the Internet. This paper studied the above problems and proposed an improved model—LSTM (Long Short-term Memory) dynamic prediction and a priori information sequence generation model by combining RNN-LSTM and public events a priori information. In prediction tasks, the model is qualified for determining trends, and its accuracy also is validated. This model generates a better performance and prediction results than the previous one. Using a priori information can increase the accuracy of prediction; LSTM can better adapt to the changes of time sequence; LSTM can be widely applied to the same type of prediction tasks, and other prediction tasks related to time sequence.