Dynamic Network for Language-based Fashion Retrieval
Open Access
- 29 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 49-57
Abstract
Language-based fashion image retrieval, as a kind of composed image retrieval, presents a substantial challenge in the domain of multi-modal retrieval. This task aims to retrieve the target fashion item in the gallery given a reference image and a modification text. Existing approaches primarily concentrate on developing a static multi-modal fusion module to learn the combined semantics of the reference image and modification text. Despite their commendable advancements, these approaches are still limited by a deficiency in flexibility, which is attributed to the application of a singluar fusion module across diverse input queries. In contrast to static fusion methods, we propose a novel method termed Dynamic Fusion Network (DFN) to compose the multi-granularity features dynamically by considering the consistency of routing path and modality-specific information simultaneously. In specific, our proposed method is consisted of two modules: (1) Dynamic Network. The dynamic network enables a flexible combination of the different operation modules, providing multi-granularity modality interaction for each reference image and modifier text. (2) Modality Specific Routers (MSR). The MSR generates precise routing decisions based on the distinct semantics and distributions of each reference image and modifier text. Extensive experiments on three benchmarks,\textiti.e., FashionIQ, Shoes and Fashion200K, demonstrate the effectiveness of our proposed model compared with the existing methods.Keywords
Funding Information
- Zhejiang Province Nature Science Foundation of China (DT23F020010)
- National Natural Science Foundation of China (62006123 and 62202436)
- China Postdoctoral Science Foundation Grant (2022M722911)
This publication has 32 references indexed in Scilit:
- Cross-Modal Attention With Semantic Consistence for Image–Text MatchingIEEE Transactions on Neural Networks and Learning Systems, 2020
- Study on Fashion Image Retrieval Methods for Efficient Fashion Visual Search2019
- Composing Text and Image for Image Retrieval - an Empirical Odyssey2019
- Stacked Cross Attention for Image-Text Matching2018
- Large-Scale Image Retrieval with Attentive Deep Local Features2017
- Memory-Augmented Attribute Manipulation Networks for Interactive Fashion Search2017
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Deep Residual Learning for Image Recognition2016
- Glove: Global Vectors for Word Representation2014
- ImageNet: A large-scale hierarchical image database2009