ZXZishan Xu

Zishan Xu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Zishan Xu has a total of 32 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • MilleniaGuard: An Event-Driven Edge-AI and AIGC-Based IoT System for Ancient Mural Monitoring and Restoration

    by Zishan Xu, Jiansen Zhang, Wei Chen, Xiaofeng Zhang, Jueting Liu, Tingting Xu, Zehua Wang, Fei Richard Yu, Victor C.M. Leung


  • Open Access

    Mobiflip: Information-Bottleneck-Guided Minimal Federated Adaptation for Cross-Modal Models

    by Zishan Xu, Jiansen Zhang, Wei Chen, Jueting Liu, Tingting Xu, Zehua Wang, Abdulmotaleb El Saddik

    Cross-modal federated learning is constrained by bandwidth and on-device compute. We present Mobiflip: a minimalist strategy that freezes a lightweight backbone and communicates only a channel-wise \(1\times 1\) scaling adapter appended to the image branch. Guided by the Information Bottleneck, we prove that under common distributional and linear-encoder surrogates, per-channel scaling attains the linear optimum; coupled with the directional geometry of (Mobile)CLIP, the adapter is, in first-order approximation, an optimal preconditioner of the cosine-similarity space—preserving discriminative directions while compressing redundancy and suppressing inter-client drift. We adopt MobileCLIP as a mobile-friendly backbone to jointly minimize compute and communication. On CIFAR-10/100 and medical imaging, a single aggregation already yields stable Bacc; each round transmits only about 0.7% of backbone parameters with \(>92\%\) reduction in communication. Compared with recent federated multimodal/large-model methods, Mobiflip maintains—or even improves—accuracy under ultra-low communication.

  • GaussFocus RPN for Remote Sensing Object Detection

    by Zilong Wang, Zishan Xu, Wei Yang, Wei Chen, Yunyue Li, Hongxian Tian, Jueting Liu, Tingting Xu, Zehua Wang

    Objects in remotesensing imagery are often unevenly distributed and densely clustered, making foreground selection particularly challenging. In two-stage detectors, the Region Proposal Network (RPN) is pivotal, yet proposals from conventional RPNs can be imprecise and redundant, hurting recall and inflating computation.We introduce GaussFocus, a density-guided two-stage detector that builds a unified Gaussian foreground prior from rotated annotations and injects it into proposal generation, feature sampling, and RoI score calibration. Specifically, we derive an image-level Gaussian foreground density map and train a lightweight density branch inside the RPN, where MDeformConv uses regression-guided offsets and density-aware gating to produce features that better align with object geometry and foreground regions. A Gauss Guided Selector further leverages the predicted density map to retain proposals concentrated in foreground-dense areas, reducing background clutter and second-stage computation. In addition, an inference-only Gauss-Guided RoI logit shift calibrates RoI classification scores using the same density prior without retraining, leading to more reliable foreground–background separation and ranking. Across four challenging datasets (including DOTA-v1.0, DOTA-v2.0, and HRSC2016), GaussFocus consistently improves proposal quality and detection accuracy over strong baselines while reducing redundant RoIs and computation.


  • Foreground Attention Loss and Attention-Guided Convolution for Remote Sensing Object Detection

    by Zilong Wang, Hongxian Tian, Wei Yang, Zishan Xu, Wei Chen, Yunyue Li, Tingting Xu, Jueting Liu, Zehua Wang

    Remote sensing (RS) imagery contains vast backgrounds and densely packed, rotated small objects. One-stage rotated detectors therefore allocate most convolutional samples to background because they lack an explicit foreground discovery step. We close this gap with a lightweight discover→focus pipeline composed of two modules. First, a Foreground Attention Loss (FAL) supervises the classifier’s foreground heatmap by matching it to a rotated target–density map rendered from OBB annotations, giving the detector an explicit notion of “where the objects are.” Second, AGConv computes closed-form semantic–geometric dual offsets by fusing the learned foreground attention with anchor geometry, thereby concentrating sampling in salient regions while suppressing clutter without an extra offset-conv branch. This preserves one-stage efficiency while approximating the proposal-guided foreground extraction of two-stage detectors. Under identical ResNet–50+FPN settings, our single-stage detector attains 76.48% mAP on DOTA-v1.0, 68.65% on DOTA-v1.5, 67.2% on DIOR–R, and 97.28% (VOC12) / 90.82% (VOC07) on HRSC2016, with negligible parameter and latency overhead—demonstrating a strong accuracy–efficiency trade-off for complex RS scenes with dense, rotated targets.


  • Open Access

    MuralAgent: Enhancing Ancient Mural Outpainting with RAG-Based Texts and Multimodal Integration

    by Zishan Xu, Xiaofeng Zhang, Yuqing Yang, Wei Chen, Jueting Liu, Tingting Xu, Zehua Wang, Abdulmotaleb El Saddik

    In the context of the digital age, utilizing cutting-edge technology for the digitization and creative expansion of ancient murals is crucial, aimed at preserving and passing on cultural heritage. Existing image outpainting techniques suffer from a lack of semantic guidance. This paper introduces MuralAgent, a multimodal model based on Retrieval-Augmented Generation (RAG) technology. It precisely extracts key information from mural images and integrates it with a constructed ancient texts knowledge base to ensure the cultural and semantic consistency of the expanded images. Moreover, fine-tuning the Stable Diffusion model ensures the fidelity of the generated image styles. Specifically, this study involves constructing an ancient texts knowledge base for accurate matching, designing specific prompts for GPT-4V(ision) to extract key information, and innovatively expanding artworks through Stable Diffusion, providing a novel way for the public to reinterpret ancient murals.


  • SDANet: A Federated Efficient Remote Sensing Object Detection for Space-Air–Ground IoT

    by Zilong Wang, Wei Yang, Zishan Xu, Wei Chen, Jueting Liu, Tingting Xu, Zehua Wang, Victor C. M. Leung

    The explosive growth of remote-sensing images generated by emerging space-air–ground integrated IoT networks makes centralized detector training infeasible due to limited bandwidth and strict data privacy constraints. While lightweight single-stage object detectors offer efficiency, they suffer significant accuracy degradation for small, dense, and arbitrarily oriented targets. Furthermore, existing federated object detection frameworks typically neglect client heterogeneity. To overcome these limitations, we propose a two-stage personalized federated detection framework. In Stage 1, we independently train a conventional single-stage rotated object detector on each client and aggregate model updates using an adaptive similarity momentum aggregation (ASMA) strategy, effectively pooling knowledge across non-IID client datasets to improve global generalization. In Stage 2, each client is equipped with a private selective depthwise attention convolution (SDAConv) module, leveraging Stage-1 priors to reconstruct fine-grained, client-specific features without additional communication overhead, thus tailoring predictions to local data distributions. Experiments conducted on five non-IID splits derived from DOTA-1.0, along with DIOR and VisDrone datasets, demonstrate improvements of up to +3.5 mAP compared to federated learning baselines under the same communication budget, simultaneously maintaining global robustness and enhancing local detection accuracy.


  • Open Access

    Wakeup-Darkness: When Multimodal Meets Unsupervised Low-Light Image Enhancement

    by Xiaofeng Zhang, Zishan Xu, Hao Tang, Chaochen Gu, Wei Chen, Abdulmotaleb El Saddik

    Low-light image enhancement is a crucial visual task, and many unsupervised methods overlook the degradation of visible information in low-light scenes, adversely affecting the fusion of complementary information and hindering the generation of satisfactory results. To address this, we introduce Wakeup-Darkness, a multimodal enhancement framework that innovatively enriches user interaction through voice and textual commands. This approach signifies a technical leap and represents a paradigm shift in user engagement. We introduce a Cross-Modal Feature Fusion (CMFF) that synergizes semantic and depth context with low-light enhancement operations. Moreover, we propose a Gated Residual Block (GRB) and a channel-aware Look-Up Table (LUT) to adjust the intensity distribution of each channel. Crucially, the proposed Wakeup-Darkness scheme demonstrates remarkable generalization in unsupervised scenarios. The source code can be accessed from https://github.com/zhangbaijin/Wakeup-Dakness .


  • Open Access

    Shadclips:When Parameter-Efficient Fine-Tuning with Multimodal Meets Shadow Removal

    by Xiaofeng Zhang, Zishan Xu, Hao Tang, Chaochen Gu, Shanying Zhu, Xinping Guan

    Segment Anything Model (SAM), an advanced universal image segmentation model trained on an expansive visual dataset, has set a new benchmark in image segmentation and computer vision. However, it faced challenges when it came to distinguishing between shadows and their backgrounds. To address this, we proposed ShadClips, which consists of SAM-optimizer and SONet. It has dramatically enhanced SAM’s ability to segment shadow images, differentiating between the background and both soft and hard shadows adeptly. Due to its dependence on pixel point inputs, the SAM-Optimizer interface could do better. This method presents challenges, especially when dealing with long, extended shadows. To make the user experience more intuitive and effective, we incorporated the capabilities of CLIPs. Therefore, simple text descriptions like “A photo of a shadow” can be used to guide the SAM-Optimizer, allowing it to select the most relevant shadow mask from SAM’s comprehensive category list. Meanwhile, we introduce SONet to shadow removal. A large number of experiments on ISTD/SRD prove that the proposed method is effective and satisfactory. The source code of the ShadClips can be accessed from https://github.com/zhangbaijin/SAM-helps-Shadow.


  • Top Cited Papers

    MuralDiff: Diffusion for Ancient Murals Restoration on Large-Scale Pre-Training

    by Zishan Xu, Xiaofeng Zhang, Wei Chen, Jueting Liu, Tingting Xu, Zehua Wang

    This paper focuses on the crack detection and digital restoration of ancient mural cultural heritage, proposing a comprehensive method that combines the Unet network structure and diffusion model. Firstly, the Unet network structure is used for efficient crack detection in murals by constructing an ancient mural image dataset for training and validation, achieving accurate identification of mural cracks. Next, an edge-guided optimized masking strategy is adopted for mural restoration, effectively preserving the information of the murals and reducing the damage to the original murals during the restoration process. Lastly, a diffusion model is employed for digital restoration of murals, improving the restoration performance by adjusting parameters to achieve natural repair of mural cracks. Experimental results show that comprehensive method based on the Unet network and diffusion model has significant advantages in the tasks of crack detection and digital restoration of murals, providing a novel and effective approach for the protection and restoration of ancient murals. In addition, this research has significant implications for the technological development in the field of mural restoration and cultural heritage preservation, contributing to the advancement and technological innovation in related fields.