Asif Raza

Zero-shot temporal action detection (ZS-TAD), aiming to recognize and detect new and unseen video actions, is an emerging and challenging task with limited solutions. Recent studies have adapted the vision-language pre-trained model CLIP for this task in a parameter-efficient fine-tuning fashion to achieve open-vocabulary detection. However, they suffer from insufficient vision-text alignment because of the dual-stream structure of CLIP and yield inferior TAD results due to the lack of accurate action prior. In this paper, we target the above limitations and propose to learn multimodal Prompts and Text-Enhanced Actionness (mProTEA) for ZS-TAD. Specifically, we insert learnable layer-wise prompts into the vision and text branches of the frozen CLIP and establish a strong coupling between them, resulting in multimodal prompts that can boost cross-modal alignment. To ease computation costs, we propose to conduct multimodal prompt learning on an image recognition dataset with rich concepts (e.g., ImageNet) first and then keep them frozen during TAD fine-tuning. For improving TAD, we introduce text-enhanced actionness modeling, where we leverage the concise semantics of text to assist the calculation of class-agnostic actionness scores, to offer accurate prior information for both action classification and localization. With the above designs, our mProTEA excels in extensive TAD experiments, surpassing the strong competitor STALE by 5.1% on ActivityNet under the zero-shot setting and achieving state-of-the-art performance in conventional supervised scenarios. Ablation studies confirm the effectiveness of our proposals and show superior domain generalization of multimodal prompts learned on ImageNet against the other 10 image recognition datasets.

Understanding the structure–function relationship in a neuronal network is one of the major challenges in neuroscience research. Despite increasing researches at circuit connectivity and neural network structure, their structure-based biological interpretability remains unclear. Based on the attractor theory, here we develop an analytical framework that links neural circuit structures and their functions together through fixed point attractor in Caenorhabditis elegans. In this framework, we successfully established the structural condition for the emergence of multiple fixed points in C. elegans connectome. Then we construct a finite state machine to explain how functions related to bistable phenomena at the neural activity and behavioral levels are encoded. By applying the proposed framework to the command circuit in C. elegans, we provide a circuit level interpretation for the forward-reverse switching behaviors. Interestingly, network properties of the command circuit and first layer amphid interneuron circuit can also be inferred from their functions in this framework. Our research indicates the reliability of the fixed point attractor bridging circuit structure and functions, suggesting its potential applicability to more complex neuronal circuits in other species.

Change detection (CD) is an essential remote sensing application for Earth observations widely used for several monitoring, management, and surveillance-related purposes. Generally, pixel-to-pixel prediction roles are susceptible to position details, as high-resolution bitemporal images contain abundant ground details, and so it needs more precautions to extract features. Recently, various efforts have been made for deep semantic change features, but the importance of grained features has always been ignored, leading to uncertainty of change ground entities. Besides, for an idealistic model, both accuracy and speed need focus. Thus, a novel method EUNet-CD is proposed, based on an efficient nested connection for a robust CD as a remedy for the above issues. The proposed scheme comprises: 1) an efficient convolution module (ECM)-based encoder/decoder to concise the ground entities’ details to minimize the parameters and 2) ultimate fusion strategies (based on spatial/channel attention) act as a repeater to refine the multiscale features and avoid subsampling loss. Experimental validation on different criteria shows the superiority of the proposed EUNet-CD among the state-of-the-arts (SOTA) methods. Our EUNet-CD remarkably improves the overall accuracy and minimizes the computational parameters and time floating point of operations per second (FLOPs) by 33% and 20%.

Infrared (IR) and visible images are heterogeneous data, and their fusion is one of the important research contents in the remote sensing field. In the last decade, deep networks have been widely used in image fusion due to their ability to preserve high-level semantic information. However, due to the lower resolution of IR images, deep learning-based methods may not be able to retain the salient features of IR images. In this article, a novel IR and visible image fusion based on IR Features & Multiscale Dense Network (IR-MSDNet) is proposed to preserve the content and key target features from both visible and IR images in the fused image. It comprises an encoder, a multiscale decoder, a traditional processing unit, and a fused unit, and can capture incredibly rich background details in visible images and prominent target details in IR features. When the dense and multiscale features are fused, the background details are obtained by utilizing attention strategy, and then combined with complimentary edge features. While IR features are extracted by traditional quadtree decomposition and Bezier interpolation, and further intensified by refinement. Finally, both the decoded multiscale features and IR features are used to reconstruct the final fused image. Experimental evaluation with other state-of-the-art fusion methods validates the superiority of our proposed IR-MSDNet in both subjective and objective evaluation metrics. Additional objective evaluation conducted on the object detection (OD) task further verifies that the proposed IR-MSDNet has greatly enhanced the details in the fused images, which bring the best OD results.

Multimodal fusion is an essential research area in computer vision application. Though, there are still many obstacles in image fusion domain, that cause the loss key content, due to method limitation or its least efficiency. To solve these problems, a novel method PFAF-Net (Pyramid Feature Attention Fusion strategy) based on multi-scale features with core idea of different level fusion strategy. Firstly, multiscale high-level features with different receptive fields are extracted by pyramid feature extraction module. Secondly, high level and low features are fused with different fusion techniques i.e. attention based fusion strategies are adopted to efficiently fused the global and local features separately. Finally, the fused image is reconstructed with the enhanced fused features by the decoder. Thus, the proposed method not only integrate multimodal data but also has rich content details inferring a superior image fusion. In addition, proposed PFAF-Net has better generalization ability than existing methods on four different multimodal (i.e. multi-modal medical, multi-exposure, multifocus and visual-infrared) of benchmark datasets. Compared with other state-of-the-art fusion methods experimentally, the proposed method has shown comparable or better fusion performance in both subjective and objective evaluation.

Remote sensing image scene classification has drawn significant attention for its potential applications in the economy and livelihoods. Unlike the traditional handcrafted features, the convolutional neural networks (CNNs) provides an excellent avenue in obtaining powerful discriminative features. Although tremendous efforts have been made so far in this domain, there are still many open challenges in scene classification due to the scene complexity with higher within-class diversity and between-class similarity. To solve the above-mentioned problems, D- CapsNet is proposed to learn the richer and more robust features for scene classification. It is an end to end network with four types of layers and incorporates visual attention mechanisms. Its diverse capsules encode different properties of complex image scenes, including deep high-level features, spatial attention based on the fusion of multilayers features, both spatial and channel attention based on high-level features, and their fusion. Experiments on three image scene datasets demonstrate that D-CapsNet outperforms other baselines and state-of-the-art methods with a significant improvement in both classification accuracy and speed.

During the course of ubiquitous data monitoring in the underwater environment, achieving sustainable communication links among the sensor nodes with astute link quality seems an ordeal challenge. Energy utilization has a direct impact because all active devices are battery dependent and no charging or replacement actions can be made when cost- effective data packet delivery has been set as the benchmark. Hop link inspection and the selection of a Shrewd link through a resurrecting link factor have been nothing short of a bleak challenge, and only possible after meticulous research to develop a shrewd underwater routing synergy using extra porous energy shells (SURS-PES) which has never been conducted before. After broadcasting packets, the sensor node conducts a link inspection phase, thereby, if any link is found to be less than or equal to 50% shaky, the destination receiving node adds its residual energy status and returns it to the source node which adds some unusable energy porous shell to strengthen the link from 5% to a maximum of 90% and sends it only to the targeted node, therefore, an unaltered data packet delivery is anticipated. Performance evaluation was carried out using an NS2 simulator and the obtained results were compared with depth-based routing (DBR) and energy efficient DBR (EEDBR) to observe the outcomes with results that confirmed the previously mentioned direction for research in this area.

Underwater sensors link establishment and quality inspection challenges are blurt out during ubiquitous data monitoring. The energy utilization has a direct impact because all active devices are battery dependents and no charging or replacement actions could be made when cost- effective data packet delivery has been set as a benchmark. The hop link inspection and the selection of a Shrewd link through resurrecting link factor was a nothing short of bleak challenge which could only be made possible after going through meticulous research by developing a shrewd underwater routing synergy using extra porous energy shells (SURS-PES) which might never have conducted of before. After broadcasting packets the sensor node conducts a link inspection phase thereby, if any link is found to be less than or equal to 50% shaky; the destination receiving node puts in own residual energy status and return back to the source node which in result adds some unusable energy porous shell to strengthen the link from 50 to 90% at most and send it only to the targeted node and an unaltered data packet delivery is anticipated. Performance evaluation has been carried out using NS2 simulator and obtained results have been compared with DBR and EEDBR to observe the distinguish outcomes thereon results in vouches for the statement that has been made earlier for this research direction.

The accomplishment of sustainable communication among source and destination sink node is a rigors challenge and even establishing bodacious communication link between these nodes is nothing short of a miracle because data routes are governed by the underwater environment. Energy consumption has a significant influence as all active devices rely on the battery. As cost-effective data packet transmission is established as a norm, no charging or replacement can be achieved. Hop link evaluation and shrewd connection discovery by way of a resurrecting linking element were just a genuinely grim task, and only feasible to create the extra powered energy pods (URR-SAEP) that had never been carried out before after detailed study. After packet transfer, the sensor node performs the link inspection process, and when a link is deemed shaky at less than or equivalent to 50 percent of capacity, the target node incorporates its residual capacity status and returns it to the source node that attaches other unoptimizable energy pods to improve only the targeted node link from 50 percent to 90 percent. Performance evaluation using NS2 with Aqua-Sim 2.0 simulator has been obtained comparing with DBR and EEDBR protocols in terms of point-to-point delay, Packet dissemination ratio, Network lifespan and Energy Diminution.