High-Efficiency Visual Processing through Context-Aware Selective Attention

Authors:
Open Access
Publisher website
Google Scholar
Abstract
While the increasing resolution of Complementary Metal-Oxide-Semiconductor (CMOS) image sensors offers greater detail, it imposes significant computational burdens. Drawing inspiration from the human visual system, selective attention methods mitigate this by identifying and transmitting only the most informative pixels. However, pixel-wise selection often results in irregular regions that hinder downstream tasks and limit processing efficiency. To address these limitations, we propose the patch-wise selective transformer (PST). By shifting from pixel-level to patch-level processing, PST improves both selection accuracy and structural consistency. The framework incorporates a feature pyramid to capture multi-scale objects with richer feature representations. To further optimize efficiency, we introduce Selective Batchify Attention (SBA), which utilizes a lightweight gating layer to dynamically select and regroup informative tokens into new batches. This mechanism enables flexible token counts during Graphics Processing Unit (GPU) training and inference. Experimental results demonstrate that our approach achieves superior selection accuracy, minimizes information loss, and enhances object detection performance while significantly reducing computational overhead.

This publication has 38 references indexed in Scilit: