Kangping Wang

Large-scale pre-trained Vision Transformer (ViT) models have demonstrated remarkable performance on visual tasks but are computationally expensive to transfer to downstream tasks. Parameter-Efficient Fine-Tuning (PEFT) offers a promising transferring approach by updating only a subset of parameters. However, PEFT's effectiveness is hindered by discrepancies between pre-training and downstream tasks in terms of object scale and granularity. Downstream tasks often focus on finer-grained and more specialized recognition, requiring more detailed features. The diversity of feature scales of existing PEFT methods for ViT is limited. To address this, we propose a novel PEFT method named Wavelet-based multi-Scale Tuning (WST), which learns multi-scale features in a simple and efficient way. WST introduces a parallel fine-tuning patch embedding branch with a smaller patch size than the pre-trained model to capture finer-grained features. Furthermore, to handle the computational challenge from the resulting longer token sequence, WST designs wavelet fine-tuning blocks that balance both efficiency and performance. In the block, wavelet transform enables invertible and lossless down-sampling of the longer token sequence, aligning it with that of the backbone, and two lightweight linear mappings are employed to learn task-specific features. This design facilitates efficient multi-scale information exchange between the pre-trained backbone and fine-tuning branch. Extensive experiments on transfer learning demonstrate the promising performance and efficiency of our WST.

The exploration of utilizing reconfigurable circuits with parallel computing capabilities has been conducted to enhance sorting performance and reduce power consumption. However, most sorting algorithms using dedicated processors are based on parallelization designs of serial algorithms without considering the design method of large-scale integrated circuits. This results in various issues, including the overuse of I/O interface resources, on-chip storage resources, and complex layout wiring. In this paper, we extend the 2-tuple relation in the Uniform Recurrence Equation (URE) structure used to define the systolic array to n-tuples, and the extended structure is flexible in defining I/O bandwidth and concurrency. Then we define the Multi-Concurrency Systolic Sorter Array (MCSSA) algorithm based on the extended URE structure, which has a flexible 4N/n time complexity based on the n-tuple relation. Moreover, this systolic array can simultaneously sort two independent sequences, increasing the reuse of resources. Afterwards, we encapsulate each n-tuple into a Processing Element(PE) cell. The entire MCSSA consists of these interconnected PE cells, each of which can be customised in terms of data bit width and type. Last but not least, we have improved the merge tree structure called MC-merge tree. The concurrency of this algorithm can also be flexibly defined, we use this algorithm combined with MCSSA to cope with large-scale sorting scenarios. In our experiments, we have demonstrated the speed-up ratio of MCSSA relative to other state of the art (SOTA) sorting algorithms. Inheriting the unity and simplicity from the Systolic Array architecture, MCSSA achieves a maximum 73.17x acceleration ratio on the U200. In addition, the MC-merge tree expands the MCSSA sorting scale with a maximum of 450.56 times while maintaining the advantage of the acceleration ratio. The results of our study demonstrate that MCSSA and MC-merge tree have better acceleration, throughput and scalability advantages over other SOTA algorithms.

New artificial intelligence scenarios, such as high-precision online industrial detection, unmanned driving, etc., are constantly emerging and have resulted in an increasing demand for real-time image processing with high frame rates and low power consumption. Histogram equalization (HE) is a very effective and commonly used image preprocessing algorithm designed to improve the quality of image processing results. However, most existing HE acceleration methods, whether run on general-purpose CPUs or dedicated embedded systems, require further improvement in their frame rate to meet the needs of more complex scenarios. In this paper, we propose an HE acceleration method for FPGAs based on a two-dimensional configurable pipeline architecture. We first optimize the parallelizability of HE with a fully configurable two-dimensional pipeline architecture according to the principle of adapting the algorithm to the hardware, where one dimension can compute the cumulative histogram in parallel and the other dimension can process multiple inputs simultaneously. This optimization also helps in the construction of a simple architecture that achieves a higher frequency when implementing HE on FPGAs, which consist of configurable input units, calculation units, and output units. Finally, we optimize the pipeline and critical path of the calculation units. In the experiments, we deploy the optimized HE on a VCU118 test board and achieve a maximum frequency of 891 MHz (which is up to 22.6 times more acceleration than CPU implementations), as well as a frame rate of 1899 frames per second for 1080p images.

Neural architecture search (NAS) is an extremely complex optimization task. Recently, population-based optimization algorithms, such as evolutionary algorithm, have been adopted as search strategies for designing neural networks automatically. Various population-based NAS methods are promising in searching for high-performance neural architectures. The explosion gravitation field algorithm (EGFA) inspired by the formation process of planets is a novel population-based optimization algorithm with excellent global optimization capability and remarkable efficiency, compared with the classical population-based algorithms, such as GA and PSO. Thus, this paper attempts to develop a more efficient NAS method, called EGFA-NAS, by utilizing the work mechanisms of EGFA, which relaxes the search discrete space to a continuous one and then utilizes EGFA and gradient descent to optimize the weights of the candidate architectures in conjunction. To reduce the computational cost, a training strategy by utilizing the population mechanism of EGFA-NAS is proposed. In addition, a weight inheritance strategy for the new generated dust individuals is proposed during the explosion operation to improve performance and efficiency. The performance of EGFA-NAS is investigated in two typical micro search spaces: NAS-Bench-201 and DARTS, and compared with various kinds of state-of-the-art NAS competitors. The experimental results demonstrate that EGFA-NAS is able to match or outperform the state-of-the-art NAS methods on image classification tasks with remarkable efficiency improvement.

Since its inception, Vision Transformer (ViT) has emerged as a prevalent model in the computer vision domain. Nonetheless, the multi-head self-attention (MHSA) mechanism in ViT is computationally expensive due to its calculation of relationships among all tokens. Although some techniques mitigate computational overhead by discarding tokens, this also results in the loss of potential information from those tokens. To tackle these issues, we propose a novel token pruning method that retains information from non-crucial tokens by merging them with more crucial tokens, thereby mitigating the impact of pruning on model performance. Crucial and non-crucial tokens are identified by their importance scores and merged based on similarity scores. Furthermore, multi-scale features are exploited to represent images, which are fused prior to token pruning to produce richer feature representations. Importantly, our method can be seamlessly integrated with various ViTs, enhancing their adaptability. Experimental evidence substantiates the efficacy of our approach in reducing the influence of token pruning on model performance. For instance, on the ImageNet dataset, it achieves a remarkable 33% reduction in computational costs while only incurring a 0.1% decrease in accuracy on DeiT-S.