Abstract
ReLU, as the core nonlinear unit in numerous deep learning backbones, plays a crucial role in this field. In recent years, various smooth ReLU-Like activation functions have demonstrated superior performance across different tasks and network architectures. In this paper, we propose a novel trainable ReLU-Like activation function, Sperf, constructed by multiplying Softplus with the Gaussian error function (erf). Sperf enhances nonlinear representation capability while maintaining numerical stability. Experimental results show that networks equipped with Sperf achieve significant performance gains across diverse tasks: six image-classification datasets; Cityscapes for semantic segmentation; and MICCAI BraTS2020 for unsupervised image registration.

This publication has 25 references indexed in Scilit: