Sperf: A trainable smooth non-monotonic activation function with enhanced nonlinearity
- 1 January 2026
- journal article
- Published by American Institute of Mathematical Sciences (AIMS) in Mathematical Foundations of Computing
- Vol. 11, 120-135
Abstract
ReLU, as the core nonlinear unit in numerous deep learning backbones, plays a crucial role in this field. In recent years, various smooth ReLU-Like activation functions have demonstrated superior performance across different tasks and network architectures. In this paper, we propose a novel trainable ReLU-Like activation function, Sperf, constructed by multiplying Softplus with the Gaussian error function (erf). Sperf enhances nonlinear representation capability while maintaining numerical stability. Experimental results show that networks equipped with Sperf achieve significant performance gains across diverse tasks: six image-classification datasets; Cityscapes for semantic segmentation; and MICCAI BraTS2020 for unsupervised image registration.Keywords
This publication has 25 references indexed in Scilit:
- SignReLU neural network and its approximation abilityJournal of Computational and Applied Mathematics, 2023
- Activation functions in deep learning: A comprehensive survey and benchmarkNeurocomputing, 2022
- EIS - Efficient and Trainable Activation Functions for Better Accuracy and Performance2021
- TanhSoft—Dynamic Trainable Activation Functions for Faster Learning and Better PerformanceIEEE Access, 2021
- Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic featuresScientific Data, 2017
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Deep learningNature, 2015
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989