Abstract
The problem of Lp-norm constrained coding is to convert signal into code that lies inside an Lp-ball and most faithfully reconstructs the signal. Previous works under the name of sparse coding considered the cases of L0 and L1 norms. The cases with p > 1 values, i.e. non-sparse coding studied in this paper, remain a difficulty. We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the Frank-Wolfe algorithm for solving an Lp-norm constrained problem with p ≥ 1. We show that the Frank-Wolfe solver for the Lp-norm constraint leads to a novel closed-form nonlinear unit, which is parameterized by p and termed poolp. The poolp unit links the conventional pooling, activation, and normalization operations, making F-W Net distinct from existing deep networks either heuristically designed or converted from projected gradient descent algorithms. We further show that the hyper-parameter p can be made learnable instead of pre-chosen in F-W Net, which gracefully solves the non-sparse coding problem even with unknown p. We evaluate the performance of F-W Net on an extensive range of simulations as well as the task of handwritten digit recognition, where F-W Net exhibits strong learning capability. We then propose a convolutional version of FWNet, and apply the convolutional F-W Net into image denoising and super-resolution tasks, where F-W Net all demonstrates impressive effectiveness, flexibility, and robustness.
Funding Information
  • National Key Research and Development Plan (2017YFB1002401)
  • Natural Science Foundation of China (61772483)

This publication has 47 references indexed in Scilit: