MatUp: Repurposing Image Upsamplers for SVBRDFs
- 24 July 2024
- journal article
- research article
- Published by Wiley in Computer Graphics Forum
- Vol. 43 (4)
Abstract
We propose MatUp, an upsampling filter for material super‐resolution. Our method takes as input a low‐resolution SVBRDF and upscales its maps so that their rendering under various lighting conditions fits upsampled renderings inferred in the radiance domain with pre‐trained RGB upsamplers. We formulate our local filter as a compact Multilayer Perceptron (MLP), which acts on a small window of the input SVBRDF and is optimized using a data‐fitting loss defined over upsampled radiance at various locations. This optimization is entirely performed at the scale of a single, independent material. Doing so, MatUpleverages the reconstruction capabilities acquired over large collections of natural images by pre‐trained RGB models and provides regularization over self‐similar structures. In particular, our light‐weight neural filter avoids retraining complex architectures from scratch or accessing any large collection of low/high resolution material pairs – which do not actually exist at the scale RGB upsamplers are trained with. As a result, MatUpprovides fine and coherent details in the upscaled material maps, as shown in the extensive evaluation we provide.Keywords
This publication has 39 references indexed in Scilit:
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment2022
- Investigating Tradeoffs in Real-World Video Super-Resolution2022
- Real-world single image super-resolution: A brief reviewInformation Fusion, 2022
- Designing a Practical Degradation Model for Deep Blind Image Super-Resolution2021
- Explorable Super Resolution2020
- Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model2019
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks2019
- The Perception-Distortion Tradeoff2018
- Sparse regression using mixed normsApplied and Computational Harmonic Analysis, 2009