Qualitative results on the Darmstadt Noise Dataset for real-world denoising. Compared with the state-of-the-art CBDNet, Path-Restore could successfully address more severe noise (see the left columns) and recover more detailed textures (see the right colums).

The policy of path selection. The green color represents short network paths while the red color stands for long paths. It is observed that dark regions with severe noise are processed with long paths while bright regions with slight noise are processed with short paths.

Abstract


Very deep Convolutional Neural Networks (CNNs) have greatly improved the performance on various image restoration tasks. However, this comes at a price of increasing computational burden, which limits their practical usages. We believe that some corrupted image regions are inherently easier to restore than others since the distortion and content vary within an image. To this end, we propose Path-Restore, a multi-path CNN with a pathfinder that could dynamically select an appropriate route for each image region. We train the pathfinder using reinforcement learning with a difficulty-regulated reward, which is related to the performance, complexity and "the difficulty of restoring a region". We conduct experiments on denoising and mixed restoration tasks. The results show that our method could achieve comparable or superior performance to existing approaches with less computational cost. In particular, our method is effective for real-world denoising, where the noise distribution varies across different regions of a single image. We surpass the state-of-the-art CBDNet by 0.94 dB and run 29% faster on the realistic Darmstadt Noise Dataset. Models and codes will be released.

Path-Restore



Framework Overview. Path-Restore is composed of a multi-path CNN and a pathfinder. The multi-path CNN contains N dynamic blocks, each of which has M optional paths. The number of paths is made proportional to the number of distortion types we aim to address. The pathfinder is able to dynamically select paths for different image regions according to their contents and distortions.

Materials


Citation

@Article{yu2019path,
 author = {Yu, Ke and Wang, Xintao and Dong, Chao and Tang, Xiaoou and Loy, Chen Change},
 title = {Path-Restore: Learning Network Path Selection for Image Restoration},
 journal = {arXiv preprint arXiv:1904.10343},
 year = {2019} 
}