Learning Based Noise Component Map Estimation for Image Denoising , link to the paper
Problem of image denoising when images are corrupted by a non-stationary noise is considered in this paper. Since in practice no a priori information on noise is available, often noise statistics should be estimated before applying adenoising method. In this paper a method of estimation of map of local, patch-wise, standard deviations of noise (so-calledsigma-map) is proposed. The method based on deep convolutional neural network shows state-of-the-art accuracy of estimation both sigma-map for non-stationary noise and variance for additive white Gaussian noise. Extensive experiments on image denoising using estimated sigma-maps demonstrate that our methodout performs state-of-the-art blind image denoising methods (by up to 6 dB in PSNR) as well as other methods based on sigma-map estimation (up to 0.5 dB), providing better usage flexibility. Comparison with an ideal case when denoising is applied with the ground-truth sigma-map shows that a difference in PSNR for most of noise levels is 0.1-0.2 dB and does not exceeds 0.6 db.
- Structure scheme of proposed SDNet (SConv - stride convolution , TConv - transpose convolution)
Run sdnet_rgb_structure.m and sdnet_gray_structure.m to create initial structures of SDNet both for rgb and gray scale input images.
Run visulaize_sdnet.m to visualize network structure and see layers in detail.
There are two models available one for estimating sigma-map of grayscale images and the other one for rgb color images. Here is the link to pre-trained SDNet models pre-trained gray scale and pre-trained rgb
MATLAB 2021
For estimating sigma-map of a single noisy image run Demo.m
You can find mat files of ground truth test images in testset_rgb and testset_grayscale folders
Groud truth sigma maps can be found in ground_true_sigma_maps. you can use ground truth sigma maps to create noisy images. See the example in Demo.m
Relative error εm of sigma-maps estimation, σav is mean standard deviation of sigma-map
LADCT[1] and RHDCT[2] are conventional methods that are based on transform coefficients of image patches. DCAE[3] and VDNet[4] are Neural Network based methods.
Three types of sigma-maps with σav = 5,7,10 and results of estimations by compared methods, SDNet estimated sigma-map shows the smallest relative error in all cases
SDNet also is used for estimating AWGN sigma value by calculating median of the sigm-map. We have provided relative error ε of AWGN sigma estimation and an examples of this task with ε values in below. Results are reported both in non-clipped and clipped noise cases
Relative error ε of AWGN sigma estimation for non-clipped noise
Relative error ε of AWGN sigma estimation for clipped noise
One test image is contaminated by non-clipped AWGN with σ = 3,5,7,10,15,20,30,50 and 75, relative errors for methods of SDNet, PCA and IEDD are reported
In addition to ground truth sigma-maps, we have created random sigma-maps to test SDNet estimator and its efficiency in denoising. Denoising is performed by DRUNet[6], both by true sigma-map and estimated sigma-map by SDNet.
Flow chart of denoising by true and estimated sigma-map and DRUNet. σav = 15
Flow chart of denoising by true and estimated sigma-map and DRUNet. σav = 30
Conventionl denoisers that remove AWGN can also utilize sigma-maps to perform denoising, e.g. BM3D[5]. For denoising, we divide noisy image to sigma map and result image is called equalized image, which is believed to have AWGN distbution. Therefore, BM3D can easily remove equalized image noise and after de-equalization, denoised image is achieved.
Flow chart of denoising by sigma-map and BM3D
| Noisy image (rgb) | Ground truth sigma map | Estimated sigma map by SDNet εm = 0.0402 |
|---|---|---|
 |
 |
 |
| Noisy image (gray scale) | Ground truth sigma map | Estimated sigma map by SDNet εm = 0.0821 |
|---|---|---|
 |
 |
 |
Results of SDNet on estimation of real-life images are illustrated in below.
Noisy image here is a blue channel of mosaic Bayer CFA image (RAW image produced by Canon EOS 250D camera).
Since camera parameters (we have calibrated the camera) and shooting parameters (ISO) are known, we have the ground truth sigma map for image. Variance for Poisson noise is equal to ground truth pixel value, but ground truth pixel values are not available and we only have noisy pixel values. Therefore, ground truth sigma map is an approximate noisy ground truth sigma map (calculated based on noisy pixels).
| Noisy image | Ground truth sigma map | Estimated sigma map by SDNet |
|---|---|---|
 |
 |
 |
 |
 |
 |
| Ground truth image | Noisy image , PSNR = 24.9 |
|---|---|
 |
 |
| Denoised image by CBDNet , PSNR = 32.57 | Denoised image by VDNet , PSNR = 34.79 |
 |
 |
| Denoised image by SDNet+DRUNet , PSNR = 34.97 | - |
 |
Tampere17 image set of noise-free images is available here
Tampere21 image set of noise-free images is available here
[1] V. V. Lukin, D. V. Fevralev, N. N. Ponomarenko, S. K. Abramov, O. B. Pogrebnyak, K. O. Egiazarian, and J. T. Astola, “Discrete cosine transform- based local adaptive filtering of images corrupted by nonstationary noise,” Journal of Electronic Imaging, vol. 19, no. 2, p.023007, 2010.
[2] A. A. Shulev, A. Gotchev, A. Foi, and I. R. Roussev, “Threshold selection in transform-domain denoising of speckle pattern fringes,” in Holography 2005: International Conference on Holography, Optical Recording, and Processing of Information, vol. 6252. International Society for Optics and Photonics, 2006, p. 625220.
[3] S. G. Bahncmiri, M. Ponomarenko, and K. Egiazarian, “Deep convolutional autoencoder for estimation of nonstationary noise in images,” in 2019 8th European Workshop on Visual Information Processing (EUVIP). IEEE, 2019, pp. 238–243
[4] Yue, H. Yong, Q. Zhao, L. Zhang, and D. Meng, “Variational denoising network: Toward blind noise modeling and removal,” arXiv preprint arXiv:1908.11314, 2019.
[5] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-d transform-domain collaborative filtering,” IEEE Transactions on image processing, vol. 16, no. 8, pp. 2080–2095, 2007
[6] K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,”arXiv preprintarXiv:2008.13751, 2020.