TAMPERE UNIVERSITY OF TECHNOLOGY Department of Signal Processing

Abstract

We propose a novel image denoising strategy based on an enhanced sparse representation in transform-domain. The enhancement of the sparsity is achieved by grouping similar 2D image fragments (e.g. blocks) into 3D data arrays which we call "groups".

Collaborative filtering is a special procedure developed to deal with these 3D groups. We realize it using the three successive steps: 3D transformation of 3D group, shrinkage of transform spectrum, and inverse 3D transformation. The result is a 3D estimate that consists of the jointly filtered grouped image blocks. By attenuating the noise, the collaborative filtering reveals even the finest details shared by grouped blocks and at the same time it preserves the essential unique features of each individual block. The filtered blocks are then returned to their original positions.

Because these blocks are overlapping, for each pixel we obtain many different estimates which need to be combined. Aggregation is a particular averaging procedure which is exploited to take advantage of this redundancy.

A significant improvement is obtained by a specially developed collaborative Wiener filtering.

We develop algorithms based on this novel denoising strategy. The experimental results presented here demonstrate that the developed methods achieve state-of-the-art denoising performance in terms of both peak signal-to-noise ratio and subjective visual quality.

Software

Download BM3D Matlab software v4.0.2, released 30 June 2024 Download BM3D Python software v4.0.3, released 6 September 2024 also available on the Python Package Index (PyPI): pip install bm3d README changelog   Download BM4D Matlab software v4.2.4, released 30 June 2024 Download BM4D Python software v4.2.5, released 6 September 2024 also available on the Python Package Index (PyPI): pip install bm4d README changelog   Download zipped RF3D MATLAB software v1.1.1, released 29 December 2020 removal of random and fixed-pattern noise from video README Download zipped test sequences   Download zipped BM3D MATLAB software LEGACY RELEASE v2.01, released 3 October 2021 README Download zipped test images Download zipped test sequences   Download zipped BM4D MATLAB software LEGACY RELEASE v3.2, released 2 April 2015 README VST framework for Rician-distributed data   Download zipped V-BM4D MATLAB software v1.0, released 11 December 2014 README Download zipped test sequences

Application of BMxD to signal-dependent noise:

- Poisson and mixed Poisson-Gaussian (photon-limited imaging) - Rice distribution (MRI) - Clipped Poisson-Gaussian (raw data)

Results

¹ The noisy images/videos were created by adding zero-mean white Gaussian noise with the following MATLAB commands:

randn('seed', 0);
noisy = original + sigma*randn(size(original));

People

Ymir Mäkinen
Lucio Azzari
Enrique Sánchez-Monge
Matteo Maggioni
Aram Danielyan
Kostadin Dabov
Alessandro Foi
Vladimir Katkovnik
Karen Egiazarian

Related work

Compressed Sensing Image Reconstruction, Image Upsampling, and Image/Video Super-Resolution via Recursive Spatially Adaptive Filtering

A preliminary version of BM3D using exclusively the DFT

Shape-Adaptive Transforms Filtering (Pointwise SA-DCT algorithm).

Publications

Y. Mäkinen, L. Azzari, and A. Foi, “Collaborative Filtering of Correlated Noise: Exact Transform-Domain Variance for Improved Shrinkage and Patch Matching”, IEEE Trans. Image Process., vol. 29, pp. 8339-8354, 2020.  https://doi.org/10.1109/TIP.2020.3014721

Y. Mäkinen, L. Azzari, and A. Foi, “Exact Transform-Domain Noise Variance for Collaborative Filtering of Stationary Correlated Noise”, Proc. 2019 IEEE Int. Conf. Image Process. (ICIP), pp. 185-189, Taipei, Taiwan, September 22-25, 2019.  http://doi.org/10.1109/ICIP.2019.8802964

L. Azzari and A. Foi, “Collaborative Filtering based on Group Coordinates for Smoothing and Directional Sharpening”, Proc. 2015 IEEE Int. Conf. Acoustics, Speech, Signal Process. (ICASSP 2015), pp. 1573-1577, Brisbane, Australia, April 2015.  http://doi.org/10.1109/ICASSP.2015.7178235

M. Maggioni, E. Sánchez-Monge, and A. Foi, “Joint Removal of Random and Fixed-Pattern Noise through Spatiotemporal Video Filtering”, IEEE Trans. Image Process., vol. 23, no. 10, pp. 4282-4296, October 2014.  http://doi.org/10.1109/TIP.2014.2345261

M. Maggioni, V. Katkovnik, K. Egiazarian, and A. Foi, “A Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction”, IEEE Trans. Image Process., vol. 22, no. 1, pp. 119-133, January 2013.  http://doi.org/10.1109/TIP.2012.2210725

M. Maggioni, G. Boracchi, A. Foi, and K. Egiazarian, “Video Denoising, Deblocking and Enhancement Through Separable 4-D Nonlocal Spatiotemporal Transforms”, IEEE Trans. Image Process., vol. 21, no. 9, pp. 3952-3966, September 2012.  http://doi.org/10.1109/TIP.2012.2199324

A. Danielyan, V. Katkovnik, and K. Egiazarian, “BM3D Frames and Variational Image Deblurring”, IEEE Trans. Image Process., vol. 21, no. 4, pp. 1715-1728, April 2012.  http://doi.org/10.1109/TIP.2011.2176954

M. Maggioni and A. Foi, “Nonlocal transform-domain denoising of volumetric data with groupwise adaptive variance estimation”, Proc. SPIE Electronic Imaging 2012, Computational Imaging X, 8296-22, Burlingame (CA), USA, January 2012.  http://doi.org/10.1117/12.912109

M. Maggioni, G. Boracchi, A. Foi, and K. Egiazarian, “Video denoising using separable 4D nonlocal spatiotemporal transforms”, Proc. SPIE Electronic Imaging 2011, Image Processing: Algorithms and Systems IX, 7870-2, San Francisco (CA), USA, January 2011.

M. Mäkitalo and A. Foi, “Spatially adaptive alpha-rooting in BM3D sharpening”, Proc. SPIE Electronic Imaging 2011, Image Processing: Algorithms and Systems IX, 7870-39, San Francisco (CA), USA, January 2011.

A. Danielyan, A. Foi, V. Katkovnik, and K. Egiazarian, “Spatially adaptive filtering as regularization in inverse imaging: compressive sensing, upsampling, and super-resolution”, in Super-Resolution Imaging (P. Milanfar, ed.), CRC Press / Taylor & Francis, ISBN: 978-1-4398-1930-2, September 2010         Examples of super-resolution reconstruction as zipped Matlab MAT-files.

K. Dabov, Image and video restoration with nonlocal transform-domain filtering, Tampere University of Technology, Publication 909, ISBN 978-952-15-2421-9, September 2010.

A. Danielyan, A. Foi, V. Katkovnik, and K. Egiazarian, “Denoising of Multispectral Images via Nonlocal Groupwise Spectrum-PCA”, Proc. 5th European Conf. Colour in Graphics, Imaging, and Vision, 12th Int. Sym. Multispectral Colour Science, CGIV2010/MCS'10, pp. 261-266, Joensuu, Finland, June 2010.

V. Katkovnik, A. Foi, K. Egiazarian, and J. Astola, “From local kernel to nonlocal multiple-model image denoising”, Int. J. Computer Vision, vol. 86, no. 1, pp. 1-32, January 2010. doi:10.1007/s11263-009-0272-7

A. Danielyan, V. Katkovnik, and K. Egiazarian, “Image deblurring by augmented Lagrangian with BM3D frame prior”, Proc. 2010 Workshop on Information Theoretic Methods in Science and Engineering, WITMSE 2010, Tampere, Finland, August 2010.

V. Katkovnik and K. Egiazarian, “Nonlocal image deblurring: variational formulation with nonlocal collaborative l₀-norm imaging”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2009, pp. 46-55, Tuusula, Finland, August 2009.

A. Danielyan and A. Foi, “Noise variance estimation in nonlocal transform domain”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2009, Tuusula, Finland, pp. 41-45, August 2009.

A. Danielyan, M. Vehviläinen, A. Foi, V. Katkovnik, and K. Egiazarian, “Cross-color BM3D filtering of noisy raw data”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2009, Tuusula, Finland, pp. 125-129, August 2009.

V. Katkovnik “Nonlocal collaborative l₀-norm prior for image denoising”, in Festschrift in Honor of Jaakko Astola on the Occasion of his 60th Birthday, TICSP Series 47, pp. 305-319, 2009.

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “BM3D Image Denoising with Shape-Adaptive Principal Component Analysis”, Proc. Workshop on Signal Processing with Adaptive Sparse Structured Representations (SPARS'09), Saint-Malo, France, April 2009. Poster

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “A nonlocal and shape-adaptive transform-domain collaborative filtering”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2008, Lausanne, Switzerland, August 2008.

V. Katkovnik, A. Foi, K. Egiazarian, and J. Astola, “Nonparametric regression in imaging: from local kernel to multiple-model nonlocal collaborative filtering”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2008, Lausanne, Switzerland, August 2008.

G. Boracchi and A. Foi, “Multiframe raw-data denoising based on block-matching and 3-D filtering for low-light imaging and stabilization”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2008, Lausanne, Switzerland, August 2008.

A. Danielyan, A. Foi, V. Katkovnik, and K. Egiazarian, “Image and video super-resolution via spatially adaptive block-matching filtering”, Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA 2008, Lausanne, Switzerland, August 2008.

A. Danielyan, A. Foi, V. Katkovnik, and K. Egiazarian, “Image Upsampling Via Spatially Adaptive Block-Matching Filtering”, Proc. 16th European Signal Process. Conf., EUSIPCO 2008, Lausanne, Switzerland, August 2008.

K. Dabov, A. Foi, and K. Egiazarian, “Image restoration by sparse 3D transform-domain collaborative filtering,” Proc. SPIE Electronic Imaging '08, no. 6812-07, San Jose, California, USA, January 2008.

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Joint image sharpening and denoising by 3D transform-domain collaborative filtering,” Proc. 2007 Int. TICSP Workshop Spectral Meth. Multirate Signal Process., SMMSP 2007, Moscow, Russia, September 2007.

K. Dabov, A. Foi, and K. Egiazarian, “Video denoising by sparse 3D transform-domain collaborative filtering,” Proc. 15th European Signal Processing Conference, EUSIPCO 2007, Poznan, Poland, September 2007.

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Color image denoising via sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space,” Proc. IEEE Int. Conf. Image Process., ICIP 2007, San Antonio, TX, USA, September 2007.

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, August 2007.

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising with block-matching and 3D filtering,” Proc. SPIE Electronic Imaging '06, no. 6064A-30, San Jose, California, USA, January 2006.