Limits...
Computationally efficient locally adaptive demosaicing of color filter array images using the dual-tree complex wavelet packet transform.

Aelterman J, Goossens B, De Vylder J, Pi┼żurica A, Philips W - PLoS ONE (2013)

Bottom Line: By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way.Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost.The wavelet approach also allows for computationally effective denoising or deblurring approaches.

View Article: PubMed Central - PubMed

Affiliation: IPI-TELIN-IMINDS, Ghent University, Ghent, Belgium. jaelterm@telin.ugent.be

ABSTRACT
Most digital cameras use an array of alternating color filters to capture the varied colors in a scene with a single sensor chip. Reconstruction of a full color image from such a color mosaic is what constitutes demosaicing. In this paper, a technique is proposed that performs this demosaicing in a way that incurs a very low computational cost. This is done through a (dual-tree complex) wavelet interpretation of the demosaicing problem. By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way. Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost. The wavelet approach also allows for computationally effective denoising or deblurring approaches.

Show MeSH
Demonstration of demosaicing artifacts due to local high bandwidth.Bilinear Demosaicing (right) on the Barbara image (original version on the left). note how the local high bandwidth content of the stripes introduces discolorations in the black/white veil, indicated by the highlighted regions.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3643977&req=5

pone-0061846-g004: Demonstration of demosaicing artifacts due to local high bandwidth.Bilinear Demosaicing (right) on the Barbara image (original version on the left). note how the local high bandwidth content of the stripes introduces discolorations in the black/white veil, indicated by the highlighted regions.

Mentions: Natural images still have other characteristics that can be exploited in order to improve demosaicing results. One important characteristic is locality. A natural scene is often composed of different objects so the spectral content of the image normally changes locally across the image. Calculating the global Discrete Fourier Transform (DFT) disallows any local interpretation by averaging any local change in spectral content. As an illustration, we show the result of bilinear demosaicing of the Barbara image in Figure 4. The Barbara image is a public domain test image that is well suited for showing high spectral bandwidth content in images due to the striped cloth and texture. Note the significant demosaicing artifacts on the stripes. The reason is that locally, i.e. if one would only look at the patch of stripes, the power spectral density of these stripes has a very high bandwidth. In light of this, a global set of low-pass demosaicing filters, such as the ones for the bilinear demosaicing in Section 1.1, is a bad choice. Because this significant drawback to global processing, state-of-the-art demosaicing algorithms perform locally adaptive processing in one way or another (such as [2], [5], [7], [19]). Pixel-domain algorithms typically calculate an (sometimes elaborate) edge indicator function, which is then used to fuse multiple directional filter outputs. Several edge indicator functions are used, they are called in [19], in [2], in [7] and in [5]. In many algorithms, these indicators are used to create convex combinations of different directional estimates [2], [5], [19]. Sometimes, the edge indicators are given a statistical interpretation such as standard deviation for and in [5] and [2]. When these are subsequently used in a convex combination, the resulting algorithm implies a Gaussian MMSE estimator of unknown pixel values (in fact, this was formally shown in [2]). In order to keep computational complexity down, we will not use a convex combination to combine directional estimates in our proposed method, but we will rather switch between directional estimates using a statistics-based decision mechanism (explained in Section 2.2). Such decision mechanisms are also used in [7]. For a more thorough explanation of the aforementioned algorithms, we refer to their respective papers. In this paper, we will incorporate the idea of local adaptivity into the wavelet demosaicing framework, which will turn out to be very elegant. For this, we first establish that wavelets can indeed be used for demosaicing.


Computationally efficient locally adaptive demosaicing of color filter array images using the dual-tree complex wavelet packet transform.

Aelterman J, Goossens B, De Vylder J, Pi┼żurica A, Philips W - PLoS ONE (2013)

Demonstration of demosaicing artifacts due to local high bandwidth.Bilinear Demosaicing (right) on the Barbara image (original version on the left). note how the local high bandwidth content of the stripes introduces discolorations in the black/white veil, indicated by the highlighted regions.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3643977&req=5

pone-0061846-g004: Demonstration of demosaicing artifacts due to local high bandwidth.Bilinear Demosaicing (right) on the Barbara image (original version on the left). note how the local high bandwidth content of the stripes introduces discolorations in the black/white veil, indicated by the highlighted regions.
Mentions: Natural images still have other characteristics that can be exploited in order to improve demosaicing results. One important characteristic is locality. A natural scene is often composed of different objects so the spectral content of the image normally changes locally across the image. Calculating the global Discrete Fourier Transform (DFT) disallows any local interpretation by averaging any local change in spectral content. As an illustration, we show the result of bilinear demosaicing of the Barbara image in Figure 4. The Barbara image is a public domain test image that is well suited for showing high spectral bandwidth content in images due to the striped cloth and texture. Note the significant demosaicing artifacts on the stripes. The reason is that locally, i.e. if one would only look at the patch of stripes, the power spectral density of these stripes has a very high bandwidth. In light of this, a global set of low-pass demosaicing filters, such as the ones for the bilinear demosaicing in Section 1.1, is a bad choice. Because this significant drawback to global processing, state-of-the-art demosaicing algorithms perform locally adaptive processing in one way or another (such as [2], [5], [7], [19]). Pixel-domain algorithms typically calculate an (sometimes elaborate) edge indicator function, which is then used to fuse multiple directional filter outputs. Several edge indicator functions are used, they are called in [19], in [2], in [7] and in [5]. In many algorithms, these indicators are used to create convex combinations of different directional estimates [2], [5], [19]. Sometimes, the edge indicators are given a statistical interpretation such as standard deviation for and in [5] and [2]. When these are subsequently used in a convex combination, the resulting algorithm implies a Gaussian MMSE estimator of unknown pixel values (in fact, this was formally shown in [2]). In order to keep computational complexity down, we will not use a convex combination to combine directional estimates in our proposed method, but we will rather switch between directional estimates using a statistics-based decision mechanism (explained in Section 2.2). Such decision mechanisms are also used in [7]. For a more thorough explanation of the aforementioned algorithms, we refer to their respective papers. In this paper, we will incorporate the idea of local adaptivity into the wavelet demosaicing framework, which will turn out to be very elegant. For this, we first establish that wavelets can indeed be used for demosaicing.

Bottom Line: By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way.Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost.The wavelet approach also allows for computationally effective denoising or deblurring approaches.

View Article: PubMed Central - PubMed

Affiliation: IPI-TELIN-IMINDS, Ghent University, Ghent, Belgium. jaelterm@telin.ugent.be

ABSTRACT
Most digital cameras use an array of alternating color filters to capture the varied colors in a scene with a single sensor chip. Reconstruction of a full color image from such a color mosaic is what constitutes demosaicing. In this paper, a technique is proposed that performs this demosaicing in a way that incurs a very low computational cost. This is done through a (dual-tree complex) wavelet interpretation of the demosaicing problem. By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way. Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost. The wavelet approach also allows for computationally effective denoising or deblurring approaches.

Show MeSH