Limits...
Computationally efficient locally adaptive demosaicing of color filter array images using the dual-tree complex wavelet packet transform.

Aelterman J, Goossens B, De Vylder J, Pižurica A, Philips W - PLoS ONE (2013)

Bottom Line: By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way.Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost.The wavelet approach also allows for computationally effective denoising or deblurring approaches.

View Article: PubMed Central - PubMed

Affiliation: IPI-TELIN-IMINDS, Ghent University, Ghent, Belgium. jaelterm@telin.ugent.be

ABSTRACT
Most digital cameras use an array of alternating color filters to capture the varied colors in a scene with a single sensor chip. Reconstruction of a full color image from such a color mosaic is what constitutes demosaicing. In this paper, a technique is proposed that performs this demosaicing in a way that incurs a very low computational cost. This is done through a (dual-tree complex) wavelet interpretation of the demosaicing problem. By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way. Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost. The wavelet approach also allows for computationally effective denoising or deblurring approaches.

Show MeSH
From the filtered wavelet coefficients , whose support is indicated by the colored area on the left, and two initial hypotheses: either high horizontal luminance bandwidth () or high vertical luminance bandwidth () is dominant, the costs of making an “incorrect” decision  or an “unsure” decision  is indicated.The cost for a “correct” decision  is not indicated.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3643977&req=5

pone-0061846-g008: From the filtered wavelet coefficients , whose support is indicated by the colored area on the left, and two initial hypotheses: either high horizontal luminance bandwidth () or high vertical luminance bandwidth () is dominant, the costs of making an “incorrect” decision or an “unsure” decision is indicated.The cost for a “correct” decision is not indicated.

Mentions: We will now look at detecting demosaicing artifacts. In the previous section, it was remarked that, when there are no artifacts, . When artifacts occur, these coefficients are corrupted with an extra luminance term, such as in (24). The aim now is to detect which of these coefficients is the least corrupted. Since the visual quality of locally adaptive processing is sensitive to incorrect detections (see the zipper artifact in Figure 6), we will develop a Bayesian multihypothesis technique to decide which one of both coefficients is corrupted. There are two hypotheses: , which means that there is a dominant local horizontal feature (i.e. locally larger vertical luminance bandwidth) and , which means a local vertical feature. Based on those two starting hypotheses, there are three possible decisions: , which means a vertical feature is detected; , which means a horizontal feature is detected and , which means either of the previous decisions is too dangerous with respect to the cost function (this is the “unsure” decision). By introducing this third hypothesis, we can avoid visual artifacts (Figure 6) that would otherwise originate from incorrect detection of or . The Bayesian risk that is to be minimized by the decision is:(25)where is the cost of an incorrect decision (a “miss”), is the cost of an “unsure” decision and , the cost of a “correct” decision. Instead of basing the decision on the vector of all wavelet coefficients associated with a given spatial location, we base the decision on a corruption measure. In order to distinguish the (luminance) corruption from the chrominance in the analysis, we propose the use of a third level in the wavelet packet decomposition, which can be performed at no extra memory cost using the dual-tree complex wavelet transform. We apply . Reconstruction of this filtered third scale will lead to a approximate chroma-free coefficients, which we define as and . Note how this zero setting operation acts as a simple band reject filter, where the dual-tree complex wavelet transform provides an efficient way to implement it. These coefficients are subsequently used as the corruption measure, i.e. an estimate for high frequency luminance. In this framework, the decision is made that minimizes the risk (26), given the corruption measure:(26)with the observed vector of filtered coefficients at a given spatial location in the four complex wavelet trees. The hypotheses and associated cost for a “miss” decision or an “unsure” decision are visualized in Figure 8. Assuming , which means that horizontal edges and vertical edges are equally probable, (26) can be expanded intowhich can be further simplified intowhere we make use of the fact that the decision only depends on the measurement vector such thatwith . Now, we note that the decision is deterministic such that the functions are binary. The minimizer of this risk in this setting is shown in (27).(27)We now propose a Laplacian model for the statistics of filtered wavelet coefficients and :The motivation for a Laplacian model lies in the highly leptokurtic statistics of bandpass filter output when dealing with natural images. This is well known in image restoration [26]–[30]. For the specific case here, on a filtered dual-tree complex wavelet packet transform band, we illustrate the validity of this using statistics extracted from the goldhill image, shown in Figure 9. The figure shows a logarithmically plot histogram of coefficients from a natural image, along with a Laplacian fit, and a Gaussian fit. It can be seen that the Laplacian fit is indeed very accurate. The parameters and , which are related to the variance in respectively the dominant and the subordinate direction remain to be estimated. Since we have available, and our initial hypothesis model assumes a dominant direction in all scenarios, we estimate these from the sample coefficients in both the horizontal and vertical direction in an maximum likelihood sense. The largest of these two estimates is then used as a maximum likelihood estimate for the parameter , the smallest for the parameter :For computational simplicity we assume that the four coefficients , as well as the four coefficients , are conditionally independent on the initial hypothesis. This is an effective simplification: the choice for a single , respectively parameter for the coefficients in a single filter direction reflects the filters in the four complex wavelet trees having nearly the same magnitude response. The choice for a model without correlations between the coefficients is motivated by the significant phase shift between coefficients in the different trees and computational simplicity. Using this Laplacian model, this decision rule can be effectively simplified:(28)The costs and are chosen to minimize reconstruction error, which isexplained in the Section.


Computationally efficient locally adaptive demosaicing of color filter array images using the dual-tree complex wavelet packet transform.

Aelterman J, Goossens B, De Vylder J, Pižurica A, Philips W - PLoS ONE (2013)

From the filtered wavelet coefficients , whose support is indicated by the colored area on the left, and two initial hypotheses: either high horizontal luminance bandwidth () or high vertical luminance bandwidth () is dominant, the costs of making an “incorrect” decision  or an “unsure” decision  is indicated.The cost for a “correct” decision  is not indicated.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3643977&req=5

pone-0061846-g008: From the filtered wavelet coefficients , whose support is indicated by the colored area on the left, and two initial hypotheses: either high horizontal luminance bandwidth () or high vertical luminance bandwidth () is dominant, the costs of making an “incorrect” decision or an “unsure” decision is indicated.The cost for a “correct” decision is not indicated.
Mentions: We will now look at detecting demosaicing artifacts. In the previous section, it was remarked that, when there are no artifacts, . When artifacts occur, these coefficients are corrupted with an extra luminance term, such as in (24). The aim now is to detect which of these coefficients is the least corrupted. Since the visual quality of locally adaptive processing is sensitive to incorrect detections (see the zipper artifact in Figure 6), we will develop a Bayesian multihypothesis technique to decide which one of both coefficients is corrupted. There are two hypotheses: , which means that there is a dominant local horizontal feature (i.e. locally larger vertical luminance bandwidth) and , which means a local vertical feature. Based on those two starting hypotheses, there are three possible decisions: , which means a vertical feature is detected; , which means a horizontal feature is detected and , which means either of the previous decisions is too dangerous with respect to the cost function (this is the “unsure” decision). By introducing this third hypothesis, we can avoid visual artifacts (Figure 6) that would otherwise originate from incorrect detection of or . The Bayesian risk that is to be minimized by the decision is:(25)where is the cost of an incorrect decision (a “miss”), is the cost of an “unsure” decision and , the cost of a “correct” decision. Instead of basing the decision on the vector of all wavelet coefficients associated with a given spatial location, we base the decision on a corruption measure. In order to distinguish the (luminance) corruption from the chrominance in the analysis, we propose the use of a third level in the wavelet packet decomposition, which can be performed at no extra memory cost using the dual-tree complex wavelet transform. We apply . Reconstruction of this filtered third scale will lead to a approximate chroma-free coefficients, which we define as and . Note how this zero setting operation acts as a simple band reject filter, where the dual-tree complex wavelet transform provides an efficient way to implement it. These coefficients are subsequently used as the corruption measure, i.e. an estimate for high frequency luminance. In this framework, the decision is made that minimizes the risk (26), given the corruption measure:(26)with the observed vector of filtered coefficients at a given spatial location in the four complex wavelet trees. The hypotheses and associated cost for a “miss” decision or an “unsure” decision are visualized in Figure 8. Assuming , which means that horizontal edges and vertical edges are equally probable, (26) can be expanded intowhich can be further simplified intowhere we make use of the fact that the decision only depends on the measurement vector such thatwith . Now, we note that the decision is deterministic such that the functions are binary. The minimizer of this risk in this setting is shown in (27).(27)We now propose a Laplacian model for the statistics of filtered wavelet coefficients and :The motivation for a Laplacian model lies in the highly leptokurtic statistics of bandpass filter output when dealing with natural images. This is well known in image restoration [26]–[30]. For the specific case here, on a filtered dual-tree complex wavelet packet transform band, we illustrate the validity of this using statistics extracted from the goldhill image, shown in Figure 9. The figure shows a logarithmically plot histogram of coefficients from a natural image, along with a Laplacian fit, and a Gaussian fit. It can be seen that the Laplacian fit is indeed very accurate. The parameters and , which are related to the variance in respectively the dominant and the subordinate direction remain to be estimated. Since we have available, and our initial hypothesis model assumes a dominant direction in all scenarios, we estimate these from the sample coefficients in both the horizontal and vertical direction in an maximum likelihood sense. The largest of these two estimates is then used as a maximum likelihood estimate for the parameter , the smallest for the parameter :For computational simplicity we assume that the four coefficients , as well as the four coefficients , are conditionally independent on the initial hypothesis. This is an effective simplification: the choice for a single , respectively parameter for the coefficients in a single filter direction reflects the filters in the four complex wavelet trees having nearly the same magnitude response. The choice for a model without correlations between the coefficients is motivated by the significant phase shift between coefficients in the different trees and computational simplicity. Using this Laplacian model, this decision rule can be effectively simplified:(28)The costs and are chosen to minimize reconstruction error, which isexplained in the Section.

Bottom Line: By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way.Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost.The wavelet approach also allows for computationally effective denoising or deblurring approaches.

View Article: PubMed Central - PubMed

Affiliation: IPI-TELIN-IMINDS, Ghent University, Ghent, Belgium. jaelterm@telin.ugent.be

ABSTRACT
Most digital cameras use an array of alternating color filters to capture the varied colors in a scene with a single sensor chip. Reconstruction of a full color image from such a color mosaic is what constitutes demosaicing. In this paper, a technique is proposed that performs this demosaicing in a way that incurs a very low computational cost. This is done through a (dual-tree complex) wavelet interpretation of the demosaicing problem. By using a novel locally adaptive approach for demosaicing (complex) wavelet coefficients, we show that many of the common demosaicing artifacts can be avoided in an efficient way. Results demonstrate that the proposed method is competitive with respect to the current state of the art, but incurs a lower computational cost. The wavelet approach also allows for computationally effective denoising or deblurring approaches.

Show MeSH