Limits...
Image quality assessment based on inter-patch and intra-patch similarity.

Zhou F, Lu Z, Wang C, Sun W, Xia ST, Liao Q - PLoS ONE (2015)

Bottom Line: According to local image contents, the intra-patch similarity is measured by adaptively comparing image curvature and gradient.Besides, a nonlinear integration of the inter-patch and intra-patch similarity is presented to obtain an overall score of image quality.The experiments conducted on six publicly available image databases show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

View Article: PubMed Central - PubMed

Affiliation: Department of Electronic Engineering, Tsinghua University, Beijing, 10084, China; The Shenzhen Key Laboratory of Information Science & Technology/Graduate School at Shenzhen, Tsinghua University, Shenzhen, 518055, China.

ABSTRACT
In this paper, we propose a full-reference (FR) image quality assessment (IQA) scheme, which evaluates image fidelity from two aspects: the inter-patch similarity and the intra-patch similarity. The scheme is performed in a patch-wise fashion so that a quality map can be obtained. On one hand, we investigate the disparity between one image patch and its adjacent ones. This disparity is visually described by an inter-patch feature, where the hybrid effect of luminance masking and contrast masking is taken into account. The inter-patch similarity is further measured by modifying the normalized correlation coefficient (NCC). On the other hand, we also attach importance to the impact of image contents within one patch on the IQA problem. For the intra-patch feature, we consider image curvature as an important complement of image gradient. According to local image contents, the intra-patch similarity is measured by adaptively comparing image curvature and gradient. Besides, a nonlinear integration of the inter-patch and intra-patch similarity is presented to obtain an overall score of image quality. The experiments conducted on six publicly available image databases show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

Show MeSH
Spatial relationship between a patch and its neighbors for different radius R.(A) R = 4; (B) R = 5; (C) R = 6. The center pixel of current patch is indicated by the “black” square, and center pixels of neighbors are denoted by the “grey” squares.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4368764&req=5

pone.0116312.g002: Spatial relationship between a patch and its neighbors for different radius R.(A) R = 4; (B) R = 5; (C) R = 6. The center pixel of current patch is indicated by the “black” square, and center pixels of neighbors are denoted by the “grey” squares.

Mentions: For the patch xi (denoted in lexicographic order) centered at the i-th pixel xi in the reference image Ir, its neighbors are the patches centered at adjacent pixels on a diamond of radius R using the Manhattan distance. For different radius R, the spatial relationship between a patch and its neighbors is illustrated in Fig. 2, where only the center pixels of patches are highlighted. The default value of radius R is 6 in this work, and its impact will be presented at the end of results and discussions. It is easy to find that the number of neighbors N is proportional to the radius R:N=η⋅R,(2)where η is the proportional coefficient and equals to 4 in the case of Fig. 2. The inter-patch feature is represented by a vector with size of N×1, and each element of the vector is the visual disparity between the current patch and its neighbor. For the patch xi from the reference image Ir, we denote the feature vector as vri. In the perception of visual signal, the luminance masking and contrast masking are two important features of the HVS. The former declares that the disparity in very bright areas is less likely to be visible, and the latter states that the reduction of visibility increases with the strength of the contrast masker [2]. Accordingly, the j-th (1≤ j≤ N) element of the vector vri is calculated asvri(j)=sgn(μri−μrij)⋅‖xi−xij‖22+C1M⋅max(μri2,σri2)+C1,(3)where //•//2 denotes l2-norm, sgn(•) is the signum function, M is the number of pixels in each patch, the subscript r stands for reference image, μri is the mean intensity of the pixels in the patch xi, σri is the standard deviation of the pixel intensities in xi, the patch xij is the j-th neighbor of xi, μrij is the mean intensity of the patch xij, and the constant C1 is used to avoid instability when the denominator is very small. In this work, C1 is set to M∙(K1∙L)2, where K1 = 0.01 is a small constant and L is the dynamic range of the pixel intensities. For 8-bit grayscale images, L equals to 255. By using the signum function sgn(•), vri(j) is positive if the patch xi is averagely brighter than its neighbor xij. Otherwise, vri(j) is negative.


Image quality assessment based on inter-patch and intra-patch similarity.

Zhou F, Lu Z, Wang C, Sun W, Xia ST, Liao Q - PLoS ONE (2015)

Spatial relationship between a patch and its neighbors for different radius R.(A) R = 4; (B) R = 5; (C) R = 6. The center pixel of current patch is indicated by the “black” square, and center pixels of neighbors are denoted by the “grey” squares.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4368764&req=5

pone.0116312.g002: Spatial relationship between a patch and its neighbors for different radius R.(A) R = 4; (B) R = 5; (C) R = 6. The center pixel of current patch is indicated by the “black” square, and center pixels of neighbors are denoted by the “grey” squares.
Mentions: For the patch xi (denoted in lexicographic order) centered at the i-th pixel xi in the reference image Ir, its neighbors are the patches centered at adjacent pixels on a diamond of radius R using the Manhattan distance. For different radius R, the spatial relationship between a patch and its neighbors is illustrated in Fig. 2, where only the center pixels of patches are highlighted. The default value of radius R is 6 in this work, and its impact will be presented at the end of results and discussions. It is easy to find that the number of neighbors N is proportional to the radius R:N=η⋅R,(2)where η is the proportional coefficient and equals to 4 in the case of Fig. 2. The inter-patch feature is represented by a vector with size of N×1, and each element of the vector is the visual disparity between the current patch and its neighbor. For the patch xi from the reference image Ir, we denote the feature vector as vri. In the perception of visual signal, the luminance masking and contrast masking are two important features of the HVS. The former declares that the disparity in very bright areas is less likely to be visible, and the latter states that the reduction of visibility increases with the strength of the contrast masker [2]. Accordingly, the j-th (1≤ j≤ N) element of the vector vri is calculated asvri(j)=sgn(μri−μrij)⋅‖xi−xij‖22+C1M⋅max(μri2,σri2)+C1,(3)where //•//2 denotes l2-norm, sgn(•) is the signum function, M is the number of pixels in each patch, the subscript r stands for reference image, μri is the mean intensity of the pixels in the patch xi, σri is the standard deviation of the pixel intensities in xi, the patch xij is the j-th neighbor of xi, μrij is the mean intensity of the patch xij, and the constant C1 is used to avoid instability when the denominator is very small. In this work, C1 is set to M∙(K1∙L)2, where K1 = 0.01 is a small constant and L is the dynamic range of the pixel intensities. For 8-bit grayscale images, L equals to 255. By using the signum function sgn(•), vri(j) is positive if the patch xi is averagely brighter than its neighbor xij. Otherwise, vri(j) is negative.

Bottom Line: According to local image contents, the intra-patch similarity is measured by adaptively comparing image curvature and gradient.Besides, a nonlinear integration of the inter-patch and intra-patch similarity is presented to obtain an overall score of image quality.The experiments conducted on six publicly available image databases show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

View Article: PubMed Central - PubMed

Affiliation: Department of Electronic Engineering, Tsinghua University, Beijing, 10084, China; The Shenzhen Key Laboratory of Information Science & Technology/Graduate School at Shenzhen, Tsinghua University, Shenzhen, 518055, China.

ABSTRACT
In this paper, we propose a full-reference (FR) image quality assessment (IQA) scheme, which evaluates image fidelity from two aspects: the inter-patch similarity and the intra-patch similarity. The scheme is performed in a patch-wise fashion so that a quality map can be obtained. On one hand, we investigate the disparity between one image patch and its adjacent ones. This disparity is visually described by an inter-patch feature, where the hybrid effect of luminance masking and contrast masking is taken into account. The inter-patch similarity is further measured by modifying the normalized correlation coefficient (NCC). On the other hand, we also attach importance to the impact of image contents within one patch on the IQA problem. For the intra-patch feature, we consider image curvature as an important complement of image gradient. According to local image contents, the intra-patch similarity is measured by adaptively comparing image curvature and gradient. Besides, a nonlinear integration of the inter-patch and intra-patch similarity is presented to obtain an overall score of image quality. The experiments conducted on six publicly available image databases show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

Show MeSH