Limits...
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation.

Bach S, Binder A, Montavon G, Klauschen F, Müller KR, Samek W - PLoS ONE (2015)

Bottom Line: Although machine learning methods are solving very successfully a plethora of tasks, they have in most cases the disadvantage of acting as a black box, not providing any information about what made them arrive at a particular decision.These pixel contributions can be visualized as heatmaps and are provided to a human expert who can intuitively not only verify the validity of the classification decision, but also focus further analysis on regions of potential interest.We evaluate our method for classifiers trained on PASCAL VOC 2009 images, synthetic image data containing geometric shapes, the MNIST handwritten digits data set and for the pre-trained ImageNet model available as part of the Caffe open source package.

View Article: PubMed Central - PubMed

Affiliation: Machine Learning Group, Fraunhofer Heinrich Hertz Institute, Berlin, Germany; Machine Learning Group, Technische Universität Berlin, Berlin, Germany.

ABSTRACT
Understanding and interpreting classification decisions of automated image classification systems is of high value in many applications, as it allows to verify the reasoning of the system and provides additional information to the human expert. Although machine learning methods are solving very successfully a plethora of tasks, they have in most cases the disadvantage of acting as a black box, not providing any information about what made them arrive at a particular decision. This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers. We introduce a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks. These pixel contributions can be visualized as heatmaps and are provided to a human expert who can intuitively not only verify the validity of the classification decision, but also focus further analysis on regions of potential interest. We evaluate our method for classifiers trained on PASCAL VOC 2009 images, synthetic image data containing geometric shapes, the MNIST handwritten digits data set and for the pre-trained ImageNet model available as part of the Caffe open source package.

No MeSH data available.


Flipping of digit and non-digit pixels with positive responses.Pixels with highest positive scores are flipped first. The pixel-wise decomposition was computed for the true digit classes three (left) and four (right).
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4498753&req=5

pone.0130140.g019: Flipping of digit and non-digit pixels with positive responses.Pixels with highest positive scores are flipped first. The pixel-wise decomposition was computed for the true digit classes three (left) and four (right).

Mentions: Once we have established the reason why we do not use segmentation masks for evaluating the quality of pixel-wise prediction we can observe the effects of flipping the highest scoring pixels, independently of whether they are a digit or non-digit pixel. We can see from Fig 19 that the prediction of the correct digit class decreases sharply in this case as well. This decrease can be compared to the case when we flip those pixels whose absolute value of the decomposition score s is closest to zero by sorting the pixels according to 1 − ∣s∣. We see in Fig 20 that the classifier prediction for the true class decreases only very slowly, thus neutrally scored pixels are less relevant for the prediction. The comparison between Figs 19 and 20 shows that the decomposition score is able to identify those pixels which play little role in the classification of a digit and those pixels which are important for identifying a digit.


On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation.

Bach S, Binder A, Montavon G, Klauschen F, Müller KR, Samek W - PLoS ONE (2015)

Flipping of digit and non-digit pixels with positive responses.Pixels with highest positive scores are flipped first. The pixel-wise decomposition was computed for the true digit classes three (left) and four (right).
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4498753&req=5

pone.0130140.g019: Flipping of digit and non-digit pixels with positive responses.Pixels with highest positive scores are flipped first. The pixel-wise decomposition was computed for the true digit classes three (left) and four (right).
Mentions: Once we have established the reason why we do not use segmentation masks for evaluating the quality of pixel-wise prediction we can observe the effects of flipping the highest scoring pixels, independently of whether they are a digit or non-digit pixel. We can see from Fig 19 that the prediction of the correct digit class decreases sharply in this case as well. This decrease can be compared to the case when we flip those pixels whose absolute value of the decomposition score s is closest to zero by sorting the pixels according to 1 − ∣s∣. We see in Fig 20 that the classifier prediction for the true class decreases only very slowly, thus neutrally scored pixels are less relevant for the prediction. The comparison between Figs 19 and 20 shows that the decomposition score is able to identify those pixels which play little role in the classification of a digit and those pixels which are important for identifying a digit.

Bottom Line: Although machine learning methods are solving very successfully a plethora of tasks, they have in most cases the disadvantage of acting as a black box, not providing any information about what made them arrive at a particular decision.These pixel contributions can be visualized as heatmaps and are provided to a human expert who can intuitively not only verify the validity of the classification decision, but also focus further analysis on regions of potential interest.We evaluate our method for classifiers trained on PASCAL VOC 2009 images, synthetic image data containing geometric shapes, the MNIST handwritten digits data set and for the pre-trained ImageNet model available as part of the Caffe open source package.

View Article: PubMed Central - PubMed

Affiliation: Machine Learning Group, Fraunhofer Heinrich Hertz Institute, Berlin, Germany; Machine Learning Group, Technische Universität Berlin, Berlin, Germany.

ABSTRACT
Understanding and interpreting classification decisions of automated image classification systems is of high value in many applications, as it allows to verify the reasoning of the system and provides additional information to the human expert. Although machine learning methods are solving very successfully a plethora of tasks, they have in most cases the disadvantage of acting as a black box, not providing any information about what made them arrive at a particular decision. This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers. We introduce a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks. These pixel contributions can be visualized as heatmaps and are provided to a human expert who can intuitively not only verify the validity of the classification decision, but also focus further analysis on regions of potential interest. We evaluate our method for classifiers trained on PASCAL VOC 2009 images, synthetic image data containing geometric shapes, the MNIST handwritten digits data set and for the pre-trained ImageNet model available as part of the Caffe open source package.

No MeSH data available.