Is saliency really captured by gradient?

Abstract

Numerous feature attribution (or saliency) measures have been proposed that utilise the gradients of the output with respect to features. Gradients in this setting unequivocally tell us about feature sensitivity by definition of the gradient, but do they really tell us about feature importance? We challenge the idea that sensitivity and importance are the same, and empirically show that gradients do not necessarily find important features that should be attributed to a models' prediction.

More information

Identifiers

ORCID for Jonathon Hare: orcid.org/0000-0003-2921-4283

Catalogue record

Export record